Data quality detection method and system applying deep learning

Abstract

The embodiment of the invention relates to the technical field of data processing, in particular to a data quality detection method and system applying deep learning, which can efficiently process large-scale marine environment monitoring data and accurately detect quality problems in the data through an automatic data processing and analysis process. By constructing a special monitoring data analysis network, deep knowledge vector mining is performed on original marine environment sensing monitoring sample data and augmented data, so that key features reflecting environment quality are extracted, data processing efficiency is improved, and data analysis comprehensiveness and reliability are enhanced. And a solid foundation is provided for subsequent network optimization by accurately determining the target network debugging error of the monitoring data analysis network. By improving the network configuration variables, the data quality detection network can quickly identify and reject low-quality data in practical application, so that the credibility and effectiveness of the marine environment monitoring data are greatly improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a data quality detection method and system using deep learning. Background Art

[0002] In the field of marine environmental monitoring, data accuracy and quality are of vital importance. However, traditional data quality detection methods often rely on manual analysis and judgment, which is not only inefficient but also easily affected by subjective factors, resulting in instability and unreliability of data quality detection results. At the same time, with the continuous increase and complexity of marine environmental monitoring data, traditional methods have been unable to cope with the processing needs of large-scale data. Summary of the invention

[0003] In view of this, the present application provides a data quality detection method and system using deep learning.

[0004] The present application provides a data quality detection method using deep learning, which is applied to a data quality detection system. The method includes: Acquire at least one marine environment sensor monitoring sample data in a first sensor monitoring sample data set included in the sample data set, and acquire at least one marine environment sample augmented data in a second sensor monitoring sample data set included in the sample data set, wherein the number of marine environment sensor monitoring sample data in the first sensor monitoring sample data set is greater than the number of marine environment sample augmented data in the second sensor monitoring sample data set; Through the monitoring data analysis network, knowledge vector mining is performed on each of the at least one marine environment sensor monitoring sample data to obtain the environmental monitoring quality knowledge vector of each of the marine environment sensor monitoring sample data; and through the monitoring data analysis network, knowledge vector mining is performed on each of the at least one marine environment sample augmented data to obtain the environmental monitoring quality knowledge vector of each of the marine environment sample augmented data; Determining a target network debugging error of the monitoring data analysis network according to the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the environmental monitoring quality knowledge vector of each marine environment sample augmented data; Based on the rule of converging the debugging error of the target network, the network configuration variables in the monitoring data analysis network are improved to obtain a data quality detection network, and the data quality detection network is used to perform data quality detection on the marine environment sample augmented data.

[0005] In some technical solutions, determining the target network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the environmental monitoring quality knowledge vector of each marine environment sample augmented data includes: Determining a first network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data, wherein the prior training annotation of any marine environment sensor monitoring sample data is used to characterize the marine environment area corresponding to the any marine environment sensor monitoring sample data; Determining a second network debugging error of the monitoring data analysis network according to the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the prior training annotation of each marine environment sample augmented data, wherein the prior training annotation of any marine environment sample augmented data is used to characterize the associated environmental area corresponding to any marine environment sample augmented data; The first network debugging error and the second network debugging error are integrated to obtain a target network debugging error of the monitoring data analysis network.

[0006] In some technical solutions, the method further includes: Acquire at least one target environment sensor monitoring sample data in the sample data set, wherein the at least one target environment sensor monitoring sample data includes at least one target marine environment sample augmented data, or at least one target marine environment sensor monitoring sample data and at least one target marine environment sample augmented data; Through the monitoring data analysis network, knowledge vector mining is performed on each target marine environment sample augmented data in the at least one target environment sensor monitoring sample data to obtain an environmental monitoring quality knowledge vector of each target marine environment sample augmented data; Determining the target network debugging error of the monitoring data analysis network according to the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the environmental monitoring quality knowledge vector of each marine environment sample augmented data includes: The target network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data, and the environmental monitoring quality knowledge vector of each target marine environment sample augmented data.

[0007] In some technical solutions, determining the target network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data, and the environmental monitoring quality knowledge vector of each target marine environment sample augmented data includes: Determining a first network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data, wherein the prior training annotation of any marine environment sensor monitoring sample data is used to characterize the marine environment area corresponding to the any marine environment sensor monitoring sample data; Determining a second network debugging error of the monitoring data analysis network according to the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the prior training annotation of each marine environment sample augmented data, wherein the prior training annotation of any marine environment sample augmented data is used to characterize the associated environmental area corresponding to any marine environment sample augmented data; Determine a third network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each target marine environment sample augmented data and the prior training annotation of each target marine environment sample augmented data, wherein the prior training annotation of any target marine environment sample augmented data is used to characterize the associated environmental area corresponding to any target marine environment sample augmented data; The first network debugging error, the second network debugging error and the third network debugging error are integrated to obtain a target network debugging error of the monitoring data analysis network.

[0008] In some technical solutions, the monitoring data analysis network includes a first sub-cluster confidence of the marine environment area corresponding to each marine environment sensor monitoring sample data in the first sensor monitoring sample data set, and a second sub-cluster confidence of the associated environment area corresponding to each marine environment sample augmented data in the second sensor monitoring sample data set; the third network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each target marine environment sample augmented data and the prior training annotation of each target marine environment sample augmented data, including: Swimming the target ocean environment sample augmented data in the at least one target environment sensor monitoring sample data, and integrating the wandering target ocean environment sample augmented data with each first sub-cluster confidence and each second sub-cluster confidence, respectively, to obtain the possibility that the wandering target ocean environment sample augmented data belongs to the ocean environment area corresponding to each first sub-cluster confidence, and the possibility that the wandering target ocean environment sample augmented data belongs to the associated environment area corresponding to each second sub-cluster confidence; After walking through the target ocean environment sample augmented data in the at least one target environment sensor monitoring sample data, the third network debugging error of the monitoring data analysis network is determined based on the possibility of each target ocean environment sample augmented data and the prior training annotation of each target ocean environment sample augmented data.

[0009] In some technical solutions, determining the first network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data includes: Wandering the at least one marine environment sensor monitoring sample data, and integrating the environmental monitoring quality knowledge vector of the wandering marine environment sensor monitoring sample data and each first sub-cluster confidence, to obtain the possibility that the wandering marine environment sensor monitoring sample data belongs to the marine environment area corresponding to each first sub-cluster confidence; after wandering the at least one marine environment sensor monitoring sample data, determining the first network debugging error of the monitoring data analysis network according to the possibility of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data; The step of determining the second network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the prior training annotation of each marine environment sample augmented data comprises: Wander the at least one marine environment sample augmented data, and integrate the environmental monitoring quality knowledge vector of the wandered marine environment sample augmented data and each second sub-cluster confidence to obtain the possibility that the wandered marine environment sample augmented data belongs to the associated environmental area corresponding to each second sub-cluster confidence; after wandering the at least one marine environment sample augmented data, determine the second network debugging error of the monitoring data analysis network according to the possibility of each marine environment sample augmented data and the prior training annotation of each marine environment sample augmented data.

[0010] In some technical solutions, the at least one target environment sensor monitoring sample data includes at least one target marine environment sensor monitoring sample data, or at least one target marine environment sample augmented data, or at least one target marine environment sensor monitoring sample data and at least one target marine environment sample augmented data; the monitoring data analysis network is used to perform knowledge vector mining on each target marine environment sample augmented data in the at least one target environment sensor monitoring sample data, and the environmental monitoring quality knowledge vector of each target marine environment sample augmented data is obtained, including: Through the monitoring data analysis network, knowledge vector mining is performed on each target environment sensor monitoring sample data in the at least one target environment sensor monitoring sample data to obtain the environmental monitoring quality knowledge vector of each target environment sensor monitoring sample data; The step of determining the target network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data, and the environmental monitoring quality knowledge vector of each target marine environment sample augmented data comprises: The target network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data, and the environmental monitoring quality knowledge vector of each target environment sensor monitoring sample data.

[0011] In some technical solutions, the monitoring data analysis network includes a knowledge vector mining branch, and the knowledge vector mining branch is used to perform knowledge vector mining on each marine environment sensor monitoring data; the network configuration variables in the monitoring data analysis network are improved based on the rule of converging the target network debugging error to obtain a data quality detection network, including: Based on the rule of converging the debugging error of the target network, the vector mining indication parameter of the knowledge vector mining branch is improved to obtain a data quality detection network; wherein the data quality detection network includes the knowledge vector mining branch after the vector mining indication parameter is improved.

[0012] In some technical solutions, the method further includes: Filtering at least one marine environment sensor monitoring sample data from the first sensor monitoring sample data set included in the sample data set; Through the basic monitoring data analysis network, knowledge vector mining is performed on each of the at least one selected marine environment sensor monitoring sample data to obtain the environmental monitoring quality knowledge vector of each of the selected marine environment sensor monitoring sample data; Determining a pre-debugging network debugging error of the basic monitoring data analysis network based on the environmental monitoring quality knowledge vector of each of the screened marine environmental sensor monitoring sample data and the prior training annotation of each of the screened marine environmental sensor monitoring sample data; Based on the rule of converging the debugging error of the pre-debugging network, the network configuration variables in the basic monitoring data analysis network are improved to obtain the monitoring data analysis network.

[0013] The present application also provides a data quality detection system, comprising: a memory for storing program instructions and data; a processor for coupling with the memory and executing instructions in the memory to implement the above method.

[0014] The present application also provides a computer storage medium, comprising instructions, which implement the above method when executed on a processor.

[0015] The embodiment of the present application can efficiently process large-scale marine environment monitoring data and accurately detect quality problems in the data through automated data processing and analysis processes. In detail, by constructing a special monitoring data analysis network, the original marine environment sensor monitoring sample data and augmented data are deeply mined with knowledge vectors, thereby extracting key features reflecting environmental quality. In this process, not only the efficiency of data processing is improved, but also the comprehensiveness and reliability of data analysis are enhanced. In addition, by accurately determining the target network debugging error of the monitoring data analysis network, a solid foundation is provided for subsequent network optimization. By improving the network configuration variables, an efficient and accurate data quality detection network is finally obtained. The data quality detection network can quickly identify and eliminate low-quality data in practical applications, thereby greatly improving the credibility and effectiveness of marine environment monitoring data.

[0016] Therefore, the technical solution of the embodiment of the present application aims to solve the problems existing in the traditional data quality detection method and provide a more efficient and accurate data analysis tool for the field of marine environment monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, are used to explain the principles of the present application.

[0018] Figure 1 A schematic diagram of the steps of a data quality detection method using deep learning provided in an embodiment of the present application.

[0019] Figure 2 A structural block diagram of a data quality detection system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0021] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application. It should be noted that the terms "first", "second", etc. in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0022] The method embodiments provided in the embodiments of the present application can be executed in a data quality detection system, a computer device or a similar computing device. Taking running on a data quality detection system as an example, the data quality detection system may include one or more processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory for storing data. Optionally, the above-mentioned data quality detection system may also include a transmission device for communication functions. It can be understood by those of ordinary skill in the art that the above-mentioned structure is only illustrative and does not limit the structure of the above-mentioned data quality detection system. For example, the data quality detection system may also include more or fewer components than those shown above, or have a configuration different from that shown above.

[0023] The memory can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a data quality detection method for applying deep learning in an embodiment of the present application, and the processor executes various functional applications and data processing by running the computer program stored in the memory, that is, implementing the above method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the data quality detection system via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0024] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the data quality detection system. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0025] In conjunction with the above, please refer to Figure 1 , Figure 1 It is a flow chart of a data quality detection method using deep learning provided in an embodiment of the present application. The method is applied to a data quality detection system and may further include steps 202 to 208.

[0026] Step 202: obtain at least one marine environment sensor monitoring sample data in a first sensor monitoring sample data set included in the sample data set, and obtain at least one marine environment sample augmented data in a second sensor monitoring sample data set included in the sample data set, wherein the number of marine environment sensor monitoring sample data in the first sensor monitoring sample data set is greater than the number of marine environment sample augmented data in the second sensor monitoring sample data set.

[0027] Step 204: Through the monitoring data analysis network, knowledge vector mining is performed on each of the at least one marine environment sensor monitoring sample data to obtain the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data; and through the monitoring data analysis network, knowledge vector mining is performed on each of the at least one marine environment sample augmented data to obtain the environmental monitoring quality knowledge vector of each marine environment sample augmented data.

[0028] Step 206: Determine the target network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the environmental monitoring quality knowledge vector of each marine environment sample augmented data.

[0029] Step 208: Based on the rule of converging the target network debugging error, improve the network configuration variables in the monitoring data analysis network to obtain a data quality detection network, and the data quality detection network is used to perform data quality detection on the marine environment sample augmented data.

[0030] In an exemplary application scenario, a marine environment monitoring project is involved, which aims to collect marine environment data through sensors and use these data for quality detection and analysis. The data quality detection system plays a key role in this process.

[0031] The data quality detection system first obtains data from two sensor monitoring sample data sets: the first sensor monitoring sample data set and the second sensor monitoring sample data set. The first sensor monitoring sample data set contains a large amount of marine environment sensor monitoring sample data, which is collected in real time by sensors deployed in the ocean, covering multiple environmental parameters such as temperature, salinity, dissolved oxygen, etc. The second sensor monitoring sample data set contains a small amount of marine environment sample augmented data, which can be data processed by certain augmentation techniques (such as data interpolation, noise addition, etc.) to enrich the training set of the model and improve the generalization ability of the model. Importantly, the amount of data in the first data set is significantly larger than that in the second data set. This is because the original sensor monitoring data is easier to obtain, while the augmented data may require additional processing steps.

[0032] Next, the data quality detection system uses a pre-built monitoring data analysis network to perform knowledge vector mining on the data in the two data sets. This process aims to extract meaningful feature representations, namely environmental monitoring quality knowledge vectors, from the original data. These vectors not only contain the original information of the data, but also reflect the inherent connections and laws between the data. For each marine environment sensor monitoring sample data and marine environment sample augmented data, the data quality detection system will generate the corresponding environmental monitoring quality knowledge vector through the monitoring data analysis network.

[0033] After obtaining these knowledge vectors, the data quality detection system will determine the target network debugging error of the monitoring data analysis network based on the difference between the vectors. The target network debugging error reflects the accuracy of the network in extracting data features. The smaller the error, the stronger the network's ability to understand the data. The system compares the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the environmental monitoring quality knowledge vector of each marine environment sample augmented data, calculates the difference between the two, and determines the target network debugging error accordingly.

[0034] Finally, the data quality detection system will improve the network configuration variables in the monitoring data analysis network based on the rules of converging the target network debugging error. This process aims to optimize the performance of the network to make it more accurate and efficient in future data quality detection. By continuously adjusting the parameters and structure of the network, the target network debugging error can eventually obtain an optimized data quality detection network. The data quality detection network can not only perform high-quality data detection on the marine environment sample augmented data, but also provide strong support for subsequent data analysis and decision-making.

[0035] Based on the above content, the data quality detection function of the data quality detection network is mainly reflected in the following aspects: (1) Feature extraction and representation: The data quality detection network first extracts meaningful feature representations, namely environmental monitoring quality knowledge vectors, from the original marine environment sensor monitoring sample data and the marine environment sample augmented data through knowledge vector mining. These vectors can effectively capture the core information of the data and provide a basis for subsequent data quality detection; (2) Error determination and optimization: By comparing the environmental monitoring quality knowledge vectors extracted from the two data sets, the data quality detection network can determine the target network debugging error. This error reflects the accuracy of the network in extracting and processing data features. Based on this error, the network can continuously optimize and adjust itself to improve its understanding of data features and processing accuracy; (3) Data quality detection: The optimized data quality detection network has the ability to perform high-quality detection of marine environmental data. It can accurately identify outliers, noise, or unexpected data changes in the data, thereby ensuring the accuracy and reliability of the data. This detection function is crucial for subsequent marine environmental analysis and decision-making because it can prevent erroneous or low-quality data from entering the analysis process; (4) Augmented data verification: For the augmented data in the second sensor monitoring sample data set, the data quality detection network can also perform effective quality verification. By comparing the knowledge vectors of the original data and the augmented data, the network can evaluate the effect of the augmentation technology and ensure that these technologies do not introduce too much noise or bias, thereby maintaining the authenticity and usability of the data; (5) Continuous learning and improvement: The data quality detection network can not only perform quality detection on the current data, but also has the ability to continuously learn and improve. As new data is continuously input, the network can adapt to new data features through self-learning and adjustment, thereby continuously improving its data quality detection performance and accuracy.

[0036] Therefore, the data quality detection function of the data quality detection network is a comprehensive process, which covers multiple aspects such as feature extraction, error determination and optimization, data quality verification, and continuous learning and improvement. These functions together ensure the accuracy and reliability of marine environmental data and provide a solid foundation for subsequent marine environmental monitoring, analysis and decision-making.

[0037] Based on the above content, steps 202 to 208 are respectively introduced and explained by way of example, so that those skilled in the art know how to clearly and completely implement the embodiments of the present application.

[0038] First, in step 202, the sample data set refers to a data set used for analysis and processing by the data quality detection system. These data can be collected from the actual environment and used to train, verify or test the effectiveness and accuracy of the data quality detection network. The sample data set may contain multiple types of data, such as ocean environment parameters such as temperature, salinity, dissolved oxygen, etc., which are crucial for understanding and evaluating the state of the ocean environment.

[0039] The first sensor monitoring sample dataset refers to a dataset composed of a large amount of raw data collected in real time by sensors deployed in the ocean. These data directly reflect the real state of the marine environment without any augmentation or modification. The first sensor monitoring sample dataset contains rich environmental monitoring information and is the basis for the data quality detection system to analyze. Due to its large amount of data and strong authenticity, it plays a key role in training and optimizing the data quality detection network.

[0040] The marine environment sensor monitoring sample data refers to the specific data points obtained from the first sensor monitoring sample data set. Each data point represents the value of a marine environment parameter at a specific time point and location, such as temperature, salinity, etc. These data are directly monitored by sensors and are highly real-time and accurate, which is of great significance for evaluating the quality of the marine environment.

[0041] The second sensor monitoring sample dataset refers to a dataset containing a smaller amount of data, in which the data are marine environment samples processed by certain augmentation techniques. Augmentation techniques may include data interpolation, noise addition, etc., which aim to enrich the training set of the model and improve its generalization ability. Compared with the first sensor monitoring sample dataset, the second sensor monitoring sample dataset has a smaller amount of data, but it is still very important for enhancing the robustness and adaptability of the data quality detection network.

[0042] Marine environment sample augmented data refers to marine environment data samples processed by augmentation technology. In the second sensor monitoring sample data set, these data are transformed or expanded by specific augmentation methods (such as adding noise, data interpolation, etc.) to simulate more diverse marine environment conditions. The introduction of augmented data helps to improve the adaptability of the data quality detection network to various complex environments.

[0043] In detail, the data quality detection system first accesses the sample data set stored in it based on step 202. This sample data set actually contains two sub-data sets: a first sensor monitoring sample data set and a second sensor monitoring sample data set. The system obtains at least one marine environment sensor monitoring sample data from the first sensor monitoring sample data set. These data are collected in real time by sensors in the ocean, have high accuracy and real-time performance, and can truly reflect the status of the marine environment. At the same time, the system also obtains at least one marine environment sample augmented data from the second sensor monitoring sample data set. These data are processed by augmentation technology. Although the amount of data is relatively small, it plays an important role in improving the generalization ability and robustness of the data quality detection network. It is worth noting that the amount of data in the first sensor monitoring sample data set is significantly larger than that in the second sensor monitoring sample data set. This is because the original sensor monitoring data (i.e., the data in the first data set) is easier to obtain, and they come directly from the real-time monitoring of the sensor. The augmented data (i.e., the data in the second data set) needs to go through additional processing steps, such as data interpolation, noise addition, etc., to simulate more diverse marine environmental conditions, so the number is relatively small. This step is the basis for subsequent analysis by the data quality detection system. It ensures that the system can obtain sufficient and diverse data to accurately assess the quality status of the marine environment.

[0044] Secondly, in step 204, the monitoring data analysis network is a specially designed deep learning model or a collection of machine learning algorithms for processing and analyzing marine environmental monitoring data. This network is able to receive raw sensor data as input and extract key information and features from the data through its complex hierarchical structure and algorithms. The core function of the monitoring data analysis network is to perform in-depth processing and analysis of the data to draw important conclusions and predictions about the quality of the marine environment.

[0045] Knowledge vector mining is a process of extracting feature vectors from raw data using a monitoring data analysis network. In this process, the network transforms complex marine environmental data into concise, representative numerical feature vectors, i.e., knowledge vectors, by learning and identifying patterns, trends, and correlations in the data. These knowledge vectors not only contain key information about the raw data, but can also be used more efficiently for subsequent data analysis, comparison, and decision-making.

[0046] Environmental monitoring quality knowledge vectors are specific numerical feature vectors obtained through knowledge vector mining through monitoring data analysis networks. These vectors can be represented in the form of a series of numerical values, each of which corresponds to a specific feature or attribute in the data, such as temperature, salinity, dissolved oxygen content, etc. These feature vectors not only reflect the real-time status of the marine environment, but can also be used to assess environmental quality, predict environmental change trends, and promptly identify potential environmental problems. For example, an environmental monitoring quality knowledge vector may contain data in multiple dimensions, such as [temperature value, salinity value, dissolved oxygen content value, ...], which together constitute a comprehensive description of the quality of the marine environment.

[0047] In detail, the data quality detection system processes at least one marine environment sensor monitoring sample data obtained from the first sensor monitoring sample data set using the pre-built monitoring data analysis network based on step 204. For example, the system inputs each marine environment sensor monitoring sample data into the monitoring data analysis network respectively, and performs knowledge vector mining through the complex calculation and feature extraction process inside the network. This mining process aims to extract a numerical vector that can represent its essential characteristics and environmental quality from the original data, that is, the environmental monitoring quality knowledge vector. Similarly, the system also performs the same operation on at least one marine environment sample augmented data obtained from the second sensor monitoring sample data set. Although these augmented data have undergone certain transformations or expansions, they also need to be mined through the monitoring data analysis network to obtain their respective corresponding environmental monitoring quality knowledge vectors. The output of this step is a series of environmental monitoring quality knowledge vectors, which not only contain the core information of the original data, but also provide an important data basis for subsequent data quality evaluation, determination of network debugging errors, and optimization of data quality detection networks. By comparing and analyzing these vectors, the system can more accurately assess the quality of the marine environment, promptly identify potential environmental problems, and take appropriate measures to address them.

[0048] Then, in step 206, the target network debugging error is a key indicator for measuring the difference between the monitoring data analysis network and the expected results when processing marine environmental data. This error is calculated by comparing the difference between the environmental monitoring quality knowledge vector output by the network and the actual marine environmental data. The size of the target network debugging error directly reflects the accuracy of the network in extracting data features and its ability to assess environmental quality. A smaller target network debugging error means that the output result of the network is closer to the actual situation, indicating that the network has a strong ability to parse and process data. On the contrary, if the error is large, it means that the network has certain deviations or deficiencies when processing data, and corresponding adjustments and optimizations are required.

[0049] In detail, the core task of the data quality detection system based on step 206 is to determine the target network debugging error of the monitoring data analysis network. The system first obtains the environmental monitoring quality knowledge vectors of each marine environment sensor monitoring sample data, which are obtained by deep processing and analysis of the original sensor data by the monitoring data analysis network. At the same time, the system also obtains the environmental monitoring quality knowledge vectors of each marine environment sample augmented data, which are also obtained by processing the augmented data through the network. Next, the system compares the two sets of environmental monitoring quality knowledge vectors. By calculating the difference between the two sets of vectors, such as using metrics such as Euclidean distance and cosine similarity, the system can quantify the performance difference of the monitoring data analysis network when processing different types of data (original data and augmented data). This difference reflects the accuracy of the network in extracting data features and understanding environmental quality. Finally, the system determines the target network debugging error of the monitoring data analysis network based on these differences. This error value is not only a specific value, but also a comprehensive evaluation of network performance. By analyzing and interpreting this error, the system can understand in which aspects the network performs well and which aspects need to be improved, thereby providing a clear direction and goal for subsequent network optimization. In general, step 206 is a crucial link in the data quality detection system. It connects the two links of data input and network optimization, and provides strong support for improving the performance and accuracy of the entire system.

[0050] Finally, in step 208, network configuration variables refer to parameters and settings that can be adjusted when building and training a deep learning model or neural network. These variables include, but are not limited to, learning rate, batch size, number of iterations, number of hidden layers, number of neurons per layer, choice of activation function, etc. In the context of the marine environment monitoring data analysis network, network configuration variables are key factors in determining how the network learns and processes input data, and by optimizing these variables, the performance, accuracy, and generalization ability of the network can be improved.

[0051] The data quality detection network is a specially trained and optimized neural network used to evaluate the quality of marine environmental sample augmented data. This network can determine whether the data is true, accurate and complete by learning and identifying features in the data, thereby helping researchers identify and eliminate low-quality or abnormal data. The data quality detection network is particularly important in marine environmental monitoring because it can ensure the reliability of the data used for analysis, thereby improving the accuracy and effectiveness of the entire monitoring system.

[0052] In detail, after determining the target network debugging error of the monitoring data analysis network, the next key step (step 208) is to improve the network configuration variables based on this error to obtain a more accurate and efficient data quality detection network. First, the system analyzes the source of the target network debugging error and identifies the main factors that cause the error. This may include insufficient network structure, improper setting of learning rate, inappropriate selection of activation function, etc. Through in-depth analysis of these factors, a clear direction can be provided for subsequent network improvements. Next, the system will proceed to adjust the network configuration variables. This may include increasing or decreasing the number of hidden layers, adjusting the number of neurons in each layer, changing the activation function, optimizing the learning rate, etc. These adjustments are intended to make the network better adapt to the input data and improve the accuracy of feature extraction and pattern recognition. In the process of adjusting the network configuration variables, the system will continuously evaluate the changes in network performance. This can be achieved through cross-validation, test set evaluation, etc. to ensure that the network improvement is effective and no new errors or deviations are introduced. Finally, through multiple iterations and optimizations, the system will obtain an improved data quality detection network. This network not only has stronger feature extraction and data processing capabilities, but also can more accurately evaluate the quality of marine environmental sample augmented data. In practical applications, this data quality detection network will be used to perform real-time quality detection and evaluation of marine environmental data, thereby ensuring the accuracy and reliability of subsequent analysis. It can be seen that step 208 is an iterative and optimization process that aims to improve the performance and accuracy of the data quality detection network by adjusting network configuration variables. This process is crucial to ensuring the reliability and effectiveness of marine environmental monitoring data.

[0053] The embodiment of the present application significantly improves the quality detection efficiency and accuracy of marine environment monitoring data through an innovative data processing and analysis method. First, through a carefully constructed monitoring data analysis network, the embodiment of the present application can perform in-depth knowledge vector mining on the original marine environment sensor monitoring sample data and augmented data, thereby accurately extracting key features that reflect environmental quality. In this process, the network not only processes a large amount of sensor monitoring data, but also enhances the comprehensiveness and reliability of data analysis by comparing the original data with the augmented data.

[0054] More importantly, the embodiment of the present application utilizes these excavated environmental monitoring quality knowledge vectors to accurately determine the target network debugging error of the monitoring data analysis network. The accurate definition of this error provides a solid foundation for subsequent network optimization. By using the rules based on the convergence target network debugging error, the embodiment of the present application further improves the network configuration variables in the monitoring data analysis network, and finally obtains an efficient and accurate data quality detection network. This network can not only perform accurate quality detection on the marine environment sample augmented data, but also quickly identify and eliminate low-quality data in practical applications, thereby greatly improving the credibility and effectiveness of marine environment monitoring data.

[0055] In summary, the embodiments of the present application not only improve the automation level of data quality detection through a series of precise data processing and analysis steps, but also bring more efficient and accurate data analysis tools to the field of marine environment monitoring, which has important practical application value and promotion significance.

[0056] In an optional embodiment, the target network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the environmental monitoring quality knowledge vector of each marine environment sample augmented data, including: determining a first network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data, the prior training annotation of any marine environment sensor monitoring sample data being used to characterize the marine environment area corresponding to any marine environment sensor monitoring sample data; determining a second network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the prior training annotation of each marine environment sample augmented data, the prior training annotation of any marine environment sample augmented data being used to characterize the associated environment area corresponding to any marine environment sample augmented data; integrating the first network debugging error and the second network debugging error to obtain the target network debugging error of the monitoring data analysis network.

[0057] Based on this embodiment, the data quality detection system adopts a more refined method when determining the target network debugging error of the monitoring data analysis network. This process mainly includes the following steps: First, the system determines the first network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the prior training annotations of these sample data. The prior training annotations here are used to characterize the specific marine environment area corresponding to each marine environment sensor monitoring sample data. By comparing the environmental monitoring quality knowledge vectors of these sample data with their corresponding prior training annotations, the system can quantify the error of the network when processing these data, that is, the first network debugging error.

[0058] Secondly, the system will also determine the second network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environmental sample augmented data and the prior training annotations of these augmented data. Similar to the sensor monitoring sample data, the prior training annotations of the augmented data are also used to characterize the associated environmental area corresponding to each augmented data. By comparing the environmental monitoring quality knowledge vector of the augmented data with its corresponding prior training annotations, the system can also quantify the error of the network when processing this data, that is, the second network debugging error.

[0059] Finally, the system will integrate the first network debugging error and the second network debugging error to obtain the overall target network debugging error of the monitoring data analysis network. This integration process comprehensively considers the performance of the network when processing the original sensor monitoring data and augmented data, so that the target network debugging error can more comprehensively reflect the performance status of the network.

[0060] Through this refined error determination method, the data quality detection system can more accurately evaluate the performance of the monitoring data analysis network and provide a more reliable basis for subsequent network optimization. At the same time, this method also improves the accuracy and reliability of the system's data quality detection, allowing the system to more effectively identify and eliminate low-quality or abnormal data, thereby ensuring the accuracy and reliability of subsequent analysis.

[0061] In this way, the data quality detection system can more accurately determine the target network debugging error of the monitoring data analysis network, providing a solid foundation for subsequent network optimization. This not only improves the automation level of data quality detection, but also brings more efficient and accurate data analysis tools to the field of marine environmental monitoring. At the same time, by integrating the first network debugging error and the second network debugging error, the system can more comprehensively evaluate the performance status of the network, further improving the accuracy and reliability of data quality detection.

[0062] It is worth noting that the process of determining the target network debugging error of the monitoring data analysis network can be introduced by the following formula: First, define the first network debugging error (Error1) and the second network debugging error (Error2). For each marine environment sensor monitoring sample data, its first network debugging error can be calculated by the following formula: [Error1_i=LossFunction(KV_i, Label_i)]; Among them, (KV_i) represents the environmental monitoring quality knowledge vector of the (i)th marine environmental sensor monitoring sample data, (Label_i) represents the prior training label of the sample data (i.e. the corresponding marine environmental area label), and (LossFunction) is a loss function used to measure the difference between the predicted value (knowledge vector) and the true value (prior training label).

[0063] Similarly, for each marine environment sample augmented data, its second network debugging error can be calculated by the following formula: [Error2_j=LossFunction(KV'_j, Label'_j)]; Among them, (KV'_j) represents the environmental monitoring quality knowledge vector of the (j)th marine environmental sample augmented data, and (Label'_j) represents the prior training label of the augmented data (that is, the corresponding associated environmental area label).

[0064] After calculating all the first network debugging errors and the second network debugging errors, the overall target network debugging error (TotalError) of the monitoring data analysis network can be obtained by integrating these errors. The integration method can be a simple average or a weighted average, depending on the importance of different errors. Taking simple average as an example, the target network debugging error can be calculated by the following formula: [TotalError=frac{sum_{i}Error1_i+sum_{j}Error2_j}{N1+N2}]; Among them, (N1) and (N2) are the number of marine environment sensor monitoring sample data and marine environment sample augmented data respectively. This formula averages the network debugging errors of the two types of data to obtain an overall target network debugging error.

[0065] Through the above formula, the performance of the monitoring data analysis network in processing different types of marine environment monitoring data can be quantitatively evaluated, and clear guidance can be provided for subsequent network optimization.

[0066] In some alternative embodiments, the method further includes: obtaining at least one target environment sensor monitoring sample data in the sample data set, the at least one target environment sensor monitoring sample data including at least one target marine environment sample augmented data, or at least one target marine environment sensor monitoring sample data and at least one target marine environment sample augmented data; through the monitoring data analysis network, performing knowledge vector mining on each target marine environment sample augmented data in the at least one target environment sensor monitoring sample data, to obtain the environmental monitoring quality knowledge vector of each target marine environment sample augmented data.

[0067] Then, the target network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the environmental monitoring quality knowledge vector of each marine environment sample augmented data, including: determining the target network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the environmental monitoring quality knowledge vector of each target marine environment sample augmented data.

[0068] Based on this embodiment, the data quality detection system adopts a more comprehensive method to improve the accuracy of network debugging and the generalization ability of the network.

[0069] First, the data quality detection system obtains additional target environment sensor monitoring sample data from the sample data set. This data can be at least one target marine environment sample augmented data, or at least one target marine environment sensor monitoring sample data and at least one target marine environment sample augmented data. The advantage of this is that it increases the diversity and richness of data used for network debugging, allowing the network to be exposed to more types of data situations.

[0070] Next, the data quality detection system will use the trained monitoring data analysis network to mine knowledge vectors for the newly acquired target environment sensor monitoring sample data. In particular, for each target marine environment sample augmented data, the system will mine its corresponding environmental monitoring quality knowledge vector. This process is actually extracting the deep features of the data, providing more refined input for subsequent network debugging error determination.

[0071] The data quality detection system takes multiple factors into consideration when determining the target network debugging error of the monitoring data analysis network. In addition to the original environmental monitoring quality knowledge vector for each marine environment sensor monitoring sample data and the environmental monitoring quality knowledge vector for each marine environment sample augmented data, the environmental monitoring quality knowledge vector for each target marine environment sample augmented data is now added. This integration process makes the determination of the target network debugging error more comprehensive and accurate because it covers more types of data and environmental conditions.

[0072] In this way, the data quality detection system can more accurately evaluate and optimize the performance of the monitoring data analysis network. This not only helps improve the network's ability to process various types of data, but also enhances the network's generalization ability, allowing it to maintain good performance when faced with new, unseen data.

[0073] In this way, by introducing the target environment sensor monitoring sample data and comprehensively considering the environmental monitoring quality knowledge vectors of various types of data to determine the target network debugging error, the data quality detection system achieves a more refined and comprehensive network performance evaluation. This not only improves the accuracy of data quality detection, but also provides a more reliable basis for subsequent network optimization, thereby further improving the overall performance of the data quality detection system.

[0074] In the next step, the target network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data, and the environmental monitoring quality knowledge vector of each target marine environment sample augmented data, including: determining the first network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data, the prior training annotation of any marine environment sensor monitoring sample data is used to characterize the marine environment area corresponding to any marine environment sensor monitoring sample data; based on the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the prior training annotation of each marine environment sensor monitoring sample data The second network debugging error of the monitoring data analysis network is determined based on the prior training annotations of each marine environment sample augmented data, and the prior training annotations of any marine environment sample augmented data are used to characterize the associated environmental area corresponding to any marine environment sample augmented data; the third network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each target marine environment sample augmented data and the prior training annotations of each target marine environment sample augmented data, and the prior training annotations of any target marine environment sample augmented data are used to characterize the associated environmental area corresponding to any target marine environment sample augmented data; the first network debugging error, the second network debugging error and the third network debugging error are integrated to obtain the target network debugging error of the monitoring data analysis network.

[0075] In this embodiment, the data quality detection system performs a more detailed network debugging error determination process, which takes into account multiple types of data, including original marine environment sensor monitoring sample data, marine environment sample augmented data, and specially introduced target marine environment sample augmented data.

[0076] First, the data quality detection system uses the environmental monitoring quality knowledge vector of each marine environmental sensor monitoring sample data and the prior training annotations of these sample data to determine the first network debugging error of the monitoring data analysis network. The prior training annotations here are used to identify the specific marine environmental area corresponding to each sample data. By comparing the knowledge vector output by the network with these prior annotations, the system can quantify the error of the network when processing the original sensor monitoring data, that is, the first network debugging error.

[0077] Next, the system uses a similar approach to determine the second network tuning error, but this time based on the augmented data of marine environmental samples. These augmented data are derived from the original data through a series of transformations to simulate more diverse environmental conditions. The system uses the environmental monitoring quality knowledge vectors of each augmented data and their corresponding prior training annotations (these annotations indicate the association of the augmented data with specific environmental areas) to calculate the second network tuning error. This helps evaluate the performance of the network when processing the transformed data.

[0078] In addition, in order to further improve the accuracy of network debugging, the data quality detection system also introduces target marine environmental sample augmented data. These data may be collected from specific environmental areas, or have undergone a specially designed augmentation process to simulate certain extreme or rare environmental conditions. The system performs similar operations to the previous ones on these target augmented data: extracting environmental monitoring quality knowledge vectors and comparing them with their prior training annotations to determine the third network debugging error. These prior annotations reflect the correspondence between the target augmented data and the specific associated environmental areas.

[0079] Finally, the data quality detection system integrates the first, second, and third network debugging errors. This integration process may involve weighted averaging of different types of errors, or weighting them according to their importance to network performance. In this way, the system obtains a target network debugging error for the monitoring data analysis network that comprehensively considers multiple data types and environmental conditions.

[0080] In this way, by comprehensively considering the performance of original data, augmented data, and target augmented data in network debugging, the data quality detection system can more comprehensively evaluate the performance of the network and discover the network's weaknesses on different types of data. This helps with subsequent network optimization work, so that the optimized network not only performs well on standard data, but also maintains high performance when processing various complex and changing environmental data. Therefore, this process improves the accuracy and reliability of the data quality detection system and provides strong support for data analysis in the field of marine environmental monitoring.

[0081] Under some other preferred design ideas, the monitoring data analysis network includes a first cluster confidence of the marine environment area corresponding to each marine environment sensor monitoring sample data in the first sensor monitoring sample data set, and a second cluster confidence of the associated environment area corresponding to each marine environment sample augmented data in the second sensor monitoring sample data set; the third network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each target marine environment sample augmented data and the prior training annotation of each target marine environment sample augmented data, including: wandering the target marine environment sample augmented data in at least one target environment sensor monitoring sample data , and respectively integrate the wandering target ocean environment sample augmented data with each first cluster confidence and each second cluster confidence to obtain the possibility that the wandering target ocean environment sample augmented data belongs to the ocean environment area corresponding to each first cluster confidence, and the possibility that the wandering target ocean environment sample augmented data belongs to the associated environment area corresponding to each second cluster confidence; after wandering the target ocean environment sample augmented data in at least one target environment sensor monitoring sample data, determine the third network debugging error of the monitoring data analysis network according to the possibility of each target ocean environment sample augmented data and the prior training annotation of each target ocean environment sample augmented data.

[0082] Based on this design idea, the data quality detection system further introduced the concept of cluster confidence to more carefully evaluate the performance of the monitoring data analysis network. This design not only considers the network's ability to process a single piece of data, but also focuses on the network's performance in data classification and distinguishing different environmental areas.

[0083] For example, the monitoring data analysis network now includes two important components: the first cluster confidence and the second cluster confidence. The first cluster confidence corresponds to the marine environment area to which each marine environment sensor monitoring sample data in the first sensor monitoring sample data set belongs. In other words, it reflects the network's confidence in the judgment of the environmental area to which the original sensor monitoring data belongs. Similarly, the second cluster confidence is related to the marine environment sample augmented data in the second sensor monitoring sample data set, indicating the network's confidence in the associated environmental area to which these augmented data belong.

[0084] When determining the third network debugging error of the monitoring data analysis network, the data quality detection system adopts a new method. First, the system will "wander" the target marine environment sample augmented data in at least one target environment sensor monitoring sample data. The "wandering" here can be understood as checking or processing these data one by one. For each wandered target marine environment sample augmented data, the system will integrate it with each first cluster confidence and each second cluster confidence.

[0085] During the integration process, the system calculates the probability that the target marine environment sample augmented data belongs to the marine environment area corresponding to each first cluster confidence, and the probability that it belongs to the associated environment area corresponding to each second cluster confidence. These probabilities reflect the confidence and accuracy of the network in classifying specific data into different environmental areas.

[0086] When all the target marine environment sample augmented data have been roamed and processed, the data quality detection system will determine the third network debugging error of the monitoring data analysis network based on the likelihood of each target marine environment sample augmented data and its corresponding prior training annotation. This error reflects the performance gap of the network in correctly classifying the augmented data into the environmental area to which it belongs.

[0087] By introducing clustering confidence and new error determination methods, the data quality detection system can more accurately evaluate the performance of the monitoring data analysis network when processing complex and changeable marine environmental data. This not only helps to discover the network's weaknesses in data classification, but also provides more specific guidance for subsequent network optimization.

[0088] In this way, the performance evaluation capability of the data quality detection system is comprehensively improved by comprehensively considering the network's data processing capabilities and data classification accuracy. It enables the system to more accurately identify the network's deficiencies in data classification in different environmental areas, and make targeted optimization improvements accordingly, thereby further improving the overall efficiency and reliability of the data quality detection system.

[0089] In some other possible embodiments, the first network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data, including: wandering the at least one marine environment sensor monitoring sample data, and integrating the environmental monitoring quality knowledge vector of the wandered marine environment sensor monitoring sample data and each first sub-cluster confidence to obtain the possibility that the wandered marine environment sensor monitoring sample data belongs to the marine environment area corresponding to each first sub-cluster confidence; after wandering the at least one marine environment sensor monitoring sample data, the first network debugging error of the monitoring data analysis network is determined based on the possibility of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data.

[0090] Then, the second network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the prior training annotation of each marine environment sample augmented data, including: wandering the at least one marine environment sample augmented data, and integrating the environmental monitoring quality knowledge vector of the wandered marine environment sample augmented data and each second sub-cluster confidence to obtain the possibility that the wandered marine environment sample augmented data belongs to the associated environmental area corresponding to each second sub-cluster confidence; after wandering the at least one marine environment sample augmented data, the second network debugging error of the monitoring data analysis network is determined based on the possibility of each marine environment sample augmented data and the prior training annotation of each marine environment sample augmented data.

[0091] Based on this embodiment, the data quality detection system adopts a more detailed method to determine the first network debugging error and the second network debugging error of the monitoring data analysis network. This method combines the concepts of environmental monitoring quality knowledge vector, prior training annotation and clustering confidence to achieve more accurate network performance evaluation.

[0092] First, to determine the first network debugging error, the data quality detection system will "walk" at least one marine environment sensor monitoring sample data. Here, "walking" means processing these data one by one. During the walking process, the system will integrate the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the confidence of each first cluster. The confidence of the first cluster reflects the network's confidence that the data belongs to a specific marine environment area.

[0093] Through integrated processing, the system is able to calculate the likelihood that the wandering marine environment sensor monitoring sample data belongs to the marine environment area corresponding to each first cluster confidence. This likelihood assessment helps understand the accuracy of the network in classifying and identifying different marine environment areas.

[0094] After all the marine environment sensor monitoring sample data are traversed, the data quality detection system will determine the first network debugging error based on the probability of each data and its corresponding prior training annotation. The prior training annotation provides the true attribution information of the data, and by comparing it with the actual classification results, the error of the network in processing this data can be quantified.

[0095] Similarly, for the determination of the second network debugging error, the data quality detection system will walk at least one marine environment sample augmented data. Augmented data is simulated by transforming the original data to have more diverse environmental conditions. During the walking process, the system will integrate the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the confidence of each second cluster.

[0096] Through integrated processing, the system can also calculate the probability that the augmented data of the wandering marine environment samples belong to the associated environmental area corresponding to each second cluster confidence. This evaluation method helps to reveal the performance of the network when processing transformed data.

[0097] Finally, after swimming through all the augmented data of marine environment samples, the data quality detection system will determine the second network debugging error based on the probability of each augmented data and its corresponding prior training annotation. This error reflects the classification accuracy and generalization ability of the network when processing augmented data.

[0098] In this way, the data quality detection system can more accurately evaluate the performance of the monitoring data analysis network when processing different types of data. This not only helps to discover the network's weaknesses in classifying and identifying marine environmental areas, but also provides targeted guidance and suggestions for subsequent network optimization. Therefore, this method is of great significance in improving the accuracy and reliability of the data quality detection system.

[0099] In an alternative embodiment, the at least one target environment sensor monitoring sample data includes at least one target marine environment sensor monitoring sample data, or at least one target marine environment sample augmented data, or at least one target marine environment sensor monitoring sample data and at least one target marine environment sample augmented data; the monitoring data analysis network is used to perform knowledge vector mining on each target marine environment sample augmented data in the at least one target environment sensor monitoring sample data to obtain the environmental monitoring quality knowledge vector of each target marine environment sample augmented data, including: the monitoring data analysis network is used to perform knowledge vector mining on each target environment sensor monitoring sample data in the at least one target environment sensor monitoring sample data to obtain the environmental monitoring quality knowledge vector of each target environment sensor monitoring sample data.

[0100] Then, the target network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the environmental monitoring quality knowledge vector of each target marine environment sample augmented data, including: determining the target network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the environmental monitoring quality knowledge vector of each target environment sensor monitoring sample data.

[0101] In this embodiment, the object processed by the data quality detection system - at least one target environment sensor monitoring sample data, can be at least one target marine environment sensor monitoring sample data, or at least one target marine environment sample augmented data, or a combination of the two. This flexibility enables the system to select appropriate data types for analysis according to different needs and environmental conditions.

[0102] When the system needs to process the target environmental sensor monitoring sample data, it will perform knowledge vector mining on these data through the monitoring data analysis network. This process aims to extract the environmental monitoring quality information contained in the data and convert it into the form of knowledge vectors. These knowledge vectors not only contain the original information of the data, but also reflect the inherent connection between the data and the environmental monitoring quality.

[0103] For example, for each target environmental sensor monitoring sample data (whether original data or augmented data) in at least one target environmental sensor monitoring sample data, the data quality detection system will perform knowledge vector mining through the monitoring data analysis network. The mining result is the environmental monitoring quality knowledge vector corresponding to each target environmental sensor monitoring sample data.

[0104] When determining the target network debugging error of the monitoring data analysis network, the data quality detection system will comprehensively consider multiple factors. It will be based on the environmental monitoring quality knowledge vector of each marine environmental sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environmental sample augmented data, and the environmental monitoring quality knowledge vector of each target environmental sensor monitoring sample data obtained through mining.

[0105] These knowledge vectors provide rich information about data quality, allowing the system to more accurately evaluate the performance of the monitoring data analysis network. By comparing the difference between the knowledge vector output by the network and the true value, the system can calculate the debugging error of the network.

[0106] In this way, by comprehensively considering the knowledge vectors of different types of data (including original data and augmented data), the data quality detection system can more comprehensively evaluate the performance of the monitoring data analysis network. This method not only improves the accuracy of the evaluation, but also enables the system to make adaptive adjustments based on different types of data, thereby showing greater flexibility and robustness in practical applications. In this way, by introducing more flexible data processing methods and comprehensive evaluation mechanisms, the performance evaluation capabilities and scope of application of the data quality detection system are further improved.

[0107] In some exemplary embodiments, the monitoring data analysis network includes a knowledge vector mining branch, which is used to perform knowledge vector mining on each marine environment sensor monitoring data; the network configuration variables in the monitoring data analysis network are improved based on the rule of converging the target network debugging error to obtain a data quality detection network, including: based on the rule of converging the target network debugging error, the vector mining indication parameter of the knowledge vector mining branch is improved to obtain a data quality detection network; wherein the data quality detection network includes the knowledge vector mining branch after the vector mining indication parameter is improved.

[0108] Based on this embodiment, the data quality detection system further refines the functions and operation modes of its internal components. In particular, a "monitoring data analysis network" is introduced here, which contains an important component - "knowledge vector mining branch". The main task of this branch is to conduct in-depth knowledge vector mining of various marine environment sensor monitoring data.

[0109] Knowledge vector mining is a complex process that aims to extract meaningful features and information from raw data and convert this information into vector form for subsequent analysis and processing. In the context of marine environmental monitoring, this means extracting key information related to environmental quality, pollution status, etc. from a large amount of sensor data.

[0110] Now, when it comes to improving the monitoring data analysis network based on the rules of converging the target network debugging error, we are actually focusing on how to optimize the performance of this knowledge vector mining branch. Specifically, the system will adjust the "vector mining indicator parameters" in the knowledge vector mining branch by analyzing the convergence of the network debugging error.

[0111] "Vector mining indicator parameters" can be regarded as a set of parameters or settings that control the knowledge vector mining process. By adjusting these parameters, the system can change the depth, breadth or focus of mining, thereby affecting the quality and accuracy of the knowledge vectors finally extracted.

[0112] During the improvement process, the data quality detection system will continuously try different parameter combinations and evaluate the impact of each combination on network performance until a set of parameters that can minimize network debugging errors is found. This process is iterative and requires multiple attempts and verifications.

[0113] Finally, once the system determines the optimal vector mining indicator parameters, it applies them to the knowledge vector mining branch to obtain an improved "data quality detection network" that will be more efficient and accurate in extracting and processing marine environmental sensor monitoring data.

[0114] Thus, it is demonstrated how the data quality detection system can continuously optimize its overall performance by fine-tuning the parameters and settings of its internal components. In particular, by improving the vector mining indicator parameters of the knowledge vector mining branch, the system can more effectively extract key information from the marine environment sensor monitoring data, thereby improving the accuracy and reliability of data quality detection.

[0115] In some other exemplary embodiments, the method also includes: screening at least one marine environment sensor monitoring sample data from the first sensor monitoring sample data set included in the sample data set; performing knowledge vector mining on each of the screened at least one marine environment sensor monitoring sample data through a basic monitoring data analysis network to obtain an environmental monitoring quality knowledge vector of each screened marine environment sensor monitoring sample data; determining a pre-debugging network debugging error of the basic monitoring data analysis network based on the environmental monitoring quality knowledge vector of each screened marine environment sensor monitoring sample data and a priori training annotations of each screened marine environment sensor monitoring sample data; and improving network configuration variables in the basic monitoring data analysis network based on rules for converging the pre-debugging network debugging error to obtain the monitoring data analysis network.

[0116] Based on this embodiment, the method of the data quality detection system further expands its operation process and adds some key steps to optimize and improve the accuracy of data analysis.

[0117] First, the data quality detection system will select at least one marine environment sensor monitoring sample data from the sample data set. The purpose of this step is to select the most representative data sample that best reflects the actual marine environment from a large amount of data. The screening process may be based on a variety of factors, such as the data collection time, location, quality, etc.

[0118] Next, the system uses a basic monitoring data analysis network to perform knowledge vector mining on the selected marine environmental sensor monitoring sample data. Knowledge vector mining is a process of converting data into vectors containing rich information that can reflect the essential characteristics and associations of the data. In this step, the basic monitoring data analysis network will deeply analyze each sample data and extract knowledge vectors related to environmental monitoring quality.

[0119] After extracting the knowledge vectors, the data quality detection system will combine these vectors with the prior training annotations of the sample data to determine the pre-debugging network debugging error of the basic monitoring data analysis network. The prior training annotations are information given in advance that reflects the true situation of the data. By comparing them with the mined knowledge vectors, the accuracy and error of the network in processing this data can be evaluated.

[0120] Then, based on the convergence rules of the pre-debugging network debugging error, the system will further improve the network configuration variables in the basic monitoring data analysis network. Network configuration variables are key factors that affect network performance and output results. By adjusting these variables, the network's operating efficiency and accuracy can be optimized. The improvement process may include adjusting the network's structure, parameter settings, learning rate, etc.

[0121] Ultimately, after repeated iterations and optimizations of the above steps, the data quality detection system will obtain a more accurate and efficient monitoring data analysis network. This network can more accurately mine and analyze marine environmental sensor monitoring data, providing strong support for environmental protection and management.

[0122] With this design, by screening and optimizing data samples and continuously improving the basic monitoring data analysis network, the data quality detection system can gradually improve its processing capacity and accuracy of marine environmental sensor monitoring data. This not only helps to discover environmental problems in a timely manner, but also provides a scientific basis for the decision-making of relevant departments, thereby promoting the effective implementation of environmental protection work.

[0123] In some independent embodiments, the method further comprises: utilizing the data quality detection network to perform data quality detection on the marine environment sample augmented data to be processed.

[0124] Based on this independent embodiment, the function of the data quality detection system is further expanded and applied. In these embodiments, the system not only focuses on the construction and optimization of the network, but also applies it to the actual data quality detection task.

[0125] Specifically, after completing the construction and optimization of the monitoring data analysis network, the data quality detection system will use this network, the data quality detection network, to perform data quality detection on the augmented data of the marine environment samples to be processed. This process is the ultimate manifestation of the system's functions and an important aspect of its practical application value.

[0126] First of all, we need to clarify what is "augmented data of unprocessed marine environment samples". These data are usually obtained by expanding and enriching the original marine environment sensor monitoring data in some way (such as data enhancement technology). The purpose of augmented data is to increase the diversity and coverage of data, thereby improving the generalization ability and robustness of the model.

[0127] When the data quality detection system receives the augmented data of the marine environment samples to be processed, it will first input the data into the trained data quality detection network. After the previous optimization process, this network has strong feature extraction and classification capabilities and can effectively identify anomalies and noise in the data.

[0128] Next, the data quality detection network will conduct in-depth analysis and processing of the input augmented data. It will determine whether the quality of the data meets the standards and whether there are outliers or noise based on the learned knowledge vectors and rules. During this process, the network will output an evaluation result on the data quality, which can be a specific quality score or a classification label (such as "qualified" or "unqualified").

[0129] Finally, the data quality detection system will feed back the evaluation results to the user or the relevant data processing process. Based on this result, the user can understand the quality status of the data to be processed and make corresponding processing decisions. For example, if the data quality is not up to standard, the user can choose to re-collect the data, clean the data or take other remedial measures.

[0130] In this way, by using the data quality detection network to perform quality detection on the augmented data of the marine environment samples to be processed, the system can help users promptly discover and deal with problems in the data to ensure the accuracy and reliability of the data. This is crucial for subsequent data analysis, model training and other tasks, and can effectively avoid errors and deviations caused by data quality problems. At the same time, this automated data quality detection method can also greatly improve work efficiency and reduce the cost and risk of manual intervention.

[0131] It is worth mentioning that the data quality detection network is used to perform data quality detection on the augmented data of the marine environment samples to be processed, including: Receive and preprocess the augmented data of the marine environment samples to be processed, where the preprocessing includes data standardization, removal of outliers, and filling of missing values ​​to ensure that the data input into the data quality detection network meets the specifications of network processing; The preprocessed augmented data is input into the data quality detection network, and the knowledge vector mining branch in the network is used to perform in-depth feature extraction and knowledge vector mining on the data to capture key environmental quality information in the data; Based on the mined knowledge vectors, the trained classifiers or regressors in the data quality detection network are used to comprehensively evaluate the quality of the data, including but not limited to the authenticity, completeness, consistency and timeliness of the data; According to the evaluation results, a data quality score is generated for each marine environment sample augmented data to be processed. The score reflects the overall quality level of the data. The higher the score, the better the data quality. Compare the generated data quality score with the preset quality threshold. If the score is lower than the threshold, the data is automatically marked as low-quality data and the data quality warning mechanism is triggered to promptly notify the user to correct or re-collect the data. For data with a score higher than or equal to the preset quality threshold, the anomaly detection mechanism in the data quality detection network is further used to identify and mark possible local abnormal data points, providing users with more detailed data correction guidance; Finally, the augmented data of the unprocessed marine environment samples that have undergone comprehensive quality inspection and anomaly marking are output, as well as the corresponding data quality report for users to conduct subsequent data analysis and application.

[0132] In this embodiment of the data quality detection system, the system performs a series of precise and complex steps to perform data quality detection on the augmented data of the marine environment samples to be processed. First, the data quality detection system receives the augmented data of the marine environment samples to be processed. These data may come from various sensors and devices, collected at different times and places, so their formats, ranges and units may be different. In order to ensure that these data can be correctly processed by the data quality detection network, the system will first preprocess them. The preprocessing stage includes data standardization, outlier removal and missing value filling. Data standardization is to adjust the numerical range of the data to the range that the network can handle, usually by scaling or conversion. Outlier removal is to exclude those data points that are obviously unreasonable due to equipment failure, operating errors or other reasons. Filling missing values ​​is to deal with the missing data caused by missed sampling or equipment failure that may occur during the data collection process, which can be done through interpolation, estimation or other statistical methods. After completing the preprocessing, the data quality detection system will input the processed augmented data into the data quality detection network. This network contains a special knowledge vector mining branch that can perform in-depth feature extraction and knowledge vector mining on the data. This means the network analyzes patterns, trends and correlations in the data to capture key environmental quality information.

[0133] Next, based on the mined knowledge vectors, the trained classifiers or regressors in the data quality detection network will conduct a comprehensive evaluation of the data quality. This evaluation process will consider multiple aspects of the data, including but not limited to the authenticity of the data (i.e., whether the data accurately reflects the actual situation), completeness (whether the data is comprehensive and without omissions), consistency (whether there are logical contradictions between the data), and timeliness (whether the data is updated in a timely manner). Based on this comprehensive evaluation result, the data quality detection system will generate a data quality score for each marine environment sample augmented data to be processed. This score is a comprehensive indicator that reflects the overall quality level of the data. The higher the score, the better the quality of the data and the more trustworthy it is.

[0134] The system will then compare the generated data quality score with the preset quality threshold. If the score is lower than this threshold, it means that the data quality is low and may not be sufficient to support subsequent analysis and decision-making. In this case, the system will automatically mark the data as low-quality data and trigger the data quality warning mechanism to promptly notify the user to correct or re-collect the data. For data with a score higher than or equal to the preset quality threshold, the data quality detection system will further use its internal anomaly detection mechanism to identify and mark possible local abnormal data points. These anomalies may be caused by accidental measurement errors, equipment failures, or other unknown factors. By identifying these anomalies, the system can provide users with more detailed data correction guidance.

[0135] Finally, the data quality detection system will output the augmented data of the marine environment samples to be processed, which have been fully quality-checked and marked with abnormal points, as well as the corresponding data quality report. This report describes in detail the quality status of the data, existing problems, and possible improvement suggestions. Users can decide how to further process and use the data based on this report.

[0136] In this way, a comprehensive and sophisticated data quality detection method is provided. Through automated preprocessing, feature extraction, quality evaluation, and anomaly detection, the system can efficiently identify problems and deficiencies in the data and promptly notify users to make corrections. This not only improves the accuracy and reliability of the data, but also greatly reduces the need and cost of manual intervention. At the same time, this method can also help users better understand the characteristics and quality of the data, so as to make more informed decisions. Furthermore, Figure 2 The structural block diagram of the data quality detection system 300 is shown, which includes: a memory 310 for storing program instructions and data; a processor 320 for coupling with the memory 310 and executing the instructions in the memory 310 to implement the above method.

[0137] Furthermore, a computer storage medium is provided, comprising instructions, which implement the above method when executed on a processor.

[0138] In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device and method embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0139] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0140] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0141] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data quality detection method using deep learning, characterized in that: Applied to a data quality detection system, the method comprises: Acquire at least one marine environment sensor monitoring sample data in a first sensor monitoring sample data set included in the sample data set, and acquire at least one marine environment sample augmented data in a second sensor monitoring sample data set included in the sample data set, wherein the number of marine environment sensor monitoring sample data in the first sensor monitoring sample data set is greater than the number of marine environment sample augmented data in the second sensor monitoring sample data set; Through the monitoring data analysis network, knowledge vector mining is performed on each of the at least one marine environment sensor monitoring sample data to obtain the environmental monitoring quality knowledge vector of each of the marine environment sensor monitoring sample data; and through the monitoring data analysis network, knowledge vector mining is performed on each of the at least one marine environment sample augmented data to obtain the environmental monitoring quality knowledge vector of each of the marine environment sample augmented data; Determining a target network debugging error of the monitoring data analysis network according to the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the environmental monitoring quality knowledge vector of each marine environment sample augmented data; Based on the rule of converging the debugging error of the target network, the network configuration variables in the monitoring data analysis network are improved to obtain a data quality detection network, and the data quality detection network is used to perform data quality detection on the marine environment sample augmented data.

2. The method according to claim 1, characterized in that Determining the target network debugging error of the monitoring data analysis network according to the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the environmental monitoring quality knowledge vector of each marine environment sample augmented data includes: Determining a first network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data, wherein the prior training annotation of any marine environment sensor monitoring sample data is used to characterize the marine environment area corresponding to the any marine environment sensor monitoring sample data; Determining a second network debugging error of the monitoring data analysis network according to the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the prior training annotation of each marine environment sample augmented data, wherein the prior training annotation of any marine environment sample augmented data is used to characterize the associated environmental area corresponding to any marine environment sample augmented data; The first network debugging error and the second network debugging error are integrated to obtain a target network debugging error of the monitoring data analysis network.

3. The method according to claim 1, characterized in that The method further comprises: Acquire at least one target environment sensor monitoring sample data in the sample data set, wherein the at least one target environment sensor monitoring sample data includes at least one target marine environment sample augmented data, or at least one target marine environment sensor monitoring sample data and at least one target marine environment sample augmented data; Through the monitoring data analysis network, knowledge vector mining is performed on each target marine environment sample augmented data in the at least one target environment sensor monitoring sample data to obtain an environmental monitoring quality knowledge vector of each target marine environment sample augmented data; Determining the target network debugging error of the monitoring data analysis network according to the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the environmental monitoring quality knowledge vector of each marine environment sample augmented data includes: The target network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data, and the environmental monitoring quality knowledge vector of each target marine environment sample augmented data.

4. The method according to claim 3, characterized in that The step of determining the target network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data, and the environmental monitoring quality knowledge vector of each target marine environment sample augmented data comprises: Determining a first network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data, wherein the prior training annotation of any marine environment sensor monitoring sample data is used to characterize the marine environment area corresponding to the any marine environment sensor monitoring sample data; Determining a second network debugging error of the monitoring data analysis network according to the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the prior training annotation of each marine environment sample augmented data, wherein the prior training annotation of any marine environment sample augmented data is used to characterize the associated environmental area corresponding to any marine environment sample augmented data; Determine a third network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each target marine environment sample augmented data and the prior training annotation of each target marine environment sample augmented data, wherein the prior training annotation of any target marine environment sample augmented data is used to characterize the associated environmental area corresponding to any target marine environment sample augmented data; The first network debugging error, the second network debugging error and the third network debugging error are integrated to obtain a target network debugging error of the monitoring data analysis network.

5. The method according to claim 4, characterized in that The monitoring data analysis network includes a first cluster confidence of a marine environment area corresponding to each marine environment sensor monitoring sample data in the first sensor monitoring sample data set, and a second cluster confidence of an associated environment area corresponding to each marine environment sample augmented data in the second sensor monitoring sample data set; the third network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each target marine environment sample augmented data and the prior training annotation of each target marine environment sample augmented data, including: Swimming the target ocean environment sample augmented data in the at least one target environment sensor monitoring sample data, and integrating the wandering target ocean environment sample augmented data with each first sub-cluster confidence and each second sub-cluster confidence, respectively, to obtain the possibility that the wandering target ocean environment sample augmented data belongs to the ocean environment area corresponding to each first sub-cluster confidence, and the possibility that the wandering target ocean environment sample augmented data belongs to the associated environment area corresponding to each second sub-cluster confidence; After walking through the target ocean environment sample augmented data in the at least one target environment sensor monitoring sample data, the third network debugging error of the monitoring data analysis network is determined based on the possibility of each target ocean environment sample augmented data and the prior training annotation of each target ocean environment sample augmented data.

6. The method according to claim 2 or 4, characterized in that: The step of determining a first network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data comprises: Wandering the at least one marine environment sensor monitoring sample data, and integrating the environmental monitoring quality knowledge vector of the wandering marine environment sensor monitoring sample data and each first sub-cluster confidence, to obtain the possibility that the wandering marine environment sensor monitoring sample data belongs to the marine environment area corresponding to each first sub-cluster confidence; after wandering the at least one marine environment sensor monitoring sample data, determining the first network debugging error of the monitoring data analysis network according to the possibility of each marine environment sensor monitoring sample data and the prior training annotation of each marine environment sensor monitoring sample data; The step of determining the second network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sample augmented data and the prior training annotation of each marine environment sample augmented data comprises: Wander the at least one marine environment sample augmented data, and integrate the environmental monitoring quality knowledge vector of the wandered marine environment sample augmented data and each second sub-cluster confidence to obtain the possibility that the wandered marine environment sample augmented data belongs to the associated environmental area corresponding to each second sub-cluster confidence; after wandering the at least one marine environment sample augmented data, determine the second network debugging error of the monitoring data analysis network according to the possibility of each marine environment sample augmented data and the prior training annotation of each marine environment sample augmented data.

7. The method according to claim 3, characterized in that The at least one target environment sensor monitoring sample data includes at least one target marine environment sensor monitoring sample data, or at least one target marine environment sample augmented data, or at least one target marine environment sensor monitoring sample data and at least one target marine environment sample augmented data; the monitoring data analysis network is used to perform knowledge vector mining on each target marine environment sample augmented data in the at least one target environment sensor monitoring sample data, and obtains the environmental monitoring quality knowledge vector of each target marine environment sample augmented data, including: Through the monitoring data analysis network, knowledge vector mining is performed on each target environment sensor monitoring sample data in the at least one target environment sensor monitoring sample data to obtain the environmental monitoring quality knowledge vector of each target environment sensor monitoring sample data; The step of determining the target network debugging error of the monitoring data analysis network based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data, and the environmental monitoring quality knowledge vector of each target marine environment sample augmented data comprises: The target network debugging error of the monitoring data analysis network is determined based on the environmental monitoring quality knowledge vector of each marine environment sensor monitoring sample data, the environmental monitoring quality knowledge vector of each marine environment sample augmented data, and the environmental monitoring quality knowledge vector of each target environment sensor monitoring sample data.

8. The method according to any one of claims 1 to 5 or 7, characterized in that: The monitoring data analysis network includes a knowledge vector mining branch, and the knowledge vector mining branch is used to perform knowledge vector mining on each marine environment sensor monitoring data; the network configuration variables in the monitoring data analysis network are improved based on the rule of converging the target network debugging error to obtain a data quality detection network, including: Based on the rule of converging the debugging error of the target network, the vector mining indication parameter of the knowledge vector mining branch is improved to obtain a data quality detection network; wherein the data quality detection network includes the knowledge vector mining branch after the vector mining indication parameter is improved.

9. The method according to any one of claims 1 to 5 or 7, characterized in that: The method further comprises: Filtering at least one marine environment sensor monitoring sample data from the first sensor monitoring sample data set included in the sample data set; Through the basic monitoring data analysis network, knowledge vector mining is performed on each of the at least one selected marine environment sensor monitoring sample data to obtain the environmental monitoring quality knowledge vector of each of the selected marine environment sensor monitoring sample data; Determining a pre-debugging network debugging error of the basic monitoring data analysis network based on the environmental monitoring quality knowledge vector of each of the screened marine environmental sensor monitoring sample data and the prior training annotation of each of the screened marine environmental sensor monitoring sample data; Based on the rule of converging the debugging error of the pre-debugging network, the network configuration variables in the basic monitoring data analysis network are improved to obtain the monitoring data analysis network.

10. A data quality detection system, characterized in that: include: Memory, used to store program instructions and data; A processor, coupled to a memory, and configured to execute instructions in the memory to implement the method according to any one of claims 1 to 9.

Images