Data quality analysis method, system and equipment and storage medium

Abstract

The invention relates to the technical field of data processing, in particular to a data quality analysis method, system and device and a storage medium. The data quality analysis method comprises the steps of obtaining training data; inputting the training data into a neural network model to execute a model training operation so as to obtain data processing information of the neural network model on the training data, the data processing information comprising a blood relationship graph of the training data and quality information of the training data in each processing link of model training; inputting the blood relationship graph of the training data and the quality information of the training data in each processing link of model training into a data quality change analysis model, and outputting a quality change result of the training data in each processing link of model training by the data quality change analysis model; and determining a data quality analysis result according to the data quality change result. According to the method, the specific processing link causing the quality reduction of the training data can be quickly positioned.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data quality analysis method, system, device, and storage medium. Background Technology

[0002] With the widespread adoption of large-scale model training, the preparation of high-quality training data has become crucial. Existing technologies typically employ data lineage tracing schemes based on static metadata, recording the data flow path from the source table to the target table for data governance and impact analysis. However, existing methods record very limited process information during model training, usually only including the data processing flow without recording changes in data quality during the flow, and failing to quantify the specific impact of each processing step on the final data quality. Furthermore, they cannot establish a quantitative relationship between processing parameters and quality changes, making automatic tuning difficult. Summary of the Invention

[0003] This application aims to at least solve the technical problems existing in the prior art and provide a data quality analysis method, system, device and storage medium.

[0004] In a first aspect, the present invention provides a data quality analysis method, the method comprising: Obtain training data; The training data is input into the neural network model to perform model training operations, so as to obtain the data processing information of the neural network model on the training data. The data processing information includes the lineage map of the training data and the quality information of the training data in each processing stage of the model training. The kinship map of the training data and the quality information of the training data in each processing stage of model training are input into the data quality change analysis model. The data quality change analysis model is used to analyze the quality information of the kinship map and the training data in each processing stage of model training, and outputs the quality change results of the training data in each processing stage of model training. The data quality analysis results are determined based on the changes in data quality.

[0005] Optionally, the data processing information obtained by inputting training data into the neural network model to perform model training includes: Assign a globally unique identifier to each data unit of the training data and pass it in the processing flow; When a neural network model uses training data to perform model training operations, it records the derivation relationships between data units to form a lineage link, and records the key operating parameters used in each processing stage of model training. At the input and output points of each processing stage of model training, the data scores on multiple preset quality dimensions are collected to obtain the quality information of the training data at each processing stage of model training. The bloodline links, key operating parameters, and quality information of training data in each processing stage of model training are associated and stored in a graph database to obtain a bloodline relationship graph.

[0006] Optionally, the dimensions of the quality metrics may include at least two of the following: data integrity, data accuracy, semantic quality, and task utility quality.

[0007] Optionally, the steps for processing the pedigree map of the training data and the quality information of the training data in each stage of model training in the data quality change analysis model include: Based on the index scores of the input points and the index scores of the output points in each processing stage of model training, the quality change of the training data in each processing stage of model training is calculated, and the quality change result is obtained.

[0008] Optionally, the method further includes a training step for the data quality change analysis model: Acquire several sets of historical sample data. Each set of historical sample data represents the features and labels corresponding to one execution of a model training process. The configurable parameters of the data processing process are used as features, and the quality change of the training data in the processing process is used as labels. A network structure for a data quality change analysis model is constructed. Based on several sets of historical sample data, a supervised learning algorithm is used to iteratively train the model parameters of the network structure until the iteration termination condition is reached. This allows the data quality change analysis model to learn the mapping relationship between the configurable parameters of the neural network model and the amount of quality change, thus obtaining the final data quality change analysis model.

[0009] Optionally, the data quality change analysis model can be a multiple linear regression model or a random forest model.

[0010] Optionally, the method further includes optimizing the neural network model structure and model parameters based on the quality changes of the training data at each stage of model training. This optimization of the model training process and model parameters based on the quality changes of the training data at each stage of model training includes: The overall score of each processing step in the model training is determined based on the quality change results and contribution weight of each step. Based on the comprehensive score of all processing stages during model training, the processing stages to be optimized are selected, and the model parameters of the processing stages to be optimized are optimized.

[0011] In a second aspect, the present invention provides a data quality analysis system, the system comprising: The acquisition module is used to acquire training data; The processing module is used to input training data into the neural network model to perform model training operations, so as to obtain the data processing information of the neural network model on the training data. The data processing information includes the lineage map of the training data and the quality information of the training data in each processing stage of the model training. The quality change analysis module is used to input the pedigree graph of the training data and the quality information of the training data in each processing stage of the model training into the data quality change analysis model. The data quality change analysis model is used to analyze the pedigree graph and the quality information of the training data in each processing stage of the model training, and output the quality change results of the training data in each processing stage of the model training. The output module is used to determine the data quality analysis results based on the changes in data quality.

[0012] Thirdly, the present invention provides an electronic device, the electronic device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the data quality analysis method described above.

[0013] Fourthly, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the data quality analysis method described above.

[0014] In summary, this application includes the following beneficial technical effects: By constructing a data lineage graph of the entire training data chain and recording the quality information of the training data at each processing stage of model training, it is possible to intuitively determine the quality changes of the training data at each processing stage of model training. This allows for a faster identification of the specific processing stage that leads to the quality decline based on the quality change results, which in turn facilitates the optimization of the model parameters of the neural network model by the staff. The pedigree of the data is correlated with its quality status at each processing stage, providing a complete data foundation for subsequent quantitative analysis. By establishing a correlation model between processing parameters and multi-dimensional quality changes, we can accurately locate and analyze the root causes of data quality problems, thereby improving the maintainability of the data pipeline and the speed of problem response. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a data quality analysis method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an electronic device for implementing the data quality analysis method according to an embodiment of the present invention.

[0016] Reference numerals: 10, processor; 11, memory; 12, communication bus; 13, communication interface.

[0017] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0019] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0020] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0021] Reference Figure 1 The diagram shown is a flowchart illustrating a data quality analysis method according to an embodiment of the present invention. In this embodiment, the data quality analysis method includes: S1. Obtain training data.

[0022] S2. Input the training data into the neural network model to perform model training operations, so as to obtain the data processing information of the neural network model on the training data.

[0023] Data processing information includes the lineage graph of the training data and the quality information of the training data at each processing stage of model training. In this embodiment, the neural network model is a Large Language Model (LLM), and the training data is natural text data. A LLM is a deep learning model trained on a large amount of text data, enabling it to generate natural language text or understand the meaning of language text. LLMs can provide in-depth knowledge and language production on various topics through training on massive datasets. Its core idea is to learn the patterns and structures of natural language through large-scale unsupervised training, simulating the human language cognition and generation process to a certain extent.

[0024] Specifically, training data is input into the neural network model to perform model training operations, and the resulting data processing information includes: S21. Assign a globally unique identifier to each data unit of the training data and pass it in the processing flow.

[0025] S22. When the neural network model uses training data to perform model training operations, record the derivation relationship between data units to form a lineage link, and record the key operating parameters used when each processing step of the model training is executed.

[0026] Key operating parameters include the fusion algorithm version, annotator ID, and quality assessment threshold.

[0027] S23. At the input and output points of each processing stage of model training, collect the index scores of the data on multiple preset quality dimensions to obtain the quality information of the training data in each processing stage of model training.

[0028] The dimensions of quality metrics include at least two of the following: data integrity, data accuracy, semantic quality, and task utility quality.

[0029] S24. Link and store the quality information of the bloodline link, key operating parameters and training data in each processing stage of model training in the graph database to obtain the bloodline relationship map.

[0030] S3. Input the kinship map of the training data and the quality information of the training data in each processing stage of model training into the data quality change analysis model. The data quality change analysis model is used to analyze the kinship map and the quality information of the training data in each processing stage of model training, and output the quality change results of the training data in each processing stage of model training.

[0031] Specifically, the steps for the data quality change analysis model to process the lineage map of the training data and the quality information of the training data in each stage of model training include: Based on the index scores of the input points and the index scores of the output points in each processing stage of model training, the quality change of the training data in each processing stage of model training is calculated, and the quality change result is obtained.

[0032] The formula for calculating the change in quality (ΔQuality) is: ΔQuality = Output point index score - Input point index score. This step quantifies the impact of each processing step on the data for the first time into a specific, comparable value.

[0033] The core of building a data quality change analysis model lies in introducing and constructing a "processing parameter-quality change" correlation model (i.e., a data quality change analysis model). This transforms the abstract concept of "quality degradation" in the data processing stage into quantifiable values ​​and traces its root causes. This goes beyond simple quality difference comparisons and achieves modeling of the inherent laws governing quality changes.

[0034] In a preferred embodiment of this example, the data quality analysis method further includes a training step for a data quality change analysis model: S31. Obtain several sets of historical sample data.

[0035] Historical sample data is extracted from the pedigree quality map. Each set of historical sample data represents the features and labels corresponding to one execution of a model training process. For example, a sample could be "a record of the spatiotemporal alignment engine processing data block X". The configurable parameters of the data processing stage are used as features, and the quality change of the training data in the processing stage is used as labels. Configurable parameters of the data processing stage include GeoHash accuracy, time window size, and matching algorithm version.

[0036] The purpose of the data quality change analysis model is to reveal the intrinsic mapping relationship Y=f(X) between the configurable parameters (X) and the amount of quality change (Y).

[0037] The input to the data quality change analysis model is a configurable parameter (feature X), and the output of the data quality change analysis model is the prediction target, that is, the predicted value of the amount of data quality change after the input training data has been processed by this processing step.

[0038] S32. Construct the network structure of the data quality change analysis model, and use a supervised learning algorithm to iteratively train the model parameters of the network structure of the data quality change analysis model based on several sets of historical sample data until the iteration termination condition is reached, so that the data quality change analysis model learns the mapping relationship between the configurable parameters of the neural network model and the amount of quality change, and obtains the final data quality change analysis model.

[0039] Specifically, the data quality change analysis model can be a multiple linear regression model or a random forest model.

[0040] When using linear models (such as multiple linear regression), the weight coefficients learned after model training directly quantify the magnitude and direction of each parameter's impact on quality. For example, the model derives the change in semantic consistency as 0.25 * GeoHash precision - 0.1 * time window; this means that for every unit increase in GeoHash precision, semantic consistency improves by an average of 0.25, and its impact is precisely quantified.

[0041] When using a random forest model, attribution is performed by calculating feature importance. For example, the model outputs feature importance: {GeoHash precision: 0.65, time window: 0.20, ...}. This clearly shows that GeoHash precision is the decisive factor causing quality variations, contributing 65%.

[0042] By using a data quality change analysis model, it is possible to predict the quality changes of training data at each processing stage, so as to identify the processing stage to be optimized.

[0043] S4. Determine the data quality analysis results based on the changes in data quality.

[0044] In another embodiment of this application, the data quality analysis method further includes: S5. Optimize the neural network model structure and model parameters based on the quality changes of the training data in each processing stage of model training.

[0045] Specifically, the steps for optimizing the neural network model structure and model parameters based on the quality changes of the training data at each stage of model training include: S51. Determine the overall score of each processing step in the model training process based on the quality change results and contribution weight of each step.

[0046] Specifically, the average quality gain / loss of each stage is statistically analyzed to identify the "bottleneck stage" that has the greatest impact on the overall quality of the production line. A comprehensive quality gain score is then calculated for each stage in the production line, where the comprehensive score = average quality change * stage contribution weight.

[0047] The average quality change is the average of the quality changes across all historical training data. The average quality change reflects the "general impact" of this step on quality.

[0048] S52. Based on the comprehensive score of all processing steps in the model training, select the processing steps to be optimized and optimize the model parameters of the processing steps to be optimized.

[0049] All processing steps are sorted from lowest to highest based on their "overall score." The bottleneck step is the one ranked last with the lowest score. It is usually the step that has the greatest impact on data quality and is located on the critical path.

[0050] Assuming there are three stages, the calculation results are as follows: Step A (Data Analysis): Average quality change = +0.1, weight = 0.2 → overall score = 0.02; Step B (Semantic Fusion): Average quality change = -0.3, weight = 0.5 → overall score = -0.15; C-stage (quality filtering): Average quality change = +0.2, weight = 0.3 → overall score = 0.06; Conclusion: The B stage (semantic fusion) had the lowest overall score (-0.15) and was identified as the bottleneck of the current system. Resource optimization should be prioritized for this stage.

[0051] Specifically, data quality analysis results can be used to drive the automatic optimization of data processing workflows, as follows: S521. Optimization Suggestion Generation: Based on data quality analysis results, generate parameter optimization suggestions for quality bottlenecks. For example, for the spatiotemporal alignment engine, the suggestion is: "Increase the GeoHash precision from 6 bits to 7 bits, which is expected to improve the semantic consistency score by 0.15."

[0052] The system has a pre-built "optimization strategy library". Based on the conclusions of data quality analysis (e.g., parameter A is positively correlated with quality and its current value is low), the system matches the corresponding optimization strategy from the strategy library. Then, based on the matched optimization strategy, it assembles complete natural language suggestions and machine-readable instructions.

[0053] S522, Automated Execution: Machine-readable optimization instructions are generated based on the matched optimization strategy and sent to the client to control the client to automatically update the network parameters of the neural network model according to the optimization strategy, improving operational convenience. S523, Effect Verification Closed Loop: Control the neural network model to run the data processing flow under the new model parameters, re-record the quality changes, verify the optimization effect, and form a closed loop for continuous optimization.

[0054] The quality analysis results are automatically converted into optimization suggestions for processing parameters and then executed, forming a self-improving data processing system.

[0055] Based on the same inventive concept, an embodiment of the present invention provides a data quality analysis system. The data quality analysis system of the present invention can be installed in an electronic device. According to the implemented functions, the data quality analysis system includes: The acquisition module is used to acquire training data; The processing module is used to input training data into the neural network model to perform model training operations, so as to obtain the data processing information of the neural network model on the training data. The data processing information includes the lineage map of the training data and the quality information of the training data in each processing stage of the model training. The quality change analysis module is used to input the pedigree graph of the training data and the quality information of the training data in each processing stage of the model training into the data quality change analysis model. The data quality change analysis model is used to analyze the pedigree graph and the quality information of the training data in each processing stage of the model training, and output the quality change results of the training data in each processing stage of the model training. The output module is used to determine the data quality analysis results based on the changes in data quality.

[0056] The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0057] The various variations and specific examples of the data quality analysis method provided in the above embodiments are also applicable to the data quality analysis system of this embodiment. Through the foregoing detailed description of the data quality analysis method, those skilled in the art can clearly understand the implementation method of the data quality analysis system in this embodiment. For the sake of brevity, they will not be described in detail here.

[0058] The present invention also discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the data quality analysis method provided by the present invention. The computer program product should be understood as a software product that mainly implements its solution through a computer program, such as a program product integrated in the cloud or a software library.

[0059] This application also discloses an electronic device, such as Figure 2 The diagram shown is a schematic representation of an electronic device for a data quality analysis method according to an embodiment of the present invention. The electronic device may include at least one processor 10, a memory 11 communicatively connected to the at least one processor, a communication bus 12, and a communication interface 13. It may also include a computer program, such as a data quality analysis method program, stored in the memory 11 and executable on the processor 10.

[0060] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., methods for performing data quality analysis) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0061] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as code for data quality analysis methods, but also to temporarily store data that has been output or will be output.

[0062] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0063] Communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0064] Figure 2 Only electronic devices with components are shown; those skilled in the art will understand that... Figure 2 The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0065] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to at least one processor 10 via a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0066] It should be understood that the embodiments are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0067] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile.

[0068] This application provides a computer-readable storage medium, including, for example, any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM). The computer-readable storage medium stores a computer program that can be loaded by a processor and execute the data quality analysis method of the above embodiments.

[0069] In the description of this specification, the references to terms such as "an embodiment," "some embodiments," "example," "specific example," "a implementation," "a preferred implementation," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0070] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A data quality analysis method, characterized in that, The method includes: Obtain training data; The training data is input into the neural network model to perform model training operations, so as to obtain the data processing information of the neural network model on the training data. The data processing information includes the lineage map of the training data and the quality information of the training data in each processing stage of the model training. The kinship map of the training data and the quality information of the training data in each processing stage of model training are input into the data quality change analysis model. The data quality change analysis model is used to analyze the quality information of the kinship map and the training data in each processing stage of model training, and outputs the quality change results of the training data in each processing stage of model training. The data quality analysis results are determined based on the changes in data quality.

2. The data quality analysis method as described in claim 1, characterized in that, The process of inputting training data into a neural network model to perform model training, and obtaining the resulting data processing information, includes: Assign a globally unique identifier to each data unit of the training data and pass it in the processing flow; When a neural network model uses training data to perform model training operations, it records the derivation relationships between data units to form a lineage link, and records the key operating parameters used in each processing stage of model training. At the input and output points of each processing stage of model training, the data scores on multiple preset quality dimensions are collected to obtain the quality information of the training data at each processing stage of model training. The bloodline links, key operating parameters, and quality information of training data in each processing stage of model training are associated and stored in a graph database to obtain a bloodline relationship graph.

3. The data quality analysis method as described in claim 2, characterized in that, The dimensions of quality metrics include at least two of the following: data integrity, data accuracy, semantic quality, and task utility quality.

4. The data quality analysis method as described in claim 2, characterized in that, The data quality change analysis model includes the following steps in processing the lineage map of the training data and the quality information of the training data at each stage of model training: Based on the index scores of the input points and the index scores of the output points in each processing stage of model training, the quality change of the training data in each processing stage of model training is calculated, and the quality change result is obtained.

5. The data quality analysis method as described in claim 4, characterized in that, The method also includes a training step for the data quality change analysis model: Acquire several sets of historical sample data. Each set of historical sample data represents the features and labels corresponding to one execution of a model training process. The configurable parameters of the data processing process are used as features, and the quality change of the training data in the processing process is used as labels. A network structure for a data quality change analysis model is constructed. Based on several sets of historical sample data, a supervised learning algorithm is used to iteratively train the model parameters of the network structure until the iteration termination condition is reached. This allows the data quality change analysis model to learn the mapping relationship between the configurable parameters of the neural network model and the amount of quality change, thus obtaining the final data quality change analysis model.

6. The data quality analysis method as described in claim 5, characterized in that, The data quality change analysis model is either a multiple linear regression model or a random forest model.

7. The data quality analysis method as described in claim 1, characterized in that, The method further includes optimizing the neural network model structure and model parameters based on the quality changes of training data at each stage of model training. This optimization of the model training process and model parameters based on the quality changes of training data at each stage of model training includes: The overall score of each processing step in the model training is determined based on the quality change results and contribution weight of each step. Based on the comprehensive score of all processing stages during model training, the processing stages to be optimized are selected, and the model parameters of the processing stages to be optimized are optimized.

8. A data quality analysis system for implementing the data quality analysis method according to any one of claims 1 to 7, characterized in that, include: The acquisition module is used to acquire training data; The processing module is used to input training data into the neural network model to perform model training operations, so as to obtain the data processing information of the neural network model on the training data. The data processing information includes the lineage map of the training data and the quality information of the training data in each processing stage of the model training. The quality change analysis module is used to input the pedigree graph of the training data and the quality information of the training data in each processing stage of the model training into the data quality change analysis model. The data quality change analysis model is used to analyze the pedigree graph and the quality information of the training data in each processing stage of the model training, and output the quality change results of the training data in each processing stage of the model training. The output module is used to determine the data quality analysis results based on the changes in data quality.

9. An electronic device, characterized in that, The electronic device includes: At least one processor (10); and, A memory (11) communicatively connected to the at least one processor (10); The memory (11) stores a computer program that can be executed by the at least one processor (10) to enable the at least one processor (10) to perform the data quality analysis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program; when the computer program is executed by a processor, it implements the data quality analysis method as described in any one of claims 1 to 7.

Images