Data quality checking method and device, equipment and storage medium

Abstract

The application discloses a data quality checking method and device, equipment and a storage medium, relates to the technical field of data quality management, and comprises the following steps: acquiring multi-modal features of target data; performing data quality analysis on the multi-modal features according to a preset trend prediction model to obtain a trend prediction result of data quality; determining an early warning type according to the trend prediction result, wherein the early warning type comprises at least one of fluctuation early warning, peak value early warning and long-term trend early warning; adjusting a checking parameter in a configuration checking rule according to the early warning type to obtain an adjusted checking rule; and performing quality checking on the target data according to the adjusted checking rule to obtain a checking result, so that the adaptive dynamic adjustment of the checking rule is realized, the risk of missed reports and false reports is reduced, and the accuracy and flexibility of data quality checking are improved.

Description

Technical Field

[0001] This application relates to the technical field of data quality management, and in particular to a data quality verification method, apparatus, device, and storage medium. Background Technology

[0002] As enterprises deepen their digital transformation, data has become a core production factor, and its quality directly affects the accuracy of business decisions, the effectiveness of regulatory compliance, and the stability of system operation. Especially in sectors such as finance, telecommunications, and government, data quality issues can lead to business disruptions, regulatory penalties, and even systemic risks, highlighting the increasing importance of data quality monitoring.

[0003] Currently, data quality monitoring primarily employs rule-based verification, where developers pre-write verification logic to check data integrity and accuracy item by item. This static rule-based monitoring model can be effective in scenarios where data volume is controllable and business logic is relatively stable. Meanwhile, existing monitoring systems mostly use post-event alerting mechanisms, triggering alarms and notifying manual intervention after data anomalies are detected during verification. However, with increasingly complex business systems, exponentially growing data volumes, and continuously evolving regulatory requirements, the above monitoring methods face significant limitations in practical applications: on the one hand, rules rely on manual development and maintenance, resulting in long adjustment cycles and difficulty in flexibly responding to business changes; on the other hand, verification rules are mostly pre-set fixed thresholds, making them prone to missed or false alarms when business fluctuates or data distribution changes; furthermore, post-event alerting mechanisms cause data quality problems to be discovered later than they actually occur, increasing the cost of problem remediation.

[0004] Therefore, improving the accuracy and flexibility of verification during data quality monitoring is an urgent problem that needs to be solved. Summary of the Invention

[0005] The main objective of this application is to provide a data quality verification method, apparatus, device, and storage medium, aiming to solve the technical problem of how to improve the accuracy and flexibility of verification during data quality monitoring.

[0006] To achieve the above objectives, this application proposes a data quality method, the method comprising: Obtain multimodal features from the target data; Data quality analysis is performed on the multimodal features based on a preset trend prediction model to obtain trend prediction results for data quality; The warning type is determined based on the trend prediction results, and the warning type includes at least one of fluctuation warning, peak warning and long-term trend warning. The verification parameters in the configurable verification rules are adjusted according to the warning type to obtain the adjusted verification rules; The target data is quality-verified according to the adjusted verification rules to obtain the verification results.

[0007] In one embodiment, the multimodal features include business behavior features, time-series fluctuation features, and regulatory policy features. The step of obtaining the multimodal features of the target data includes: Acquire target data and extract business behavior features from the data lineage, business metadata, and historical data quality records of the target data. The business behavior features include data source complexity, cross-system association density, regulatory field marking, and information entropy value of enumerated fields. Based on a preset time window, the historical quality indicators of the target data are decomposed to obtain the time-series fluctuation characteristics of each time window. The time-series fluctuation characteristics include the periodic autocorrelation coefficient, frequency domain characteristics, moving average, and year-on-year and month-on-month growth rates. The regulatory policy text in the target data is processed into text vectorization to obtain regulatory policy features, which include regulatory system change vectors, economic indicator deviation vectors, and internal business strategy label vectors.

[0008] In one embodiment, the step of performing data quality analysis on the multimodal features according to a preset trend prediction model to obtain a trend prediction result for data quality includes: The business behavior features, time-series fluctuation features, and regulatory policy features in the multimodal features are aligned to obtain a joint feature matrix; The joint feature matrix is ​​input into a preset trend prediction model, and forward propagation is performed sequentially through the first long short-term memory network layer and the second long short-term memory network layer of the preset trend prediction model to obtain a normalized outlier prediction value. The time step of the first long short-term memory network layer is greater than the time step of the second long short-term memory network layer. The predicted outlier values ​​are inversely normalized to obtain the trend prediction results for data quality.

[0009] In one embodiment, the step of determining the warning type based on the trend prediction result includes: Based on the predicted values ​​of quality indicators in the trend prediction results and the preset actual values, the deviation value of the trend prediction is determined; When the deviation value is greater than the preset fluctuation warning threshold, the warning type is determined to be a fluctuation warning; When the maximum value in the trend prediction result is greater than the preset peak warning threshold, the warning type is determined to be a peak warning; If the deviation value is greater than the preset long-term trend warning threshold within a consecutive preset time window, the warning type is determined to be a long-term trend warning.

[0010] In one embodiment, the step of adjusting the verification parameters in the configurable verification rules according to the warning type to obtain the adjusted verification rules includes: When the warning type is fluctuation warning, the alarm threshold and execution priority of the verification parameters in the configurable verification rules are increased to obtain the adjusted verification rules; When the warning type is a peak warning, the alarm threshold and execution priority of the verification parameters are adjusted according to the supervision type of the corresponding field in the configurable verification rule to obtain the adjusted verification rule; When the warning type is a long-term trend warning, the historical verification result sequence of the configurable verification rule is obtained, and the alarm threshold of the verification parameter in the configurable verification rule is updated according to the historical verification result sequence to obtain the adjusted verification rule.

[0011] In one embodiment, the step of adjusting the alarm threshold and execution priority of the verification parameters according to the supervision type of the corresponding field in the configured verification rule when the warning type is a peak warning, to obtain the adjusted verification rule, includes: When the warning type is a peak warning, the regulatory type of the corresponding field is determined according to the regulatory flag of the corresponding field in the configurable verification rules; When the regulatory type of the corresponding field is a core regulatory field, the alarm threshold of the corresponding field in the configurable verification rule is maintained; When the regulatory type of the corresponding field is a non-core regulatory field, the alarm threshold of the corresponding field in the configurable verification rule is increased.

[0012] In one embodiment, the step of performing quality verification on the target data according to the adjusted verification rules to obtain the verification result includes: The data to be verified is determined from the target data according to the applicable data fields in the adjusted verification rules; Construct constraints based on the verification logic and verification parameters in the adjusted verification rules; When the data to be verified meets the constraints, the verification result is determined to be of normal quality. When the data to be verified does not meet the constraints, the verification result is determined to be of poor quality.

[0013] Furthermore, to achieve the above objectives, this application also proposes a data quality verification device, the device comprising: The feature extraction module is used to obtain multimodal features of the target data; The trend prediction module is used to perform data quality analysis on the multimodal features according to a preset trend prediction model, and obtain the trend prediction result of the data quality. The early warning identification module is used to determine the early warning type based on the trend prediction result, wherein the early warning type includes at least one of fluctuation early warning, peak early warning and long-term trend early warning; The parameter adjustment module is used to adjust the verification parameters in the configurable verification rules according to the warning type, so as to obtain the adjusted verification rules; The quality verification module is used to perform quality verification on the target data according to the adjusted verification rules and obtain the verification result.

[0014] In addition, to achieve the above objectives, this application also proposes a data quality verification device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the data quality verification method as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the data quality verification method described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the data quality verification method described above.

[0017] This application provides a data quality verification method. The method includes: acquiring multimodal features of target data; performing data quality analysis on the multimodal features according to a preset trend prediction model to obtain a trend prediction result for data quality; determining an early warning type based on the trend prediction result, wherein the early warning type includes at least one of fluctuation early warning, peak early warning, and long-term trend early warning; adjusting the verification parameters in the configurable verification rules according to the early warning type to obtain adjusted verification rules; and performing quality verification on the target data according to the adjusted verification rules to obtain a verification result. In summary, this application improves trend prediction accuracy through multimodal feature fusion and dynamically adjusts the verification rules according to differentiated early warning types, achieving adaptive dynamic adjustment of the verification rules, reducing the risk of missed and false alarms, and improving the accuracy and flexibility of data quality verification. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the first embodiment of the data quality verification method of this application; Figure 2 This is a trend prediction curve in one embodiment of the data quality verification method of this application; Figure 3 A flowchart illustrating the second embodiment of the data quality verification method of this application; Figure 4 A flowchart illustrating the third embodiment of the data quality verification method of this application; Figure 5 This is a schematic diagram of the module structure of the data quality verification device according to an embodiment of this application; Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the data quality verification method in the embodiments of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] It should be noted that the executing entity in this embodiment can be a data quality verification system, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the aforementioned data quality verification function. This embodiment does not specifically limit this. The following uses a data quality verification system as an example to describe this embodiment and the following embodiments.

[0025] Based on this, embodiments of this application provide a data quality verification method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the data quality verification method of this application.

[0026] In this embodiment, the data quality verification method includes steps S10 to S50: Step S10: Obtain the multimodal features of the target data.

[0027] It should be noted that in this step, the system extracts multi-dimensional feature information from the target data to construct a feature matrix for subsequent trend prediction analysis. This provides multi-dimensional input information for data quality trend prediction, improving the comprehensiveness and accuracy of the prediction. The target data refers to the business data that needs quality monitoring; it can come from a single data source or from multiple cross-system data sources. Multimodal features refer to a set of feature vectors extracted from different types and dimensions to characterize data quality-related attributes, including business behavior features, time-series fluctuation features, and regulatory policy features.

[0028] In one feasible implementation, step S10 includes: Step S101: Obtain target data and extract business behavior features from the data lineage, business metadata, and historical data quality records of the target data. The business behavior features include data source complexity, cross-system association density, regulatory field markers, and information entropy values ​​of enumerated fields.

[0029] It should be noted that the system analyzes the data lineage, business metadata definitions, and historical data quality verification records of the target data to extract characteristic information reflecting the data's business attributes. This characterizes the complexity and potential risks of the data at the business level, providing business-dimensional input for trend prediction. Specifically, data lineage refers to the entire flow of data from its generation to its final use, including the data's source system, transformation process, and downstream dependencies; business metadata refers to information describing the business meaning of the data, including field definitions, business rules, and data dictionaries; data source complexity refers to the number of different systems or data tables from which the data originates—the more sources, the higher the complexity; cross-system association density refers to the number of associations between the target data and data from other systems—the higher the association density, the greater the impact of other systems on data quality; regulatory field marking refers to the marking information indicating whether a field belongs to a key regulatory field, for example, the "transaction amount" field is marked as a key regulatory field; the information entropy value of an enumerated field is a value used to measure the degree of disorder in the distribution of values ​​of an enumerated field—the higher the information entropy value, the more dispersed the distribution of values ​​of the field, and the greater the data quality risk.

[0030] Step S102: Perform trend decomposition on the historical quality indicators of the target data according to the preset time window to obtain the time series fluctuation characteristics of each time window. The time series fluctuation characteristics include periodic autocorrelation coefficient, frequency domain characteristics, moving average and year-on-year and month-on-month growth rates.

[0031] It should be noted that the system will use multi-scale time windows to perform sliding analysis on the historical quality indicators of the target data, decompose the historical quality indicators into trend components at different time scales, thereby extracting the periodic patterns and fluctuation characteristics of data quality changes over time, and providing time-series input for trend prediction.

[0032] Additionally, it should be noted that preset time windows include short-term time windows (e.g., 7 days), medium-term time windows (e.g., 30 days), and long-term time windows (e.g., 90 days). Historical quality indicators refer to quantitative indicators of data quality within a historical period, including outlier rate, missing rate, and duplication rate. Periodic autocorrelation coefficient is a statistical measure of the correlation between a time series and itself at different lag periods, used to identify periodic patterns in the data. Frequency domain features refer to the frequency component features extracted after converting time series data to the frequency domain through Fourier transform, used to identify hidden periodic patterns in the data. Moving average refers to averaging time series data within a sliding window, used to smooth short-term fluctuations and identify long-term trends. Year-on-year and month-on-month growth rates refer to the growth rate compared to the same period last year or the same period last month, used to measure the changing trend of data quality indicators.

[0033] Step S103: Perform text vectorization processing on the regulatory policy text in the target data to obtain regulatory policy features, which include regulatory system change vectors, economic indicator deviation vectors, and internal business strategy label vectors.

[0034] It should be noted that the system uses natural language processing technology to perform semantic analysis and vectorization on financial regulatory policy texts, transforming unstructured policy texts into structured feature vectors. This allows the system to incorporate policy change factors into the data quality trend prediction, thereby improving the sensitivity of predictions to policy changes.

[0035] Additionally, it should be noted that regulatory policy texts refer to textual materials such as regulatory documents, policy interpretations, and reporting guidelines issued by regulatory agencies such as the State Financial Regulatory Commission and the People's Bank of China. Text vectorization processing refers to the process of converting text content into numerical vectors, including steps such as word segmentation, word embedding, and semantic encoding. The regulatory system change vector is a vector used to characterize the direction and intensity of changes in regulatory systems; for example, this vector will change accordingly when the requirements for a certain type of data reporting field change. The economic indicator deviation vector is a vector formed by the degree of deviation between macroeconomic indicators (such as GDP growth rate, inflation rate, etc.) and expected values. The internal business strategy label vector is a tagged vector representation of adjustments to the bank's internal business strategy (such as the launch of new products, expansion of business areas, etc.).

[0036] Step S20: Perform data quality analysis on the multimodal features according to the preset trend prediction model to obtain the trend prediction results of data quality.

[0037] It should be noted that the system inputs the extracted multimodal features into a pre-defined trend prediction model. This model performs fusion analysis and time-series modeling on the multi-dimensional features to predict data quality trends within future time windows, thus providing a basis for subsequent early warning judgments and rule adjustments. The pre-defined trend prediction model is a deep learning prediction model trained on historical data quality samples, specifically an LSTM (Long Short-Term Memory) model. This model uses historical multimodal features and corresponding data quality indicators as training samples, employs mean squared error as the loss function, and undergoes iterative training and optimization through forward propagation, loss calculation, backpropagation, and parameter updates to ultimately obtain a model capable of predicting future data quality trends.

[0038] Additionally, it should be noted that, as Figure 2 As shown, the trend prediction result of data quality refers to the predicted value sequence or trend prediction curve of data quality indicators (such as the number of outliers, the outlier rate, etc.) within a future time window, which is used to characterize the changing trend of data quality over a period of time.

[0039] In one feasible implementation, step S20 includes: Step S201: Align the business behavior features, time-series fluctuation features, and regulatory policy features in the multimodal features to obtain a joint feature matrix.

[0040] It should be noted that the system performs spatiotemporal alignment processing on features from different modalities, concatenating and combining features from various dimensions according to a unified time step and sample dimension to construct a unified feature representation containing multi-dimensional information, facilitating subsequent joint analysis by the model. Alignment refers to unifying feature data from different sources and with different time granularities to the same time step and sample dimension, including timestamp alignment, missing value imputation, and dimension unification.

[0041] Additionally, it should be noted that the joint feature matrix refers to the matrix formed by concatenating the business behavior feature vector, the time series fluctuation feature vector, and the regulatory policy feature vector according to the feature dimensions.

[0042] Step S202: Input the joint feature matrix into the preset trend prediction model, and perform forward propagation processing through the first long short-term memory network layer and the second long short-term memory network layer of the preset trend prediction model in sequence to obtain the normalized outlier prediction value. The time step of the first long short-term memory network layer is greater than the time step of the second long short-term memory network layer.

[0043] It should be noted that the system inputs the joint feature matrix into a two-layer long short-term memory network model. The first layer of the network performs preliminary temporal feature extraction on the input features, and the second layer of the network further abstracts and compresses the extracted features, finally outputting the normalized outlier prediction value. This improves the prediction accuracy by utilizing the hierarchical feature learning capability of deep networks.

[0044] Additionally, it's important to note that the Long Short-Term Memory (LSTM) network layer is a special type of recurrent neural network layer. Through gating mechanisms (input gate, forget gate, output gate), it controls the flow of information, effectively solving the gradient vanishing problem of traditional recurrent neural networks and thus capturing long-term temporal dependencies. The time step of the first LSTM network layer is set to 1-2 times the time step of the second LSTM network layer. For example, based on historical data from the past 60 months, the time step of the first LSTM layer is set to 64, and the time step of the second LSTM layer is set to 32, to balance the model's fitting ability and computational efficiency. During model training, a Dropout strategy is used to randomly discard some neurons to prevent overfitting. The optimizer is the Adam adaptive learning rate optimizer, and the loss function is the mean squared error (MSE). Since the sample size of the data for quality monitoring is usually small, the number of training epochs can be set to 30 to balance sufficiency of fit and the risk of overfitting.

[0045] Step S203: Perform inverse normalization on the predicted outlier values ​​to obtain the trend prediction results of data quality.

[0046] It should be noted that the system will denormalize the normalized outlier predictions output by the model, mapping them back from the normalized space to the original business indicator space, thus obtaining trend prediction results that can be directly used for business decision-making. During model training, input data is typically normalized to a uniform numerical range to accelerate convergence; therefore, the prediction output also needs corresponding denormalization to restore it to values ​​with practical business meaning.

[0047] Step S30: Determine the warning type based on the trend prediction result. The warning type includes at least one of fluctuation warning, peak warning and long-term trend warning.

[0048] It should be noted that the system will compare the trend prediction results with the preset warning thresholds, identify the types of abnormal patterns in data quality based on the comparison results, thereby distinguishing different types of data quality risks and providing a basis for subsequent differentiated rule adjustment strategies.

[0049] Additionally, it should be noted that the warning type refers to the category of data quality anomaly patterns identified based on trend prediction results, including at least one of fluctuation warnings, peak warnings, and long-term trend warnings. Fluctuation warnings indicate sudden, instantaneous abnormal fluctuations in data quality indicators, typically caused by sudden system anomalies or operational errors. Peak warnings indicate significant peaks in data quality indicators within a specific time period (such as at the end of the month or quarter, during peak business periods), usually related to the periodic aggregation of business volume. Long-term trend warnings indicate a continuous deterioration trend in data quality indicators over a longer time window, typically related to deeper causes such as business changes or system aging.

[0050] Step S40: Adjust the verification parameters in the configurable verification rules according to the warning type to obtain the adjusted verification rules.

[0051] It should be noted that the system will dynamically adjust the corresponding verification parameters in the configurable verification rules based on the determined alert type. Configurable verification rules are defined as hot-loadable configuration items, where the verification parameters are no longer fixed values ​​but variable parameters that can be modified in real time by external systems. This means that by adjusting the verification parameters according to the alert type, the verification rules can adaptively evolve, enabling them to flexibly respond to different risk profiles. Specifically, verification parameters refer to dynamically adjustable numerical parameters in the rules, including but not limited to alarm thresholds, rule weights, execution actions, verification windows, and sampling rates.

[0052] Step S50: Perform quality verification on the target data according to the adjusted verification rules to obtain the verification result.

[0053] It should be noted that the system will apply the adjusted verification rules to the target data, automatically verifying the data according to the verification logic and parameters defined in the rules, thereby determining whether the data meets the quality requirements and outputting the corresponding verification results. The verification results refer to the judgment results obtained after quality verification of the target data, used to characterize whether the data quality meets the standards.

[0054] In one feasible implementation, step S50 includes: Step S501: Determine the data to be verified from the target data according to the applicable data field in the adjusted verification rules.

[0055] It should be noted that the applicable data domain refers to the scope of business data to which the verification rule applies, used to define the application boundaries of the rule. In this step, the system will filter out the data to be verified corresponding to the verification rule from all target data based on the applicable data domain (such as deposit business domain, loan business domain, etc.) defined in the adjusted verification rule, so as to achieve precise data range positioning.

[0056] It is understandable that different validation rules may apply to different data domains. For example, rules for deposit business may only apply to deposit business data, while rules for loan business may only apply to loan business data. Filtering the applicable data domain can avoid performing invalid validations on irrelevant data.

[0057] Step S502: Construct constraints based on the verification logic and verification parameters in the adjusted verification rules.

[0058] It should be noted that the system will combine the validation logic in the validation rules (such as field not empty, regular expression matching, value range, etc.) with the adjusted validation parameters (such as alarm threshold, minimum sample size, etc.) to construct specific constraints for determining whether the data meets the quality requirements.

[0059] It is understood that the validation logic refers to the data quality inspection logic expression defined in the rules, including but not limited to non-empty validation, regular expression validation, numerical range validation, enumerated value validation, cross-field consistency validation, etc. For example, "field [counterpartner name] is not empty and matches the regular expression". Constraints refer to the quality rules that the data must meet, which are jointly limited by the validation logic and validation parameters, such as field not being empty, field value matching the regular expression, and numerical value being within a specified range.

[0060] Step S503: When the data to be verified meets the constraints, the verification result is determined to be of normal quality.

[0061] It should be noted that in this step, the system will compare the data to be verified with the constraints. When all the indicators of the data to be verified meet the constraints, the data quality is determined to be normal, thus indicating that the data meets the quality requirements and can enter the downstream processing flow.

[0062] Step S504: When the data to be verified does not meet the constraints, the verification result is determined to be of abnormal quality.

[0063] It should be noted that when there are indicators in the data to be verified that do not meet the constraints, the system will determine that the data quality is abnormal, thereby indicating that there is a quality problem with this part of the data. It is necessary to take corresponding actions according to the execution strategy of the verification rules. The execution strategy includes blocking, alarm only, and delayed verification.

[0064] This embodiment provides a data quality verification method. The method includes: acquiring multimodal features of target data; performing data quality analysis on the multimodal features according to a preset trend prediction model to obtain a trend prediction result for data quality; determining an early warning type based on the trend prediction result, wherein the early warning type includes at least one of fluctuation early warning, peak early warning, and long-term trend early warning; adjusting the verification parameters in the configurable verification rules according to the early warning type to obtain adjusted verification rules; and performing quality verification on the target data according to the adjusted verification rules to obtain a verification result. In summary, this embodiment improves trend prediction accuracy through multimodal feature fusion and dynamically adjusts the verification rules according to differentiated early warning types, achieving adaptive dynamic adjustment of the verification rules, reducing the risk of missed and false alarms, and improving the accuracy and flexibility of data quality verification.

[0065] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the data quality verification method of this application. Step S30 includes: Step S301: Determine the deviation value of the trend prediction based on the predicted value of the quality index in the trend prediction result and the preset true value.

[0066] It should be noted that the system calculates the deviation between the predicted quality indicators for a future time window output by the trend prediction model and the actual observed quality indicator values ​​for that time window. This deviation quantifies the degree of discrepancy between the predicted and actual values ​​and is a fundamental metric for determining the existence and severity of quality risks.

[0067] Additionally, it should be noted that the predicted quality index values ​​refer to the quality index values ​​output by the trend prediction model within a future time window. Quality indices include, but are not limited to, the number of outliers, outlier rate, missing rate, and duplication rate. The preset true values ​​refer to the historical quality index values ​​actually observed within the corresponding time window. The deviation value can be calculated using absolute deviation, relative deviation, or standardized deviation.

[0068] Step S302: When the deviation value is greater than the preset fluctuation warning threshold, determine the warning type as fluctuation warning.

[0069] It should be noted that the system compares the calculated deviation value with a preset fluctuation warning threshold. When the deviation value exceeds this threshold, it determines that the current data quality has experienced a sudden abnormal fluctuation, thereby promptly identifying drastic changes in data quality indicators within a short period. Generally, such drastic changes are usually caused by sudden system failures, abnormal upstream data sources, or human error, requiring temporary and relatively strict verification measures to prevent the spread of abnormal data.

[0070] Additionally, it should be noted that the preset fluctuation warning threshold is a pre-set deviation threshold used to determine whether abnormal fluctuations exist. This threshold can be set based on the normal fluctuation range in historical data, for example, set as the mean of historical deviation values ​​plus three standard deviations.

[0071] Step S303: When the maximum value in the trend prediction result is greater than the preset peak warning threshold, the warning type is determined to be a peak warning.

[0072] It should be noted that the system will obtain the maximum value of the trend prediction result within a future time window and compare this maximum value with a preset peak warning threshold. When the maximum value exceeds the preset peak warning threshold, it indicates that the business volume will experience a periodic peak within the future time window, and the warning type will be determined as a peak warning. Understandably, peak warnings correspond to predictable, periodic peaks in business volume, such as the reporting peaks at the end of the month, quarter, or year. During these periods, data processing pressure increases, and the false alarm rate of non-core rules may rise, requiring differentiated rule adjustment strategies.

[0073] Additionally, it should be noted that the preset peak warning threshold is a critical value used to determine whether a significant peak will occur in data quality. This threshold is usually set based on the cyclical patterns of business operations and historical peak data.

[0074] Step S304: If the deviation value is greater than the preset long-term trend warning threshold within a consecutive preset time window, the warning type is determined to be a long-term trend warning.

[0075] It should be noted that the system monitors the changes in deviation values ​​over multiple consecutive time windows. When the deviation value in each of the preset number of time windows exceeds the preset long-term trend warning threshold, it indicates a long-term and continuous deterioration trend in data quality, and the warning type is determined as a long-term trend warning. Understandably, a long-term trend warning corresponds to a slow but continuous decline in quality. This trend is usually caused by deeper reasons such as changes in business rules, aging system architecture, or continuous deterioration of data source quality, requiring permanent rule reconstruction and evolution measures.

[0076] Additionally, it should be noted that the "preset number of consecutive time windows" refers to the number of consecutive monitoring periods used to determine long-term trends, such as 3 or 5 consecutive time windows. This number is set according to the business scenario's requirements for trend sensitivity. The preset long-term trend warning threshold is a critical value used to determine whether data quality shows a continuous deterioration trend. This threshold is usually lower than the fluctuation warning threshold to identify gradual changes rather than sudden fluctuations.

[0077] In this embodiment, by comparing the trend prediction results with multi-level early warning thresholds, the continuous quality indicator prediction values ​​are classified into various discrete early warning types such as fluctuation early warning, peak early warning, and long-term trend early warning. This completes the accurate identification and classification of data quality risk patterns, solves the problem that different types of data quality risks are difficult to differentiate and handle in a targeted manner, and improves the matching accuracy and response efficiency of subsequent rule adjustment strategies and risk patterns.

[0078] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the data quality verification method of this application. Step S40 includes: Step S401: When the warning type is fluctuation warning, increase the alarm threshold and execution priority of the verification parameters in the configurable verification rules to obtain the adjusted verification rules.

[0079] It should be noted that when the alert type is a fluctuation alert, the system will automatically identify the fields affected by the fluctuation, temporarily adjust the alarm threshold of the relevant verification rules to a stricter level, increase the execution priority of the relevant rules, and change the execution action to blocking mode to prevent abnormal data from flowing into the downstream link. When the fluctuation alert is lifted and the quality indicators return to the normal range, the system will automatically roll back the above parameters to the version before the fluctuation, without manual intervention.

[0080] Additionally, it should be noted that increasing the alarm threshold means raising the quality judgment threshold of the verification rule to judge the data to be verified with stricter standards. For example, increasing the non-empty rate threshold from 90% to 100% means that the data must be completely non-empty to pass the verification. Increasing the execution priority means moving the execution order of the rule to the beginning in the verification task queue to prioritize the verification of the data quality of fields affected by fluctuations. Blocking mode refers to the execution strategy of preventing abnormal data from entering the downstream processing flow when the verification fails.

[0081] Step S402: When the warning type is a peak warning, adjust the alarm threshold and execution priority of the verification parameters according to the supervision type of the corresponding field in the configured verification rule to obtain the adjusted verification rule.

[0082] It should be noted that when the alert type is determined to be a peak alert, the system will implement differentiated adjustments to the verification parameters based on the regulatory importance of the fields targeted by each verification rule. It is understandable that peak alerts correspond to periodic peak business volumes, during which non-core rules are prone to generating a large number of false alarms due to the surge in data volume. Therefore, differentiated verification strategies are needed for different types of regulatory fields to reduce false alarms related to non-core data quality anomalies while ensuring the quality of core data.

[0083] Additionally, it should be noted that regulatory type refers to the category based on the importance of fields in regulatory reporting, including core regulatory fields and non-core regulatory fields. Core regulatory fields are those directly related to regulatory reporting requirements and have a critical impact on regulatory compliance, such as transaction amounts and customer identity information; their data quality directly affects the accuracy of regulatory reporting. Non-core regulatory fields refer to fields with relatively limited impact on regulatory compliance; their validation rules can be appropriately relaxed during peak business periods to reduce false alarms.

[0084] In one feasible implementation, step S402 includes: Step A10: When the warning type is a peak warning, determine the regulatory type of the corresponding field according to the regulatory flag of the corresponding field in the configurable verification rule.

[0085] It should be noted that when the alert type is a peak alert, the system reads the regulatory tag of the corresponding field in the configured verification rules, identifies the regulatory type of the field based on this tag, and determines the subsequent threshold adjustment strategy. Additionally, it should be noted that the regulatory tag is a field-level identifier extracted from business metadata during the feature extraction phase, used to mark whether a field belongs to a key regulatory field. The regulatory type of a field can be directly mapped and determined by the regulatory tag; for example, a field marked "highly sensitive" is mapped to a core regulatory field, and a field marked "lowly sensitive" is mapped to a non-core regulatory field.

[0086] Step A20: When the regulatory type of the corresponding field is a core regulatory field, maintain the alarm threshold of the corresponding field in the configurable verification rule.

[0087] It should be noted that when the field's regulatory type is a core regulatory field, the system will maintain the alarm threshold of its validation rules unchanged. Specifically, maintaining the alarm threshold means choosing to maintain the existing threshold level based on actual business needs, for example, keeping the non-empty rate threshold unchanged from 90%.

[0088] Step A30: When the regulatory type of the corresponding field is a non-core regulatory field, increase the alarm threshold of the corresponding field in the configurable verification rule.

[0089] It should be noted that when the field's regulatory type is a non-core regulatory field, the system will increase the alarm threshold of its verification rules, for example, by raising the trigger threshold from 85% to 95%, thereby reducing false alarms of non-core fields caused by business congestion and avoiding the consumption of monitoring resources by false alarm storms.

[0090] Step S403: When the warning type is a long-term trend warning, obtain the historical verification result sequence of the configurable verification rule, and update the alarm threshold of the verification parameter in the configurable verification rule according to the historical verification result sequence to obtain the adjusted verification rule.

[0091] It should be noted that when the warning type is a long-term trend warning, the system will obtain the historical verification result sequence of the verification rule, analyze the changing trend of the historical data, generate a new baseline threshold through extrapolation as the updated alarm threshold, and replace the original alarm threshold with the baseline threshold, so that the verification threshold can reflect the long-term evolution trend of data quality.

[0092] Additionally, it should be noted that the historical verification result sequence refers to the time-series data of quality indicators such as the pass rate and anomaly rate of the verification rule within a historical time window. Extrapolating to generate a new baseline threshold means predicting the future normal quality level based on the changing trend of the historical verification result sequence using trend extrapolation methods (such as linear extrapolation, exponential smoothing, etc.), and setting this predicted value as the new alarm threshold. For example, if the anomaly rate of a certain rule has gradually increased from 1% to 3% in the past 6 months, the system can extrapolate and predict that it will continue to rise, setting the new baseline threshold at 4% to adapt to the long-term changing trend of data quality.

[0093] In this embodiment, by adopting differentiated parameter adjustment strategies for different warning types, the verification rules are adaptively and dynamically adjusted to three types of risks: sudden fluctuations, periodic peaks, and long-term deterioration. This solves the problem that static rules cannot flexibly adapt to different risk forms and improves the adaptability and accuracy of the verification rules in different business scenarios.

[0094] This application also provides a data quality verification device, please refer to... Figure 5 The data quality verification device includes: Feature extraction module 10 is used to obtain multimodal features of the target data; The trend prediction module 20 is used to perform data quality analysis on the multimodal features according to a preset trend prediction model, and obtain the trend prediction result of the data quality. The early warning identification module 30 is used to determine the early warning type based on the trend prediction result, wherein the early warning type includes at least one of fluctuation early warning, peak early warning and long-term trend early warning; The parameter adjustment module 40 is used to adjust the verification parameters in the configurable verification rules according to the warning type, so as to obtain the adjusted verification rules; The quality verification module 50 is used to perform quality verification on the target data according to the adjusted verification rules and obtain the verification result.

[0095] The data quality verification device provided in this application, employing the data quality verification method described in the above embodiments, can solve the technical problem of how to improve the accuracy and flexibility of verification during data quality monitoring. Compared with the prior art, the beneficial effects of the data quality verification device provided in this application are the same as those of the data quality verification method described in the above embodiments, and other technical features in the data quality verification device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0096] In one embodiment, the feature extraction module 10 is further configured to acquire target data and extract business behavior features from the data lineage, business metadata, and historical data quality records of the target data. The business behavior features include data source complexity, cross-system correlation density, regulatory field markers, and information entropy values ​​of enumerated fields. The module also performs trend decomposition on the historical quality indicators of the target data according to a preset time window to obtain the time-series fluctuation features for each time window. The time-series fluctuation features include periodic autocorrelation coefficients, frequency domain features, moving averages, and year-on-year and month-on-month growth rates. Finally, the module performs text vectorization on the regulatory policy text in the target data to obtain regulatory policy features, which include regulatory system change vectors, economic indicator deviation vectors, and internal business strategy label vectors.

[0097] In one embodiment, the trend prediction module 20 is further configured to align the business behavior features, time-series fluctuation features, and regulatory policy features in the multimodal features to obtain a joint feature matrix; input the joint feature matrix into a preset trend prediction model, and perform forward propagation processing through the first long short-term memory network layer and the second long short-term memory network layer of the preset trend prediction model in sequence to obtain a normalized outlier prediction value, wherein the time step of the first long short-term memory network layer is greater than the time step of the second long short-term memory network layer; and perform inverse normalization processing on the outlier prediction value to obtain the trend prediction result of data quality.

[0098] In one embodiment, the early warning identification module 30 is further configured to determine the deviation value of the trend prediction based on the predicted value of the quality indicator in the trend prediction result and the preset true value; when the deviation value is greater than the preset fluctuation early warning threshold, determine the early warning type as a fluctuation early warning; when the maximum value in the trend prediction result is greater than the preset peak early warning threshold, determine the early warning type as a peak early warning; and when the deviation value is greater than the preset long-term trend early warning threshold within a consecutive preset time window, determine the early warning type as a long-term trend early warning.

[0099] In one embodiment, the parameter adjustment module 40 is further configured to: increase the alarm threshold and execution priority of the verification parameters in the configurable verification rules when the warning type is a fluctuation warning, to obtain an adjusted verification rule; adjust the alarm threshold and execution priority of the verification parameters according to the regulatory type of the corresponding field in the configurable verification rules when the warning type is a peak warning, to obtain an adjusted verification rule; and obtain the historical verification result sequence of the configurable verification rules when the warning type is a long-term trend warning, and update the alarm threshold of the verification parameters in the configurable verification rules according to the historical verification result sequence, to obtain an adjusted verification rule.

[0100] In one embodiment, the parameter adjustment module 40 is further configured to: determine the regulatory type of the corresponding field according to the regulatory flag of the corresponding field in the configurable verification rule when the warning type is a peak warning; maintain the alarm threshold of the corresponding field in the configurable verification rule when the regulatory type of the corresponding field is a core regulatory field; and increase the alarm threshold of the corresponding field in the configurable verification rule when the regulatory type of the corresponding field is a non-core regulatory field.

[0101] In one embodiment, the quality verification module 50 is further configured to determine the data to be verified from the target data according to the applicable data domain in the adjusted verification rules; construct constraints according to the verification logic and verification parameters in the adjusted verification rules; determine the verification result as normal quality when the data to be verified meets the constraints; and determine the verification result as abnormal quality when the data to be verified does not meet the constraints.

[0102] This application provides a data quality verification device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the data quality verification method in the first embodiment described above.

[0103] The following is for reference. Figure 6The diagram illustrates a structural schematic of a data quality verification device suitable for implementing embodiments of this application. The data quality verification device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), tablets, PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The data quality verification device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0104] like Figure 6 As shown, the data quality verification device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the data quality verification device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the data quality verification device to communicate wirelessly or wiredly with other devices to exchange data. Although data quality verification devices with various systems are shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0105] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0106] The data quality verification device provided in this application, employing the data quality verification method described in the above embodiments, can solve the technical problem of how to improve the accuracy and flexibility of verification during data quality monitoring. Compared with the prior art, the beneficial effects of the data quality verification device provided in this application are the same as those of the data quality verification method provided in the above embodiments, and other technical features of this data quality verification device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0107] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0108] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0109] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the data quality verification method described in the above embodiments.

[0110] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0111] The aforementioned computer-readable storage medium may be included in a data quality verification device; or it may exist independently and not be assembled into a data quality verification device.

[0112] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a data quality verification device, the data quality verification device performs the following actions: acquires multimodal features of the target data; performs data quality analysis on the multimodal features according to a preset trend prediction model to obtain a trend prediction result for the data quality; determines an early warning type based on the trend prediction result, wherein the early warning type includes at least one of fluctuation early warning, peak early warning, and long-term trend early warning; adjusts the verification parameters in the configurable verification rules according to the early warning type to obtain adjusted verification rules; and performs quality verification on the target data according to the adjusted verification rules to obtain a verification result.

[0113] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0115] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0116] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described data quality verification method. This solves the technical problem of how to improve the accuracy and flexibility of verification during data quality monitoring. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the data quality verification method provided in the above embodiments, and will not be repeated here.

[0117] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data quality verification method described above.

[0118] The computer program product provided in this application can solve the technical problem of how to improve the accuracy and flexibility of verification in the process of data quality monitoring. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the data quality verification method provided in the above embodiments, and will not be repeated here.

[0119] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for verifying data quality, characterized in that, The method includes: Obtain multimodal features from the target data; Data quality analysis is performed on the multimodal features based on a preset trend prediction model to obtain trend prediction results for data quality; The warning type is determined based on the trend prediction results, and the warning type includes at least one of fluctuation warning, peak warning and long-term trend warning. The verification parameters in the configurable verification rules are adjusted according to the warning type to obtain the adjusted verification rules; The target data is quality-verified according to the adjusted verification rules to obtain the verification results.

2. The method as described in claim 1, characterized in that, The multimodal features include business behavior features, time-series fluctuation features, and regulatory policy features. The steps for obtaining the multimodal features of the target data include: Acquire target data and extract business behavior features from the data lineage, business metadata, and historical data quality records of the target data. The business behavior features include data source complexity, cross-system association density, regulatory field markings, and information entropy values ​​of enumerated fields. Based on a preset time window, the historical quality indicators of the target data are decomposed to obtain the time-series fluctuation characteristics of each time window. The time-series fluctuation characteristics include the periodic autocorrelation coefficient, frequency domain characteristics, moving average, and year-on-year and month-on-month growth rates. The regulatory policy text in the target data is processed into text vectorization to obtain regulatory policy features, which include regulatory system change vectors, economic indicator deviation vectors, and internal business strategy label vectors.

3. The method as described in claim 1, characterized in that, The step of performing data quality analysis on the multimodal features based on a preset trend prediction model to obtain the trend prediction result of data quality includes: The business behavior features, time-series fluctuation features, and regulatory policy features in the multimodal features are aligned to obtain a joint feature matrix; The joint feature matrix is ​​input into a preset trend prediction model, and forward propagation is performed sequentially through the first long short-term memory network layer and the second long short-term memory network layer of the preset trend prediction model to obtain a normalized outlier prediction value. The time step of the first long short-term memory network layer is greater than the time step of the second long short-term memory network layer. The predicted outlier values ​​are inversely normalized to obtain the trend prediction results for data quality.

4. The method as described in claim 1, characterized in that, The step of determining the early warning type based on the trend prediction result includes: Based on the predicted values ​​of quality indicators in the trend prediction results and the preset actual values, the deviation value of the trend prediction is determined; When the deviation value is greater than the preset fluctuation warning threshold, the warning type is determined to be a fluctuation warning; When the maximum value in the trend prediction result is greater than the preset peak warning threshold, the warning type is determined to be a peak warning; If the deviation value is greater than the preset long-term trend warning threshold within a consecutive preset time window, the warning type is determined to be a long-term trend warning.

5. The method as described in claim 1, characterized in that, The step of adjusting the verification parameters in the configurable verification rules according to the warning type to obtain the adjusted verification rules includes: When the warning type is fluctuation warning, the alarm threshold and execution priority of the verification parameters in the configurable verification rules are increased to obtain the adjusted verification rules; When the warning type is a peak warning, the alarm threshold and execution priority of the verification parameters are adjusted according to the supervision type of the corresponding field in the configurable verification rules to obtain the adjusted verification rules; When the warning type is a long-term trend warning, the historical verification result sequence of the configurable verification rule is obtained, and the alarm threshold of the verification parameter in the configurable verification rule is updated according to the historical verification result sequence to obtain the adjusted verification rule.

6. The method as described in claim 5, characterized in that, When the warning type is a peak warning, the step of adjusting the alarm threshold and execution priority of the verification parameters according to the supervision type of the corresponding field in the configured verification rule to obtain the adjusted verification rule includes: When the warning type is a peak warning, the regulatory type of the corresponding field is determined according to the regulatory flag of the corresponding field in the configurable verification rules; When the regulatory type of the corresponding field is a core regulatory field, the alarm threshold of the corresponding field in the configurable verification rule is maintained; When the regulatory type of the corresponding field is a non-core regulatory field, the alarm threshold of the corresponding field in the configurable verification rule is increased.

7. The method as described in claim 1, characterized in that, The step of performing quality verification on the target data according to the adjusted verification rules and obtaining the verification result includes: The data to be verified is determined from the target data according to the applicable data fields in the adjusted verification rules; Construct constraints based on the verification logic and verification parameters in the adjusted verification rules; When the data to be verified meets the constraints, the verification result is determined to be of normal quality. When the data to be verified does not meet the constraints, the verification result is determined to be of poor quality.

8. A data quality verification device, characterized in that, The device includes: The feature extraction module is used to obtain multimodal features of the target data; The trend prediction module is used to perform data quality analysis on the multimodal features according to a preset trend prediction model, and obtain the trend prediction result of the data quality. The early warning identification module is used to determine the early warning type based on the trend prediction result, wherein the early warning type includes at least one of fluctuation early warning, peak early warning and long-term trend early warning; The parameter adjustment module is used to adjust the verification parameters in the configurable verification rules according to the warning type, so as to obtain the adjusted verification rules; The quality verification module is used to perform quality verification on the target data according to the adjusted verification rules and obtain the verification result.

9. A data quality verification device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the data quality verification method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the data quality verification method as described in any one of claims 1 to 7.

Images