Data checking method and device, computer equipment and storage medium
By dynamically adjusting the threshold range, the problems of false alarms and missed alarms in business data verification are solved, achieving higher verification accuracy and adaptability, supporting heterogeneous database verification, reducing labor costs and providing real-time feedback.
Patent Information
- Application Number
- CN202511797698.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-02
- Publication Date
- 2026-03-06
AI Technical Summary
Existing technologies are prone to false alarms and omissions when checking business data, and cannot effectively adapt to the dynamic distribution and changing characteristics of data.
By determining the type of target data, the corresponding threshold range is dynamically adjusted based on historical datasets, and the threshold range is updated after verification. The historical dataset is processed using a large language model or a preset algorithm to determine the accurate threshold range, and alarm information is generated to report the anomaly level.
It significantly reduces the probability of false alarms and missed errors, improves the accuracy and adaptability of data verification, supports joint verification of heterogeneous databases, reduces the workload of manual maintenance, and realizes real-time monitoring and alarms.
Smart Images
Figure CN121614465A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a data verification method, apparatus, computer equipment, and storage medium. Background Technology
[0002] In some enterprises, it is necessary to regularly check (verify and reconcile) the business data stored in the database. This is to ensure the accuracy of the data and to statistically analyze the operational status of the business. This need for data verification is particularly frequent in certain industries, such as insurance and banking, which involve finance.
[0003] In related technologies, for each type of business data that needs to be checked, a fixed threshold range is usually used for comparison to check whether the business data is compliant; however, business data has the characteristics of dynamic distribution and dynamic change, so false alarms or omissions often occur when checking business data.
[0004] Therefore, how to reduce the probability of false and false reports when checking business data is an urgent problem to be solved. Summary of the Invention
[0005] Therefore, it is necessary to provide a data verification method, apparatus, computer equipment, and storage medium that can reduce the probability of false alarms and omissions when verifying business data, in order to address the above-mentioned technical problems.
[0006] Firstly, this application provides a data verification method, including:
[0007] When determining the target data, the target type of the target data is determined;
[0008] Based on the correspondence between data types and threshold intervals, a target threshold interval corresponding to the target type is determined; wherein each threshold interval is determined based on the historical dataset of the corresponding data type.
[0009] The target data is checked based on the target threshold range to obtain the check result;
[0010] The target data is used as a historical dataset to update the historical dataset corresponding to the target type, and the target threshold range is updated based on the updated historical dataset.
[0011] In one embodiment, the method further includes:
[0012] A query statement is constructed based on query information; the query information includes a data address and processing rules, and the query statement represents the processing of source data obtained from the data address to obtain target data based on the processing rules;
[0013] Execute the query statement to determine the target data.
[0014] In one embodiment, the method further includes:
[0015] For each of the data types, obtain the historical dataset corresponding to that data type;
[0016] The threshold range corresponding to the data type is determined based on the historical dataset.
[0017] In one embodiment, determining the threshold range corresponding to the data type based on the historical dataset includes:
[0018] The historical dataset is processed using various preset algorithms to obtain each candidate threshold interval;
[0019] In response to the user's selection operation, the threshold interval corresponding to the data type is determined from each of the candidate threshold intervals.
[0020] In one embodiment, determining the threshold range corresponding to the data type based on the historical dataset includes:
[0021] The historical dataset is processed using a large language model to obtain the threshold range corresponding to the data type output by the large language model.
[0022] In one embodiment, the check result includes normal and abnormal, and the method further includes:
[0023] When the inspection result indicates that the target data is abnormal, the abnormality level of the target data is determined according to the relationship between the target data and the target threshold range;
[0024] Based on the anomaly level, a handling recommendation is determined.
[0025] In one embodiment, the method further includes:
[0026] An alarm message is generated based on the target type, the target data, the target threshold range, the anomaly level, and the processing suggestion.
[0027] Send the alarm information to the target address.
[0028] Secondly, this application also provides a data verification device, the device comprising a type determination module, a threshold determination module, a verification module, and an update module, wherein:
[0029] The type determination module is used to determine the target type of the target data when determining the target data;
[0030] The threshold determination module is used to determine a target threshold range corresponding to the target type based on the correspondence between data types and threshold ranges; wherein each threshold range is determined based on the historical dataset of the corresponding data type.
[0031] The inspection module is used to inspect the target data based on the target threshold range and obtain the inspection result;
[0032] The update module is used to update the historical dataset corresponding to the target type by using the target data as a historical dataset, and to update the target threshold range based on the updated historical dataset.
[0033] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the data verification method as described in any one of the first aspects above.
[0034] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the data verification method as described in any one of the first aspects above.
[0035] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data verification method as described in any one of the first aspects above.
[0036] The aforementioned data verification method, apparatus, computer equipment, and storage medium, when determining a target data, determine a target threshold range for verification based on the target data's target type. Since each data type corresponds to a threshold range, and the threshold range is determined based on the historical dataset corresponding to that data type, each threshold range can relatively accurately reflect the range of change of the target data of that data type. Therefore, the target data determined based on the target threshold range can obtain more accurate verification results, thereby reducing the probability of false alarms and missed alarms.
[0037] Furthermore, for each target data, after the target data has been checked, the target data is updated as a historical data record to update its corresponding historical dataset, thereby updating the corresponding target threshold range. In this way, the threshold range corresponding to each data type can be dynamically updated as the data changes, making each threshold range more consistent with the dynamic distribution and dynamic change characteristics of the corresponding data type, thus further improving the accuracy of data checking. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0039] Figure 1 This is a diagram illustrating the application environment of a data verification method in one embodiment;
[0040] Figure 2 This is a flowchart illustrating a data verification method in one embodiment;
[0041] Figure 3 This is a schematic diagram of the process for generating alarm information in one embodiment;
[0042] Figure 4 This is a structural block diagram of a data verification device in one embodiment;
[0043] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0045] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.
[0046] The data verification method provided in this application embodiment can be applied to, for example, Figure 1 The application environment is illustrated. The data verification system is configured on a computer device, which communicates with various data sources via a network. The data storage system stores the data that the data verification system needs to process; for example, it can store historical datasets corresponding to various data types. The data storage system can be integrated on a server, or it can be located on a cloud or other network server. The computer device can be a server, which can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.
[0047] In one exemplary embodiment, such as Figure 2 As shown, a data verification method is provided, specifically including the following steps 110-140, wherein:
[0048] Step 110: When determining the target data, determine the target type of the target data.
[0049] In this embodiment of the application, the target data is obtained in response to the execution of a user-created query task. The user creates a query task targeting a database by inputting query information, and the computer device executes the query task, that is, it queries the database based on the query information to obtain the target data. The target type is the data type of the target data, which can be determined based on the query information input by the user.
[0050] The target data can be data directly stored in the database, such as the daily number of paid orders, the number of orders from each province, and the daily number of returns stored in the database. The target data can also be obtained by further calculation / statistical processing based on data retrieved from the database; for example, the target data could be the daily return rate calculated based on the daily number of paid orders and the daily number of returns.
[0051] Step 120: Determine the target threshold interval corresponding to the target type based on the correspondence between data type and threshold interval; wherein each threshold interval is determined based on the historical dataset of the corresponding data type.
[0052] In this embodiment of the application, a threshold range for verification is pre-matched for each data type, and the threshold range corresponding to the target type of the target data is the target threshold range. Specifically, the threshold range corresponding to each data type is determined based on the historical data corresponding to that data type. Here, the historical dataset can refer to the full amount of historical data or historical data within a time range; since different data types have different characteristics that lead to different changes and fluctuations in the data, the range of the historical dataset used to determine the corresponding threshold range can be determined according to the data type.
[0053] For example, a product's average daily orders might be related to promotional activities. During promotional periods, the number of daily orders typically increases significantly due to discounted prices. For this data type of average daily orders, historical average daily order data can be divided into datasets with and without promotions. A threshold range can then be determined for each of these datasets. Furthermore, the presence of a promotion can be determined based on the date the user selects the target data, and a corresponding threshold range can be determined based on the presence or absence of a promotion.
[0054] Step 130: Check the target data based on the target threshold range and obtain the check results.
[0055] In this embodiment of the application, after determining the target threshold range, the target data obtained from the query task is checked against the target threshold range. If the target data is within the target threshold range, a normal check result is obtained; if the target data is not within the target threshold range, an abnormal check result is obtained. When an abnormal check result is obtained, corresponding alarm information is generated based on the data type to alert the user.
[0056] Step 140: Update the historical dataset corresponding to the target type using the target data as the historical dataset, and update the target threshold range based on the updated historical dataset.
[0057] In the embodiments of this application, for each data type, when the corresponding target data is obtained each time the query task corresponding to the data type is executed, the target data is added to the historical dataset corresponding to the data type as a historical data, thereby realizing the update of the historical dataset; since the historical dataset is updated, the threshold range determined based on the historical dataset will also be updated synchronously.
[0058] In the above data verification method, when a target data is determined each time, a target threshold range for verification is determined based on the target data type. Since each data type corresponds to a threshold range, and the threshold range is determined based on the historical dataset corresponding to that data type, each threshold range can relatively accurately reflect the range of change of the target data of that data type. Therefore, the target data determined based on the target threshold range can obtain more accurate verification results, thereby reducing the probability of false alarms and false misses.
[0059] Furthermore, for each target data, after the target data has been checked, the target data is updated as a historical data record to update its corresponding historical dataset, thereby updating the corresponding target threshold range. In this way, the threshold range corresponding to each data type can be dynamically updated as the data changes, making each threshold range more consistent with the dynamic distribution and dynamic change characteristics of the corresponding data type, thus further improving the accuracy of data checking.
[0060] In one embodiment, the process of determining the target data may specifically include: constructing a query statement based on query information and executing the query statement to determine the target data.
[0061] Specifically, the query information includes the data address and processing rules. The query statement represents the processing of the source data obtained from the data address based on the processing rules to obtain the target data. In this embodiment, users can create SQL (Structured Query Language) scripts through the visual graphical interface of the data verification system, or directly write SQL scripts; of course, users can also directly input query information in text form, and then use the LLM (Large Language Model) model to process the query information and generate the corresponding SQL script for data query. The query task is created in response to the generation of the SQL script and executed in response to user triggering.
[0062] Specifically, the data verification system employs a federated query engine (such as Presto, Trino, or Spark SQL) to support cross-database joins. Its data connection and adaptation layers support various heterogeneous data sources, including Oracle, Hive, MySQL, and PostgreSQL, facilitating unified data querying across multiple data sources. The system provides a unified data connection configuration and SQL execution interface for automatic metadata collection and synchronization within each data source. Furthermore, the system's engine supports complex SQL syntax logic (such as multi-table joins, aggregations, and window functions), automatically resolving data dependencies triggered by the user's graphical interface to generate query tasks.
[0063] In one embodiment, the process of generating the corresponding threshold range for each data type may specifically include: for each data type, obtaining the historical dataset corresponding to the data type, and determining the threshold range corresponding to the data type based on the historical dataset.
[0064] Specifically, for each data type, the corresponding historical dataset is obtained. The time range of historical data contained in the historical dataset can be determined by the time range input by the user, or it can be determined according to a pre-defined mapping table between data types and time ranges. There are at least two ways to determine the threshold range based on the historical dataset, which are described below.
[0065] The first method for determining the threshold range based on historical datasets includes: processing the historical datasets using various preset algorithms to obtain candidate threshold intervals; and, in response to the user's selection operation, determining the threshold interval corresponding to the data type from the candidate threshold intervals.
[0066] Specifically, each preset algorithm can be configured as an independent model. In this embodiment, the model corresponding to each preset algorithm may include at least an SPC (Statistical Process Control) model, a business rule model, and a time series prediction model (such as ARIMA or Prophet). The SPC model is used to calculate the upper limit (UCL) and lower limit (LCL) of the threshold interval based on historical datasets. When processing historical datasets, the time series prediction model is used to predict data changes and set dynamic boundaries, which serve as the endpoints of the threshold intervals. The business rule engine dynamically filters data from historical datasets based on user settings and the context of business logic (such as holidays and promotional activities), and dynamically adjusts the endpoints of the threshold intervals based on the filtered data; it also supports threshold interpretation and visualization.
[0067] For a given data type, when determining its corresponding historical dataset, the historical dataset is processed using models corresponding to various preset algorithms to obtain candidate threshold intervals calculated by each model. Then, based on the user's selection, one of these candidate threshold intervals is chosen as the threshold interval corresponding to that data type. Simultaneously, the model corresponding to the threshold interval selected by the user is used as the baseline model for that data type. That is, the user only needs to select once; whenever the historical dataset for that data type is subsequently updated, this baseline model is still used to process the updated historical dataset to generate new threshold intervals.
[0068] The second approach to determining the threshold range based on historical datasets includes processing the historical dataset using a large language model to obtain the threshold range corresponding to the data type output by the large language model.
[0069] Specifically, for each data type, the calculation method, source, and industry of the data corresponding to that data type are used as the attribute information of that data type. Then, a prompt template is constructed from historical data and attribute information. The prompt template is input into the LLM model for processing. Using the data analysis and processing capabilities of the LLM model, the threshold range for that data type is output.
[0070] The above describes two methods for determining threshold ranges based on historical datasets. Users can choose between these two methods, and can also specify that the first method is used to determine the threshold range for the first type of data type, and the second method is used for the second type of data type. However, the specific criteria for classifying the first and second types of data types are not specifically determined in this embodiment.
[0071] In one embodiment, an alarm message needs to be generated for the check results indicating anomalies; such as... Figure 3 The diagram shows the process for generating alarm information, which may specifically include steps 210-230, wherein:
[0072] Step 210: When the inspection results indicate that the target data is abnormal, determine the abnormality level of the target data based on the relationship between the target data and the target threshold range.
[0073] Specifically, for any data type, when the check result of its corresponding target data is determined to be an anomaly, the anomaly level is determined based on the deviation of the target data from the endpoints of the target threshold interval. Specifically, for each data type, continuous gradient intervals are pre-defined along the upper and lower boundaries of its corresponding threshold interval, and then an anomaly level is associated with each gradient interval. When the target data is not located within the corresponding target threshold interval, it is determined whether the target data is located at the upper or lower boundary of the target threshold interval, thereby determining the gradient interval in which the target data is located, and then the anomaly level associated with that gradient interval is used as the anomaly level of the target data.
[0074] In one example, the threshold range corresponding to the data type is [10-30], with an upper boundary endpoint of 30 and a lower boundary endpoint of 10. The upper boundary is divided into gradient intervals of (30-40], (40-50], and (50-∞), corresponding to anomaly levels of mild, moderate, and high anomaly, respectively. The lower boundary is divided into gradient intervals of [0-5) and [5-10), corresponding to anomaly levels of high and mild anomalies, respectively. Therefore, when the target data is 4, the anomaly level can be determined as high anomaly; when the target data is 42, the anomaly level can be determined as moderate anomaly.
[0075] Step 220: Determine the handling recommendations based on the anomaly level.
[0076] Step 230: Generate alarm information based on target type, target data, target threshold range, anomaly level, and handling suggestions.
[0077] Specifically, for each data type, processing suggestions are pre-defined for each anomaly level of that data type. After determining the anomaly level of the target data, the corresponding processing suggestions are matched according to the determined anomaly level.
[0078] The computer device generates alarm information containing target data, the corresponding data type (target type), target threshold range, anomaly level, and processing suggestions. The alarm information can be text-based, voice-based, or image-based; this embodiment does not impose specific limitations on this. Furthermore, after generating the alarm information, it is sent to a target address; for example, the target address could point to a central platform or the equipment of maintenance personnel; that is, the alarm information can be sent via SMS, email, or other communication tools.
[0079] The following content further illustrates the data verification process corresponding to this application through two scenario examples:
[0080] In scenarios where the order table and payment table are being checked for consistency:
[0081] The query task is used to retrieve the target data, the order table, from the Oracle database and the payment table from the Hive database.
[0082] Inspection task: Calculate the difference between the daily order count and payment amount;
[0083] Target threshold range: based on the mean of the differences over the past 30 days ± 3σ (σ is a preset floating value);
[0084] Verification results: The discrepancy exceeds the threshold, indicating that there may be missed orders or duplicate payments.
[0085] In scenarios involving querying the integrity of user registration data:
[0086] The query task is used to retrieve the user table from the MySQL database;
[0087] Inspection task: Calculate the null value rate of the mobile phone number field;
[0088] Target threshold range: Predicts a reasonable range based on historical missing value rate trends;
[0089] The check result showed a sudden increase in the null value rate, indicating that there may be an interface anomaly or data loss.
[0090] The data verification method adopted in the embodiments of this application has at least the following beneficial effects:
[0091] 1. Improve detection accuracy: Significantly reduce false alarm and false negative rates through a dynamic threshold mechanism;
[0092] 2. Enhanced adaptability: The threshold can be automatically adjusted according to data distribution and business cycle, making it highly adaptable;
[0093] 3. Enable cross-database collaboration: Support joint verification of heterogeneous databases, simplify processes, and improve efficiency;
[0094] 4. Reduce labor costs: Reduce the workload of manually setting and maintaining rules, and support large-scale deployment;
[0095] 5. Supports real-time feedback: Enables near real-time monitoring and alarms, improving the response speed to data issues.
[0096] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0097] Based on the same inventive concept, this application also provides a data verification device for implementing the data verification method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more data verification device embodiments provided below can be found in the limitations of the data verification method described above, and will not be repeated here.
[0098] In one exemplary embodiment, such as Figure 4 As shown, a data verification device 400 is provided, including a type determination module 401, a threshold determination module 402, a verification module 403, and an update module 404, wherein:
[0099] The type determination module 401 is used to determine the target type of the target data when determining the target data;
[0100] The threshold determination module 402 is used to determine the target threshold range corresponding to the target type based on the correspondence between data type and threshold range; wherein each threshold range is determined based on the historical dataset of the corresponding data type.
[0101] The inspection module 403 is used to inspect the target data based on the target threshold range and obtain the inspection result;
[0102] The update module 404 is used to update the historical dataset corresponding to the target type by using the target data as the historical dataset, and to update the target threshold range based on the updated historical dataset.
[0103] In the aforementioned data verification device, when a target data is determined each time, a target threshold range for verification is determined based on the target type of the target data. Since each data type corresponds to a threshold range, and the threshold range is determined based on the historical dataset corresponding to that data type, each threshold range can relatively accurately reflect the range of change of the target data of that data type. Therefore, the target data determined based on the target threshold range can obtain more accurate verification results, thereby reducing the probability of false alarms and false misses.
[0104] Furthermore, for each target data, after the target data has been checked, the target data is updated as a historical data record to update its corresponding historical dataset, thereby updating the corresponding target threshold range. In this way, the threshold range corresponding to each data type can be dynamically updated as the data changes, making each threshold range more consistent with the dynamic distribution and dynamic change characteristics of the corresponding data type, thus further improving the accuracy of data checking.
[0105] In one embodiment, the data verification device 400 further includes a query module, which is specifically used for:
[0106] A query statement is constructed based on the query information, which includes the data address and processing rules. The query statement represents the processing of the source data obtained from the data address to obtain the target data based on the processing rules.
[0107] Execute the query statement to determine the target data.
[0108] In one embodiment, the threshold determination module 402 is specifically used for:
[0109] For each data type, obtain the historical dataset corresponding to that data type;
[0110] Determine the threshold range corresponding to the data type based on historical datasets.
[0111] In one embodiment, the threshold determination module 402 is specifically used for:
[0112] The historical dataset is processed using various preset algorithms to obtain the candidate threshold intervals.
[0113] In response to the user's selection, the threshold interval corresponding to the data type is determined from each candidate threshold interval.
[0114] In one embodiment, the threshold determination module 402 is specifically used for:
[0115] By processing historical datasets using a large language model, threshold ranges corresponding to the data types output by the large language model can be obtained.
[0116] In one embodiment, the data verification device 400 further includes an anomaly handling module, which is specifically used for:
[0117] When the inspection results indicate that the target data is abnormal, the abnormality level of the target data is determined according to the relationship between the target data and the target threshold range.
[0118] Determine the appropriate handling recommendations based on the severity level of the anomaly.
[0119] In one embodiment, the exception handling module is specifically used for:
[0120] Based on the target type, target data, target threshold range, anomaly level, and handling suggestions, generate alarm information;
[0121] Send the alarm information to the target address.
[0122] Each module in the aforementioned data verification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0123] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a data verification method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.
[0124] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0125] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described in the above-described data verification method embodiments.
[0126] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described in the above data verification method embodiments.
[0127] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of any of the methods described in the above data verification method embodiments.
[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0129] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0130] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0131] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A data checking method, characterized by, The method comprises: In determining the target data, determining the target type of the target data; According to the corresponding relationship between the data type and the threshold interval, the target threshold interval corresponding to the target type is determined; wherein each threshold interval is determined based on the corresponding historical data set of the data type; Based on the target threshold interval, the target data is checked to obtain a checking result; The target data is used as a historical data set to update the historical data set corresponding to the target type, and the target threshold interval is updated based on the updated historical data set.
2. The method of claim 1, wherein, The method further comprises: Based on the query information, a query statement is constructed; the query information includes data address and processing rules, and the query statement represents processing of source data obtained from the data address based on the processing rules to obtain target data; The query statement is executed to determine the target data.
3. The method of claim 1, wherein, The method further comprises: For each data type, the historical data set corresponding to the data type is obtained; According to the historical data set, the threshold interval corresponding to the data type is determined.
4. The method of claim 3, wherein, According to the historical data set, the threshold interval corresponding to the data type is determined, comprising: Each preset algorithm is used to process the historical data set to obtain each candidate threshold interval; In response to the selection operation of the user, the threshold interval corresponding to the data type is determined from each candidate threshold interval.
5. The method of claim 3, wherein, According to the historical data set, the threshold interval corresponding to the data type is determined, comprising: The historical data set is processed by using a large language model to obtain the threshold interval corresponding to the data type output by the large language model.
6. The method according to any one of claims 1-5, characterized in that, The checking result includes normal and abnormal, and the method further comprises: When the checking result represents that the target data is abnormal, the abnormal level of the target data is determined according to the size relationship between the target data and the target threshold interval; The processing suggestion is determined according to the abnormal level.
7. The method of claim 6, wherein, The method further comprises: According to the target type, the target data, the target threshold interval, the abnormal level and the processing suggestion, an alarm information is generated; The alarm information is sent to the target address.
8. A data checking apparatus, characterized by comprising: The device comprises a type determination module, a threshold determination module, a checking module and an updating module, wherein: The type determination module is used to determine the target type of the target data when the target data is determined; The threshold determination module is used to determine the target threshold interval corresponding to the target type according to the corresponding relationship between the data type and the threshold interval; wherein each threshold interval is determined based on the corresponding historical data set of the data type; The checking module is used to check the target data based on the target threshold interval to obtain a checking result; The updating module is used to update the historical data set corresponding to the target type by using the target data as a historical data set, and update the target threshold interval based on the updated historical data set. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to realize the steps of the method in any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, which is executed by a processor, implements the steps of the method according to any one of claims 1 to 7.