Data processing method and device, computer device, and storage medium

Abstract

The application belongs to the technical field of artificial intelligence, and relates to a data processing method and device, computer equipment and a storage medium, comprising: preprocessing original data collected from a business data source to obtain target processing data; performing causal chain identification on the target processing data to obtain a causal chain identification result; performing time sequence consistency analysis on the target processing data to obtain a consistency analysis result; performing counterfactual reasoning on the target processing data to obtain a counterfactual reasoning result; integrating the causal chain identification result, the consistency analysis result and the counterfactual reasoning result to obtain a target analysis result, and screening abnormal data from the target processing data based on the target analysis result; performing work order generation processing on the abnormal data based on a work order generation strategy and the target analysis result to obtain target verification work orders, and generating a target report based on the abnormal data. The application can be applied to a data verification scene in the field of financial technology, improves the intelligence of data anomaly analysis, and improves the generation efficiency of verification work orders and reports.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology and can be applied to the financial technology field, particularly to data processing methods, devices, computer equipment and storage media. Background Technology

[0002] In the field of digital auditing, data verification is a core component for ensuring data quality, guaranteeing compliance, and implementing effective risk control. Early on, traditional rule-engine-based data verification solutions played a crucial role in data processing, with verification systems commonly employing radial rule matching mechanisms from rule engines such as Drools, LiteFlow, and URule. These solutions could handle explicit logical errors, such as empty fields, format errors, and out-of-bounds values, meeting basic data verification needs in the initial stages of data processing. However, as data has become increasingly complex, diverse, and real-time, the limitations of traditional rule-engine-based data verification solutions have become increasingly apparent. They can only identify explicit logical errors, but are powerless against implicit logical errors, significantly reducing the intelligence of data anomaly analysis. Furthermore, traditional solutions cannot automatically generate verification work orders and reports matching data anomalies. When faced with a large number of data anomalies, manual intervention is required, which is not only inefficient but also prone to omissions and errors.

[0003] In the financial insurance sector, taking insurance product data verification as an example, traditional rule engines struggle to meet practical needs. Traditional methods typically perform simple checks based on basic fields in insurance product data, such as verifying policy number format and premium amounts within specified ranges. However, insurance business involves numerous complex business logics and implicit rules, such as association rules between different insurance products and the relationship between a customer's insurance history and current insurance behavior. Traditional solutions cannot identify these implicit logical errors, potentially leading to undetected abnormal data that impacts the normal operation of insurance business and risk control. For instance, in auto insurance, a customer's vehicle information may not match their historical insurance records. Traditional rule engines may fail to detect such implicit errors, and if not addressed promptly, disputes may arise during subsequent claims processing, causing unnecessary losses for both the insurance company and the customer.

[0004] Therefore, there is an urgent need for a more intelligent and efficient data verification method to improve the accuracy and comprehensiveness of data verification and meet the ever-evolving needs of the digital auditing field. Summary of the Invention

[0005] The purpose of this application is to provide a data processing method, apparatus, computer equipment, and storage medium to solve the technical problems of low intelligence in data anomaly analysis and low efficiency in generating verification work orders and reports in existing data verification methods.

[0006] Firstly, a data processing method is provided, including: Collect raw data from a pre-defined business data source; The raw data is preprocessed to obtain the corresponding target processing data; The target data is subjected to causal chain identification to obtain the corresponding causal chain identification result; Perform time series consistency analysis on the target processing data to obtain the corresponding consistency analysis results; The target data is subjected to counterfactual reasoning to obtain the corresponding counterfactual reasoning result; The causal chain identification results, the consistency analysis results, and the counterfactual reasoning results are integrated to obtain the target analysis results, and abnormal data is filtered out from the target processing data based on the target analysis results; Based on the preset work order generation strategy and the target analysis results, the abnormal data is processed to generate corresponding target verification work orders, and corresponding target reports are generated based on the abnormal data. The target verification work order and the target report are output and processed.

[0007] Secondly, a data processing apparatus is provided, comprising: The data acquisition module is used to collect raw data from preset business data sources; The preprocessing module is used to preprocess the raw data to obtain the corresponding target processed data; The identification module is used to perform causal chain identification on the target processed data and obtain the corresponding causal chain identification result; The analysis module is used to perform time series consistency analysis on the target processing data and obtain the corresponding consistency analysis results; The reasoning module is used to perform counterfactual reasoning on the target data to obtain the corresponding counterfactual reasoning result; The processing module is used to integrate the causal chain identification result, the consistency analysis result, and the counterfactual reasoning result to obtain the target analysis result, and to filter out abnormal data from the target processing data based on the target analysis result; The generation module is used to process the abnormal data into a corresponding target verification work order based on a preset work order generation strategy and the target analysis results, and to generate a corresponding target report based on the abnormal data. The output module is used to process the output of the target verification work order and the target report.

[0008] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described data processing method.

[0009] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described data processing method.

[0010] In the above-described data processing method, apparatus, computer equipment, and storage medium, raw data is first collected from a preset business data source; the raw data is then preprocessed to obtain corresponding target processing data; causal chain identification is performed on the target processing data to obtain corresponding causal chain identification results; time series consistency analysis is performed on the target processing data to obtain corresponding consistency analysis results; counterfactual reasoning is performed on the target processing data to obtain corresponding counterfactual reasoning results; subsequently, the causal chain identification results, consistency analysis results, and counterfactual reasoning results are integrated to obtain target analysis results, and abnormal data is filtered out from the target processing data based on the target analysis results; subsequently, based on a preset work order generation strategy and the target analysis results, work order generation processing is performed on the abnormal data to obtain corresponding target verification work orders, and corresponding target reports are generated based on the abnormal data; finally, the target verification work orders and target reports are output. Based on the above automated processing flow, this application preprocesses the raw data collected from preset business data sources to obtain target processing data. Then, by performing causal chain identification, time series consistency analysis, and counterfactual reasoning on the target processing data, it can automatically, intelligently, and accurately perform anomaly analysis and processing on the data from multiple dimensions. This effectively improves the intelligence and accuracy of data anomaly analysis. Furthermore, it automatically generates work orders for abnormal data based on the combination of work order generation strategy and target analysis results, and generates target reports based on abnormal data. This efficiently generates corresponding target verification work orders and target reports, effectively improving the generation efficiency and accuracy of verification work orders and target reports. Attached Figure Description

[0011] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is an exemplary system architecture diagram to which this application can be applied; Figure 2 This is a flowchart of an embodiment of the data processing method according to this application; Figure 3 This is a schematic diagram of the structure of an embodiment of the data processing apparatus according to this application; Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0014] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0015] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0016] like Figure 1As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0017] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0018] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptops 1011, tablets 1012, or mobile phones 1013, terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, and a desktop computer, etc.

[0019] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0020] It should be noted that the data processing method provided in the embodiments of this application is generally executed by a server / terminal device, and correspondingly, the data processing device is generally located in the server / terminal device.

[0021] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0022] Continue to refer to Figure 2 A flowchart illustrating an embodiment of the data processing method according to this application is shown. The order of steps in the flowchart can be changed, and some steps can be omitted, depending on different needs. The data processing method provided in this application embodiment can be applied to any scenario requiring data verification, and thus can be applied to products in these scenarios, such as data verification products in the financial and insurance fields. The data processing method includes the following steps: Step S201: Collect raw data from the preset business data source.

[0023] In this embodiment, the data processing method runs on an electronic device (e.g., Figure 1 The server / terminal device shown can collect raw data from a preset business data source via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future-developed wireless connection methods. The executing entity of this application is specifically a data processing system, which can be simply referred to as the system. Raw data can be collected from various data sources, including collection systems (such as business systems, external interfaces), ODS (Operational Data Store) layers, temporary tables, intermediate tables, and unstructured data (such as audit logs, business descriptions). Raw data can be uniformly accessed into the system through ETL tools, API interfaces, log collectors, etc. This application can be applied to data verification scenarios in the financial and insurance fields, where data verification is crucial, relating to multiple core aspects such as risk assessment, product pricing, customer service, and compliance supervision. For example, the aforementioned raw data may include at least: 1) Sales information of insurance products recorded from the insurance sales system, including the insurance product name, coverage amount, premium, payment method, application date, and insurance period. This data is used to verify the compliance of sales operations and ensure that the sales process complies with insurance terms and regulatory requirements. For example, it checks for cases of sales beyond the scope or misleading sales. 2) Claim application information collected from the claims system, such as claim application number, accident type, accident time, accident location, claim amount, and claim document list. It also records information at each stage of the claims processing, such as claim review time, review opinion, claim payment time, and payment amount. By verifying this data, the efficiency and fairness of the claims process can be assessed, preventing claims fraud. 3) Customer medical information collected from external medical system interfaces, such as disease diagnosis records, treatment records, and physical examination reports. This data helps to accurately assess the customer's health status and determine the underwriting conditions and premium prices of insurance products. For example, a customer with a serious illness may not be able to purchase certain health insurance products or may need to increase the deductible.

[0024] Step S202: Preprocess the raw data to obtain the corresponding target processing data.

[0025] In this embodiment, the specific implementation process of preprocessing the original data to obtain the corresponding target processed data will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0026] Step S203: Perform causal chain identification on the target processing data to obtain the corresponding causal chain identification result.

[0027] In this embodiment, the specific implementation process of performing causal chain identification on the target processed data to obtain the corresponding causal chain identification result will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0028] Step S204: Perform time series consistency analysis on the target processing data to obtain the corresponding consistency analysis results.

[0029] In this embodiment, the specific implementation process of performing time series consistency analysis on the target processing data to obtain the corresponding consistency analysis results will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0030] Step S205: Perform counterfactual reasoning on the target data to obtain the corresponding counterfactual reasoning result.

[0031] In this embodiment, the counterfactual reasoning process includes: determining the intervention field and related fields: First, select a data field to be intervened in, such as the payment amount in insurance order data. Then, determine other related fields that are associated with the intervention field, such as order status (unpaid, paid, shipped, etc.) and customer points (different points may be awarded based on the payment amount). Establishing business rules and models: Based on business knowledge and experience, establish business rules and relationship models between the intervention field and related fields. For example, when the payment amount is greater than or equal to the order amount, the order status changes to "paid"; customer points are calculated according to a certain proportion based on the payment amount, etc. These rules and models can be based on simple conditional judgments or more complex mathematical models. Simulating intervention scenarios: Assume different modifications are made to the intervention field, such as changing the payment amount of an order from 100 yuan to 150 yuan. Based on the established business rules and models, simulate the expected changes in related fields under this intervention. For example, since the payment amount is greater than the order amount, the order status changes to "paid"; at the same time, according to the points calculation rules, the customer receives a corresponding increase in points. Comparative Analysis Results: Compare the expected results after the simulated intervention with the actual situation. If the simulation results match the actual business logic, the business rules and model are reasonable; if discrepancies exist, further examination is needed to determine the accuracy of the business rules and model, or whether there are other unconsidered factors. Application of Reasoning Results: Based on the counterfactual reasoning results, provide a reference for business decisions. For example, when developing marketing strategies, simulate the impact of different discount levels (intervention payment amounts) on order status and customer points to evaluate which discount strategy can better promote order conversion and improve customer loyalty.

[0032] Counterfactual reasoning, by simulating the expected output of data fields under different interventions, helps us predict the possible outcomes of business changes and assess the impact of different decision-making options. It allows us to understand various possible scenarios before actually implementing a decision, thus enabling more scientific and rational decision-making. For example, in the insurance business, counterfactual reasoning can assess the impact of different pricing strategies on the sales volume and profits of insurance products, providing a basis for insurance product pricing; in customer relationship management, it can simulate the impact of different points reward schemes on customer behavior, optimizing customer loyalty programs.

[0033] Step S206: Integrate the causal chain identification result, the consistency analysis result, and the counterfactual reasoning result to obtain the target analysis result, and filter out abnormal data from the target processing data based on the target analysis result.

[0034] In this embodiment, the target analysis results include the generated causal chain identification results, consistency analysis results, and counterfactual reasoning results. By analyzing the causal chain identification results, when data in the target processing data is found to be inconsistent with the identified causal relationships, it can be identified as abnormal data. For example, normally an increase in order amount leads to an increase in payment amount. If the order amount of an insurance order increases but the payment amount decreases, there may be an anomaly, requiring further investigation. Furthermore, time logic conflicts are detected to identify data records in the target processing data that do not conform to time rules, and these are considered abnormal data. For example, if the payment time of an insurance order is earlier than the order generation time, this clearly violates business logic and can be marked as abnormal data, requiring further investigation. Additionally, by comparing the simulation results with the actual situation based on counterfactual reasoning, data found in the target processing data that does not conform to expectations is considered abnormal data. For example, if the actual sales volume differs significantly from the simulation results after simulating a price adjustment for an insurance product, there may be anomalies in the insurance data or factors that have not been considered, requiring further analysis.

[0035] Step S207: Based on the preset work order generation strategy and the target analysis results, perform work order generation processing on the abnormal data to obtain the corresponding target verification work order, and generate the corresponding target report based on the abnormal data.

[0036] In this embodiment, the specific implementation process of generating corresponding target verification work orders from the abnormal data based on the preset work order generation strategy and the target analysis results, and generating corresponding target reports based on the abnormal data, will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0037] Step S208: Output the target verification work order and the target report.

[0038] In this embodiment, the generated target verification work order and target report can be sent to relevant business personnel via email, information, or interface display, thereby completing the output processing of the target verification work order and target report.

[0039] This application first collects raw data from a preset business data source; preprocesses the raw data to obtain corresponding target processing data; then performs causal chain identification on the target processing data to obtain corresponding causal chain identification results; performs time series consistency analysis on the target processing data to obtain corresponding consistency analysis results; and performs counterfactual reasoning on the target processing data to obtain corresponding counterfactual reasoning results; subsequently, integrates the causal chain identification results, the consistency analysis results, and the counterfactual reasoning results to obtain target analysis results, and filters out abnormal data from the target processing data based on the target analysis results; subsequently, based on a preset work order generation strategy and the target analysis results, performs work order generation processing on the abnormal data to obtain corresponding target verification work orders, and generates corresponding target reports based on the abnormal data; finally, outputs the target verification work orders and the target reports. Based on the above automated processing flow, this application preprocesses the raw data collected from preset business data sources to obtain target processing data. Then, by performing causal chain identification, time series consistency analysis, and counterfactual reasoning on the target processing data, it can automatically, intelligently, and accurately perform anomaly analysis and processing on the data from multiple dimensions. This effectively improves the intelligence and accuracy of data anomaly analysis. Furthermore, it automatically generates work orders for abnormal data based on the combination of work order generation strategy and target analysis results, and generates target reports based on abnormal data. This efficiently generates corresponding target verification work orders and target reports, effectively improving the generation efficiency and accuracy of verification work orders and target reports.

[0040] In some alternative implementations, step S203 includes the following steps: Identify the relevant data fields corresponding to the preset target business, and obtain the specified business data corresponding to the relevant data fields.

[0041] In this embodiment, the aforementioned target business can specifically be financial insurance business. In financial insurance business, the focus is on the business processes related to insurance product purchase and customer claims. The insurance product purchase process involves the customer's journey from understanding insurance product information to final purchase; the customer claims process covers the process from reporting an insured event, submitting claim documents, to receiving compensation. The relevant data fields corresponding to the target business may include: 1) Insurance product purchase data fields. Basic customer information: age, gender, occupation, income level, health status, etc. For example, age may influence a customer's preference for different types of insurance products; young people may be more inclined to purchase accident insurance, while middle-aged and elderly people may be more concerned with health insurance and pension insurance. Insurance product information: insurance type (such as life insurance, property insurance, health insurance, etc.), insurance period, sum insured, premium, coverage, etc. Different insurance types and coverage will attract customers with different needs. Purchase behavior data: purchase channel (online platform, offline agent, bank channel, etc.), purchase time, purchase frequency, etc. Purchase channels may influence a customer's purchase decision; online platforms are generally more convenient and suitable for younger customer groups. 2) Customer claims data fields. Claim application information: Reporting time, reporting method (telephone, online reporting, etc.), accident type (illness, accident, etc.), accident time, accident location, etc. Claim document data: Submission time and completeness of claim application form, medical certificate, medical expense invoices, accident report, etc. Claim processing data: Claim review time, claim decision (payment, rejection, partial payment, etc.), compensation amount, compensation payment time, etc.

[0042] The target modeling method is determined from a variety of preset modeling methods.

[0043] In this embodiment, the above modeling methods may include: (I) Bayesian networks. Financial and insurance businesses involve a large amount of uncertain information, such as the difficulty in accurately predicting a customer's health status and the randomness of accidents. Bayesian networks, based on probability theory, can handle this uncertainty. They infer causal relationships between variables by calculating conditional probabilities, such as calculating the probability of a customer purchasing a specific insurance product and the probability of a certain accident occurring based on their age, occupation, and health status. (II) DAG (Directed Acyclic Graph) structures. DAG structures visually represent the directed relationships between variables, facilitating the understanding and analysis of complex causal relationships in financial and insurance businesses. For example, DAGs can be used to represent the causal relationships between basic customer information, insurance product information, and purchase behavior data, as well as the causal relationships between claims application information, claims data, and claims processing data. The method that best matches the current business processing needs can be selected from the above various modeling methods based on actual business requirements and used as the target modeling method.

[0044] The specified business data is processed using the target modeling method to obtain the corresponding causal model.

[0045] In this embodiment, a causal model between fields is constructed using a selected target modeling method based on statistical patterns and business knowledge in specified business data. Specifically, the process of constructing a causal model for the insurance product purchase process includes: taking customer basic information, insurance product information, and purchase behavior data as examples, analyzing a large number of customer insurance product purchase records reveals the following causal relationships: a customer's age and health status are important factors influencing their purchase of health insurance. Older customers and those with poorer health are more likely to purchase health insurance. In the causal model, directed arrows point from "age" and "health status" to "purchase of health insurance." A customer's income level and occupation affect their choice of insurance product coverage and premium. High-income customers and those in specific professions (such as corporate executives and professionals) typically prefer insurance products with higher coverage and premiums. Directed arrows point from "income level" and "occupation" to "coverage" and "premium." The purchase channel affects a customer's purchase decision and purchase time. Online platforms are generally more convenient; customers may purchase insurance products online in less time and tend to purchase standardized, simple insurance products. Directed arrows point from "purchase channel" to "purchase time" and "insurance type."

[0046] Based on the causal model, the target data is processed to identify causal chains, and the corresponding output results are obtained.

[0047] In this embodiment, the above-mentioned causal chain identification process includes: 1) Clarifying normal causal relationships in the causal model. When constructing the causal model, reasonable causal relationships and corresponding rules between various data fields have been determined. For example, in the insurance product purchase process, the model shows that older customers with poor health are more likely to purchase health insurance, and the purchase channel (online or offline) will affect the purchase time and the type of insurance chosen; in the claims process, the type of accident will affect the claims review time and the amount of compensation, and the completeness and submission time of claims materials will affect the claims decision and the time of compensation payment, etc. These normal causal relationships are the benchmark for judging whether new data is abnormal. 2) Collecting and preprocessing new insurance data. New insurance data (such as target processing data) is collected, covering customer basic information, insurance product information, purchase behavior data, claims application information, claims material data, and claims processing data, etc. The new data is preprocessed, including removing duplicate data, handling missing values, data standardization, etc., to ensure data quality and make it meet the requirements of model analysis. 3) Inputting the new data into the causal model for causal chain derivation. Input the processed new data into the established causal model, and deduce the causal chains between each field based on the causal relationships and rules defined in the model. For example, for new customers purchasing insurance products, based on their age, health status, and other basic customer information, deduce the possible insurance type, coverage amount, premium, purchase channel, and purchase time according to the model; for new claims cases, deduce the approximate range of claim review time and compensation amount based on the accident type, and deduce the expected claim decision and compensation payment time based on the completeness and submission time of claim documents. 4) Compare the deduced results with the actual data. Compare the expected causal relationships and results derived through the causal model with the causal relationships and results actually recorded in the new insurance data. If the actual data matches the deduced results, it indicates that the causal chain in the new data is normal; if there is a significant difference, it may indicate an anomaly in the new data. 5) Identify anomalies and analyze the causes. Anomaly identification: Set a certain threshold or judgment standard. When the difference between the actual data and the deduced results exceeds the threshold, it is judged as an anomaly. For example, in the claims process, if the claims review time should be within 5-10 business days based on the accident type and model deduction, but the review time for a certain case in the actual new data exceeds 15 business days without a reasonable explanation (such as special circumstances like supplementary materials), then it can be determined that there is an anomaly in the claims review time for this case. Cause analysis: Conduct in-depth analysis of the data determined to be abnormal to find the cause of the anomaly. It could be a data entry error, such as incorrect information such as the customer's age or accident type; it could also be a problem in the business process, such as delays in the claims document review stage or misleading sales in the purchase process; or it could be fraudulent behavior, such as a customer deliberately concealing their health condition when purchasing insurance or fabricating an accident to file a claim.

[0048] The output result is used as the causal chain identification result.

[0049] This application identifies relevant data fields corresponding to a preset target business and obtains specified business data corresponding to those fields. Then, it selects a target modeling method from a set of preset modeling methods. Next, it performs model building processing on the specified business data based on the target modeling method to obtain a corresponding causal model. Subsequently, it performs causal chain identification processing on the target processed data based on the causal model to obtain a corresponding output result. Finally, it uses the output result as the causal chain identification result. Based on this processing flow, this application, through the use of a causal model, can efficiently and accurately complete the causal chain identification processing of target processed data, improving the intelligence of causal chain identification and ensuring the accuracy of the generated causal chain identification result.

[0050] In some optional implementations of this embodiment, step S204 includes the following steps: Extract the time field data from the target processing data.

[0051] In this embodiment, by identifying time-related fields in the target processing data, such as order generation time, payment time, shipping time, and receipt time, these time fields form the basis for time series consistency analysis. Furthermore, relevant time field data is extracted from the target processing data.

[0052] The time field data is sorted according to time order to obtain the corresponding generated data.

[0053] In this embodiment, the extracted time field data is sorted according to time order in order to check the time logic relationship one by one and obtain the corresponding generated data.

[0054] Obtain the preset time logic rules.

[0055] In this embodiment, the rules of the aforementioned time logic rules include: setting logical relationships that should be satisfied between time fields based on business logic and common sense. For example, the order generation time must be earlier than the payment time, and the shipping time should be later than the order generation time but earlier than or equal to the receipt time.

[0056] Based on the aforementioned time logic rules, the generated data is processed to detect time logic conflicts, and corresponding detection results are obtained.

[0057] In this embodiment, the process of detecting and handling the aforementioned time logic conflict includes: traversing the sorted data, checking whether the relationship between the time fields in each record meets the requirements according to the set time logic rules, and generating corresponding detection results. For example, for each order record, checking whether its payment time is earlier than the order generation time. If a situation that does not conform to the rules is found, it is marked as a time series misalignment problem.

[0058] The detection results are used as the consistency analysis results.

[0059] In this embodiment, detected time logic conflicts can be recorded in detail, including the specific record of the conflict, the fields involved, and the type of conflict. Further analysis of the causes of the conflicts may reveal issues such as data entry errors, system time synchronization problems, or abnormal business processes.

[0060] Time series consistency analysis primarily focuses on the logical consistency between time fields in the data. In business data, the time sequence often reflects the progress of business processes and the chronological relationship of events. By detecting logical time conflicts, potential errors or anomalies in the data can be identified in a timely manner. For example, incorrect order payment time records may lead to inaccurate financial statistics, and unreasonable shipping and receiving times may affect logistics efficiency assessments. Therefore, time series consistency analysis is of great significance for ensuring data quality and normal business operations.

[0061] This application extracts time field data from the target processing data; sorts the time field data according to time order to obtain corresponding generated data; then obtains preset time logic rules; and performs time logic conflict detection processing on the generated data based on the time logic rules to obtain corresponding detection results; subsequently, the detection results are used as consistency analysis results. Based on the above processing flow, this application, by using time logic rules, performs time logic conflict detection processing on the generated data obtained after sorting the target processing data in time order, and uses the obtained detection results as the corresponding consistency analysis results. This enables efficient and accurate time series consistency analysis of the target processing data, improves the intelligence of time series consistency analysis processing, and ensures the accuracy of the generated consistency analysis results.

[0062] In some optional implementations, step S207, which involves generating work orders from the abnormal data based on a preset work order generation strategy and the target analysis results to obtain corresponding target verification work orders, includes the following steps: Target information corresponding to the abnormal data is obtained based on a preset information type.

[0063] In this embodiment, the aforementioned information types include at least abnormal fields, abnormal types, and abnormal values. Correspondingly, the target information may include abnormal fields: explicitly indicating the specific field in the data where an anomaly occurs, such as the "order amount" field in order data or the "payment status" field in payment data. Abnormal types: classifying and describing the nature of the anomaly, such as missing data, data errors (including calculation errors, entry errors, etc.), and data inconsistencies (such as discrepancies between order amount and payment amount). Abnormal values: recording the specific value or status of the abnormal data, such as the actual amount when the order amount is abnormal, or the error status indicator when the payment status is abnormal. Additionally, the target information may also include other data related to the abnormal data to ensure relevant personnel have a comprehensive understanding of the situation. For example, for an abnormal order amount, in addition to the order amount itself, related data such as the unit price, quantity, shipping cost, and discount amount may be provided to help analyze the cause of the anomaly.

[0064] Based on the target analysis results, causal analysis results corresponding to the abnormal data are generated.

[0065] In this embodiment, a preliminary judgment can be made by briefly summarizing the causes of anomalies derived through causal chain identification, time series consistency analysis, and counterfactual reasoning, so as to obtain the corresponding result analysis and provide reference direction for the processing personnel. For example, "It is preliminarily determined that the abnormal order amount is due to an error in the entry of the unit price of the goods."

[0066] Call the preset work order template.

[0067] In this embodiment, the verification work order template (i.e., the work order template) can be designed jointly by relevant business personnel and system developers. The template includes necessary information fields, such as work order number, exception field, exception type, exception value, and causal analysis results. Simultaneously, the format and layout of the work order should be considered to ensure clarity and readability. The template design tool provided by the work order management system can be used to create the work order template.

[0068] The target information and the causal analysis results are filled into the work order template to obtain the corresponding generated work order.

[0069] In this embodiment, the target information and causal analysis results can be automatically filled into the corresponding information fields in the work order template, and the generated work order after filling can be used as the final target verification work order.

[0070] The generated work order is used as the target verification work order.

[0071] In this embodiment, the generated work orders can also be reviewed to ensure their accuracy and completeness. This can be done by checking whether the anomaly description is clear and whether the data is correct. If any problems are found, they are promptly reported to the system for adjustment and correction to ensure that the generated work orders meet business requirements.

[0072] This application obtains target information corresponding to abnormal data based on preset information types; then generates causal analysis results corresponding to the abnormal data based on the target analysis results; next, it calls a preset work order template; subsequently, it fills the work order template with the target information and causal analysis results to obtain the corresponding generated work order; finally, it uses the generated work order as the target verification work order. Based on the above processing flow, this application obtains target information corresponding to abnormal data based on the use of information types, generates causal analysis results corresponding to the abnormal data based on the target analysis results, fills the obtained target information and causal analysis results into a preset work order template, and uses the resulting generated work order as the corresponding target verification work order, effectively improving the generation efficiency and intelligence of verification work orders. Furthermore, the automatic generation of verification work orders realizes an automated process from data problem discovery to allocation and processing, which is conducive to improving the timeliness and accuracy of problem handling.

[0073] In some alternative implementations, step S202 includes the following steps: The original data is processed to unify its format, resulting in the corresponding first processed data.

[0074] In this embodiment, the above-mentioned data format unification process includes unifying data from different sources and in different formats into a standard format, such as unifying date formats, currency formats, etc. For example, unifying data in the business system with date formats of "YYYY / MM / DD" and data in the external interface with date formats of "DD-MM-YYYY" into the "YYYY-MM-DD" format.

[0075] Missing values ​​are processed on the first processed data to obtain the corresponding second processed data.

[0076] In this embodiment, the missing value handling includes: checking for missing values ​​in the data, and processing the missing values ​​using appropriate methods based on business rules and data characteristics, such as filling in the mean, median, specific values, or deleting records with a large number of missing values. For example, for missing customer address information in order data, if business allows, it can be filled in as "unknown" or filled in based on address information from the customer's historical orders.

[0077] The second processed data is subjected to outlier removal processing to obtain the corresponding third processed data.

[0078] In this embodiment, the outlier removal process includes: using statistical methods or machine learning algorithms to detect outliers in the data, such as using box plots, the 3σ principle, etc. For example, in the amount data of insurance orders (such as claim orders), abnormal order amounts that are significantly higher or lower than the normal range are detected.

[0079] The third processed data is subjected to field semantic annotation to obtain the corresponding fourth processed data.

[0080] In this embodiment, the semantic annotation processing of the above-mentioned fields includes: clearly annotating the meaning of data fields to ensure that different systems and different personnel have a consistent understanding of the fields. For example, for the "order status" field, the specific meaning of its different values ​​(such as "order placed", "paid", "shipped", "completed", etc.) is clearly annotated.

[0081] The fourth processed data is timestamped to obtain the corresponding fifth processed data.

[0082] In this embodiment, the timestamp alignment process includes ensuring that timestamps from different data sources are consistent for time series analysis. For example, unifying the timestamps recorded in different systems, such as order generation time, payment time, and shipping time, to the same time zone.

[0083] The fifth processed data is used as the target processed data.

[0084] Based on the above processing flow, this application performs data format unification, missing value processing, outlier removal, field semantic annotation, and timestamp alignment on the collected raw data. This enables multi-dimensional preprocessing of the raw data, effectively ensuring the accuracy and standardization of the generated target processing data, so that the target processing data can meet the requirements of subsequent data analysis.

[0085] In some optional implementations of this embodiment, step S207, generating the corresponding target report based on the abnormal data, includes the following steps: Obtain the relevant data corresponding to the abnormal data.

[0086] In this embodiment, all data related to the aforementioned abnormal data (i.e., relevant data), including raw data and target analysis results, are collected. The data is then integrated and preprocessed to ensure its integrity and consistency, such as removing duplicate data and handling missing values.

[0087] The relevant data is analyzed and interpreted based on a pre-set large model to generate report content corresponding to the relevant data.

[0088] In this embodiment, the obtained relevant data can be input into a pre-built large model. This large model analyzes and interprets the data according to preset rules and algorithms, and automatically generates detailed report content. For example, for the problem of inconsistency between insurance order data and payment data, the large model can analyze the specific inconsistencies, calculate the difference values, and combine business knowledge to infer possible causes, such as system failures, human error, etc., while assessing the potential financial risks (such as financial losses) and the impact on customer satisfaction (such as customer complaints).

[0089] Generate repair suggestions corresponding to the abnormal data.

[0090] In this embodiment, the specific implementation process of generating repair suggestions corresponding to the abnormal data will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0091] Based on the report content and the remediation suggestions, a report generation process is performed to obtain the corresponding generated report.

[0092] In this embodiment, a structured report template and content framework can be designed according to actual business needs and report usage scenarios. For example, the report content may include an overview of the data anomaly, specific anomaly fields and their differences, possible cause analysis, impact assessment on business, and remediation suggestions. Furthermore, the generated report content and remediation suggestions can be filled in according to the predetermined parts of the report template, and the filled report can be used as the corresponding generated report.

[0093] The generated report is used as the target report.

[0094] Based on the above processing flow, this application analyzes and interprets relevant data using a large-scale model, generating reports corresponding to the data. It intelligently generates reports based on the data, and then uses these reports and remediation suggestions to generate a new report, which serves as the target report. This automatic and intelligent structured reporting format makes the report content clearer and easier to understand, allowing decision-makers to quickly grasp the severity and scope of the data issues, enabling them to make scientific and reasonable decisions, take effective measures to resolve the problems, and reduce business risks.

[0095] In some optional implementations of this embodiment, generating repair suggestions corresponding to the abnormal data includes the following steps: The abnormal data is subjected to anomaly analysis processing to obtain the corresponding anomaly analysis results.

[0096] In this embodiment, a detailed analysis of the abnormal data can determine the type of anomaly, the data fields involved, and its specific manifestations, and then integrate the results to obtain the corresponding anomaly analysis. For example, regarding the discrepancy between the insurance order amount and the payment amount, the analysis can determine whether the discrepancy is due to an error in calculating the insurance order amount, an error in recording the payment amount, or both.

[0097] Based on the anomaly analysis results, corresponding review suggestions are generated, and based on the review suggestions, review results corresponding to the anomaly data are obtained.

[0098] In this embodiment, based on the data anomalies obtained through anomaly analysis (i.e., anomaly analysis results), and combined with business rules and experience, matching initial review suggestions are automatically generated. For example, for cases where the insurance order amount is calculated incorrectly, it is suggested to review the quantity, unit price, and whether there are any promotional activities in the insurance order calculation; for cases where the payment amount is recorded incorrectly, it is suggested to review the transaction records and billing information of the payment system. Furthermore, the above-mentioned abnormal data can be reviewed according to the review suggestions to obtain corresponding review results, such as confirming that the insurance order amount is calculated incorrectly.

[0099] Based on the review results, an initial repair suggestion corresponding to the abnormal data is generated.

[0100] In this embodiment, based on the review results obtained after the review process, corresponding initial repair suggestions are further generated. If it is confirmed that the order amount calculation is incorrect, the repair suggestion may be to recalculate the order amount based on the correct product quantity, unit price, and promotional activities, and update the data; if the payment amount record is incorrect, the repair suggestion may be to verify the transaction information with the payment institution and correct the payment amount record.

[0101] The initial repair suggestions are evaluated and optimized to obtain the processed target repair suggestions.

[0102] In this embodiment, the generated initial repair suggestions are evaluated to check their feasibility and effectiveness. Based on the evaluation results, the suggestions are optimized and improved to ensure that they can effectively solve the problem while minimizing the impact on business operations.

[0103] Specifically, the process of evaluating and optimizing the above initial remediation recommendations includes: 1. Implementation evaluation. 1) Feasibility assessment. Technical feasibility: Analyze the technical solutions involved in the recommendations and check whether the existing technical architecture and system can support the implementation of the recommendations. For example, if the recommendation is to introduce a new data verification algorithm, it is necessary to evaluate whether the existing server performance, software compatibility, etc. meet the requirements. Resource feasibility: Consider the human, material, and time resources required to implement the recommendations. For example, assess whether additional technical personnel need to be recruited, new hardware equipment needs to be purchased, and whether the time required to complete the implementation of the recommendations is within the acceptable range for the business. 2) Effectiveness assessment. Simulation test: Simulate the implementation of the recommendations in the evaluation environment and observe its effect on solving data anomaly problems. For example, for the recommendation to fix the inconsistency between order data and payment data, operate according to the recommended process in the simulation environment to check whether the data can be correctly matched and whether the anomaly is resolved. Case analysis: Collect historical data or similar cases, use the recommended methods for analysis and processing, compare the data differences and business results before and after processing, and evaluate the effectiveness of the recommendations. 3) Business impact assessment. Business process impact: Analyze whether the implementation of the recommendations will interfere with the existing business processes. For example, if a recommendation is made to add an extra review step to the order processing, it's necessary to assess how much processing time this will increase and whether it will affect order delivery efficiency. Customer experience impact: Consider the impact of implementing the recommendation on customer experience. For example, if the recommendation leads to longer customer wait times, it's necessary to assess the potential impact on customer satisfaction.

[0104] 2. Optimization and Refinement. 1) Problem Recording: During the evaluation process, record in detail any problems with the recommendations, such as the specific reasons for insufficient feasibility, manifestations of low effectiveness, and aspects with significant business impact. 2) Solution Adjustment. Feasibility Optimization: Adjust the recommendations to address feasibility issues. For example, if technical implementation is difficult, consider using alternative technical solutions; if resources are insufficient, adjust the implementation plan and implement the recommendations in stages. Effectiveness Improvement: For recommendations with low effectiveness, analyze the reasons and make improvements. For example, if simulation testing shows that the recommendations cannot completely solve data anomaly issues, optimize the algorithm or add data validation rules. Business Impact Reduction: For recommendations with significant business impact, find measures to reduce the impact. For example, if adding review steps affects order delivery efficiency, consider optimizing the review process and reducing unnecessary steps. 3) Re-evaluation: Re-evaluate the optimized recommendations to ensure they meet the requirements of feasibility, effectiveness, and minimal business impact. If problems still exist, continue optimization until the recommendations reach the expected standards.

[0105] The target repair suggestion is used as the repair suggestion.

[0106] This application performs anomaly analysis on abnormal data to obtain corresponding anomaly analysis results. Then, based on the anomaly analysis results, it generates corresponding review suggestions, and based on these suggestions, it obtains review results corresponding to the abnormal data. Next, based on the review results, it generates initial repair suggestions corresponding to the abnormal data. Subsequently, it evaluates and optimizes these initial repair suggestions to obtain processed target repair suggestions. Finally, it uses the target repair suggestions as the corresponding repair suggestions. Based on this process, this application automatically analyzes the anomalies in abnormal data and automatically and accurately generates targeted repair suggestions, ensuring the accuracy of the obtained repair suggestions. This reduces the workload and error rate of manual analysis, enabling relevant personnel to quickly find the root cause of the problem and take effective remedial measures, thereby ensuring data accuracy and business continuity.

[0107] In some optional implementations, the system also features automatic rule generation. Automatic rule generation, based on historical anomaly data and causal graphs, automatically extracts validation rules, improving the efficiency and accuracy of rule formulation and making the validation rules more aligned with actual business conditions. Specifically, this involves analyzing historical anomaly data, such as cases where order and payment data are inconsistent, and generating validation rules that require the order and payment amounts to be equal based on the relationship between order and payment amounts in the causal graph. These rules are then stored in a rule base. In subsequent data processing, the system can then validate order and payment data according to these rules.

[0108] Meanwhile, the system also features dynamic weight adjustment. Dynamic weight adjustment can dynamically adjust field validation weights based on the health of the data source, improving the accuracy and reliability of data auditing. When a data source encounters a problem, reducing the validation weight of fields dependent on that data source can minimize the impact of erroneous data on the audit results. Specific implementation includes real-time monitoring of data source health, for example, by monitoring indicators such as data collection success rate and data latency. When a data source has a low collection success rate or significant data latency, its health is considered poor, and the validation weight of fields dependent on that data source is reduced. For example, if a business system experiences data collection problems, leading to data latency, the validation weight of fields related to that business system will be appropriately reduced during auditing.

[0109] Furthermore, the system also features an update function based on a closed-loop learning mechanism. This mechanism enables the system to automatically optimize the causal model and rule base based on audit results, continuously improving the system's accuracy and adaptability, and allowing it to better cope with constantly changing data conditions. Specifically, when a new abnormal data pattern is discovered, it is incorporated into the causal model's learning process. For example, if a new abnormal pattern in order data is discovered, the system analyzes the relationship between this abnormal pattern and existing causal relationships, updating the causal chains and logical relationships in the causal model. Simultaneously, the system updates the validation rules in the rule base, generating new validation rules based on the new abnormal pattern or modifying existing validation rules to improve the system's ability to identify similar abnormal data.

[0110] In some optional implementations, the system also provides user interaction and visualization features, including: 1. Visualization of Verification Results. Visualization of verification results provides users with an intuitive and easy-to-understand way to display data, helping them quickly understand data anomalies, locate the source of problems, and improve their understanding and analysis efficiency of data verification results. Specific implementations include: providing users with multiple visualization methods, such as anomaly path maps, which graphically show the propagation path of data anomalies across multiple systems. For example, in a process involving multiple business systems, arrows and nodes represent data flow and the location of anomalies, helping users intuitively see how abnormal data propagates between different systems; data consistency summaries, which display data consistency in tabular or chart form, such as the consistency ratio of each field and the distribution of abnormal fields; and risk heatmaps, which use different colors to represent the risk level of different areas or types of data. For example, red represents high-risk areas or data, yellow represents medium risk, and green represents low risk, enabling users to quickly identify high-risk data.

[0111] 2. Rule Management and Configuration. Rule management and configuration supports manual intervention and rule adjustments, allowing for flexible modification, addition, or deletion of rules based on actual business needs. This ensures the rationality and effectiveness of rules, enabling them to better adapt to business changes. Specific implementation includes providing a user interface where users can view existing validation rules and modify, add, or delete them. For example, when business rules change, users can promptly adjust the corresponding validation rules, such as changing the rule that the order amount and payment amount should be equal to a rule that considers the difference between the order amount and payment amount within a certain range as normal.

[0112] 3. Abnormal Data Review Process. The abnormal data review process provides users with an interface and workflow for manual review and correction. When system-generated repair suggestions require manual confirmation or correction, users can operate through this process to ensure data accuracy and avoid potential errors from automatic system repair. Specifically, it includes providing an interface for manual review and correction. Users can view the system-generated repair suggestions on this interface and confirm or modify them. For example, if the system suggests recalculating the order amount, users can check if the recalculated result is reasonable. If not, they can manually modify the calculation result to ensure data accuracy.

[0113] 4. System Status Monitoring. System status monitoring can monitor data source status, audit progress, and anomaly distribution in real time, enabling users to understand the system's operation, promptly identify and resolve problems, and ensure the smooth progress of data verification. Specific implementation includes: real-time display of the health status of each data source, such as whether the acquisition system is functioning normally and whether there is data delay at the ODS layer. The status of the data source can be displayed through charts or indicator lights; for example, a green indicator light indicates normal operation, and a red indicator light indicates anomalies. Simultaneously, it displays the progress of current audit tasks and the distribution of abnormal data, such as the number of completed audit tasks, the number of remaining tasks, and the distribution ratio of abnormal data across different systems and fields.

[0114] In some alternative implementations, the user information obtained is subject to user consent and complies with relevant laws and policies.

[0115] Furthermore, any software tools or components not belonging to our company that appear in the embodiments of this application are merely illustrative examples and do not represent actual use.

[0116] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0117] It should be emphasized that, in order to further ensure the privacy and security of the aforementioned target verification work orders, these target verification work orders can also be stored in a blockchain node.

[0118] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0119] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0120] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0121] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0122] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a data processing apparatus, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0123] like Figure 3As shown, the data processing device 300 described in this embodiment includes: an acquisition module 301, a preprocessing module 302, an identification module 303, an analysis module 304, an inference module 305, a processing module 306, a generation module 307, and an output module 308. Wherein: The data acquisition module 301 is used to acquire raw data from a preset business data source; Preprocessing module 302 is used to preprocess the raw data to obtain the corresponding target processing data; The identification module 303 is used to perform causal chain identification on the target processed data and obtain the corresponding causal chain identification result; Analysis module 304 is used to perform time series consistency analysis on the target processing data and obtain the corresponding consistency analysis results; The reasoning module 305 is used to perform counterfactual reasoning on the target processing data to obtain the corresponding counterfactual reasoning result; The processing module 306 is used to integrate the causal chain identification result, the consistency analysis result, and the counterfactual reasoning result to obtain the target analysis result, and to filter out abnormal data from the target processing data based on the target analysis result; The generation module 307 is used to process the abnormal data into a corresponding target verification work order based on a preset work order generation strategy and the target analysis results, and to generate a corresponding target report based on the abnormal data. The output module 308 is used to output the target verification work order and the target report.

[0124] In some optional implementations of this embodiment, the identification module 303 includes: The first processing submodule is used to determine the relevant data fields corresponding to the preset target business and to obtain the specified business data corresponding to the relevant data fields; The first determination submodule is used to determine the target modeling method from a variety of preset modeling methods; A submodule is constructed to perform model building processing on the specified business data based on the target modeling method to obtain the corresponding causal model. The identification submodule is used to perform causal chain identification processing on the target processing data based on the causal model, and obtain the corresponding output results. The second determining submodule is used to use the output result as the causal chain identification result.

[0125] In some optional implementations of this embodiment, the analysis module 304 includes: The extraction submodule is used to extract time field data from the target processing data; The sorting submodule is used to sort the time field data according to the time order to obtain the corresponding generated data; The first acquisition submodule is used to acquire preset time logic rules; The detection submodule is used to perform time logic conflict detection processing on the generated data based on the time logic rules, and obtain the corresponding detection results; The third determining submodule is used to use the detection result as the consistency analysis result.

[0126] In some optional implementations of this embodiment, the generation module 307 includes: The second acquisition submodule is used to acquire target information corresponding to the abnormal data based on a preset information type. The first generation submodule is used to generate causal analysis results corresponding to the abnormal data based on the target analysis results; Call the submodule to call the preset work order template; The fill submodule is used to fill the target information and the causal analysis results into the work order template to obtain the corresponding generated work order; The fourth determination submodule is used to use the generated work order as the target verification work order.

[0127] In some optional implementations of this embodiment, the preprocessing module 302 includes: The second processing submodule is used to perform data format unification processing on the original data to obtain the corresponding first processed data. The third processing submodule is used to process the missing values ​​of the first processed data to obtain the corresponding second processed data. The fourth processing submodule is used to perform outlier removal processing on the second processed data to obtain the corresponding third processed data; The fifth processing submodule is used to perform field semantic annotation processing on the third processed data to obtain the corresponding fourth processed data; The sixth processing submodule is used to perform timestamp alignment processing on the fourth processed data to obtain the corresponding fifth processed data; The fifth determining submodule is used to use the fifth processed data as the target processed data.

[0128] In some optional implementations of this embodiment, the generation module 307 includes: The third acquisition submodule is used to acquire relevant data corresponding to the abnormal data; The second generation submodule is used to analyze and interpret the relevant data based on a preset large model, and generate report content corresponding to the relevant data. The third generation submodule is used to generate repair suggestions corresponding to the abnormal data; The fourth generation submodule is used to perform report generation processing based on the report content and the repair suggestions to obtain the corresponding generated report; The sixth determining submodule is used to use the generated report as the target report.

[0129] In some optional implementations of this embodiment, the third generation submodule includes: An analysis unit is used to perform anomaly analysis on the abnormal data and obtain the corresponding anomaly analysis results. The first processing unit is used to generate corresponding review suggestions based on the anomaly analysis results, and to obtain review results corresponding to the anomaly data based on the review suggestions; The second processing unit is used to generate initial repair suggestions corresponding to the abnormal data based on the review results. The third processing unit is used to evaluate and optimize the initial repair suggestions to obtain the processed target repair suggestions; A determining unit is used to take the target repair suggestion as the repair suggestion.

[0130] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0131] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0132] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0133] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for data processing methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0134] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions for the data processing method.

[0135] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0136] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the data processing method described above.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0138] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A data processing method, characterized in that, Includes the following steps: Collect raw data from a pre-defined business data source; The raw data is preprocessed to obtain the corresponding target processing data; The target data is subjected to causal chain identification to obtain the corresponding causal chain identification result; Perform time series consistency analysis on the target processing data to obtain the corresponding consistency analysis results; The target data is subjected to counterfactual reasoning to obtain the corresponding counterfactual reasoning result; The causal chain identification results, the consistency analysis results, and the counterfactual reasoning results are integrated to obtain the target analysis results, and abnormal data is filtered out from the target processing data based on the target analysis results; Based on the preset work order generation strategy and the target analysis results, the abnormal data is processed to generate corresponding target verification work orders, and corresponding target reports are generated based on the abnormal data. The target verification work order and the target report are output and processed.

2. The data processing method according to claim 1, characterized in that, The step of performing causal chain identification on the target processed data to obtain the corresponding causal chain identification result specifically includes: Identify the relevant data fields corresponding to the preset target business, and obtain the specified business data corresponding to the relevant data fields; The target modeling method is determined from a variety of preset modeling methods; Based on the target modeling method, the specified business data is processed to construct a model, and a corresponding causal model is obtained. Based on the causal model, the target data is subjected to causal chain identification processing to obtain the corresponding output results; The output result is used as the causal chain identification result.

3. The data processing method according to claim 1, characterized in that, The step of performing time series consistency analysis on the target processed data to obtain the corresponding consistency analysis results specifically includes: Extract time field data from the target processing data; The time field data is sorted according to time order to obtain the corresponding generated data; Obtain the preset time logic rules; Based on the aforementioned time logic rules, the generated data is processed to detect time logic conflicts, and corresponding detection results are obtained. The detection results are used as the consistency analysis results.

4. The data processing method according to claim 1, characterized in that, The step of generating work orders from the abnormal data based on the preset work order generation strategy and the target analysis results to obtain the corresponding target verification work orders specifically includes: Obtain target information corresponding to the abnormal data based on a preset information type; Based on the target analysis results, generate causal analysis results corresponding to the abnormal data; Call the preset work order template; The target information and the causal analysis results are filled into the work order template to obtain the corresponding generated work order; The generated work order is used as the target verification work order.

5. The data processing method according to claim 1, characterized in that, The step of preprocessing the original data to obtain the corresponding target processed data specifically includes: The original data is subjected to data format unification processing to obtain the corresponding first processed data; The missing values ​​in the first processed data are processed to obtain the corresponding second processed data; The second processed data is subjected to outlier removal processing to obtain the corresponding third processed data; The third processed data is subjected to field semantic annotation to obtain the corresponding fourth processed data; The fourth processed data is timestamped to obtain the corresponding fifth processed data; The fifth processed data is used as the target processed data.

6. The data processing method according to claim 1, characterized in that, The step of generating a corresponding target report based on the abnormal data specifically includes: Obtain the relevant data corresponding to the abnormal data; The relevant data is analyzed and interpreted based on a pre-set large model to generate report content corresponding to the relevant data; Generate repair suggestions corresponding to the abnormal data; Based on the report content and the remediation suggestions, a report generation process is performed to obtain the corresponding generated report; The generated report is used as the target report.

7. The data processing method according to claim 6, characterized in that, The step of generating repair suggestions corresponding to the abnormal data specifically includes: The abnormal data is subjected to anomaly analysis processing to obtain the corresponding anomaly analysis results; Based on the anomaly analysis results, corresponding review suggestions are generated, and based on the review suggestions, review results corresponding to the anomaly data are obtained; Based on the review results, an initial repair suggestion corresponding to the abnormal data is generated; The initial repair suggestions are evaluated and optimized to obtain the processed target repair suggestions; The target repair suggestion is used as the repair suggestion.

8. A data processing apparatus, characterized in that, include: The data acquisition module is used to collect raw data from preset business data sources; The preprocessing module is used to preprocess the raw data to obtain the corresponding target processed data; The identification module is used to perform causal chain identification on the target processed data and obtain the corresponding causal chain identification result; The analysis module is used to perform time series consistency analysis on the target processing data and obtain the corresponding consistency analysis results; The reasoning module is used to perform counterfactual reasoning on the target data to obtain the corresponding counterfactual reasoning result; The processing module is used to integrate the causal chain identification result, the consistency analysis result, and the counterfactual reasoning result to obtain the target analysis result, and to filter out abnormal data from the target processing data based on the target analysis result; The generation module is used to process the abnormal data into a corresponding target verification work order based on a preset work order generation strategy and the target analysis results, and to generate a corresponding target report based on the abnormal data. The output module is used to process the output of the target verification work order and the target report.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the data processing method as described in any one of claims 1 to 7.

Images