Data verification method and device based on large model, storage medium and electronic equipment

Abstract

The invention discloses a data verification method and device based on a large model, a storage medium and electronic equipment, the method is applied to the field of artificial intelligence and the field of data verification, the method comprises the steps that text data input by a user is received, and the text data is used for indicating data content needing to be verified by the user; inputting the text data and the verification rule set into a large model, and determining a target verification rule required by data verification through the large model; sampling collection is carried out in the database every preset duration, and to-be-verified data is obtained; the target verification rule is adopted to verify the to-be-verified data, a verification result is obtained, and the verification result at least comprises abnormal data and an abnormal type. Through application of the method and the device, the problem of relatively low verification efficiency caused by relatively large difference of voice information of different fields, long time consumption in manual configuration of verification rules and high error rate due to the fact that a big data service relates to a plurality of interfaces and fields in related technologies is solved.

Description

Technical Field

[0001] This application is applied to the fields of artificial intelligence and data verification. Specifically, it relates to a data verification method, apparatus, storage medium, and electronic device based on a large model. Background Technology

[0002] In today's big data processing, ensuring data quality is paramount, directly impacting the accuracy of data analysis and the effectiveness of business decisions. However, traditional data validation methods face numerous challenges, especially when dealing with complex business logic and massive datasets. Firstly, when faced with diverse data interfaces and vast sets of fields, such as interfaces A, B, C, and D, and fields like source_code and creation_time, manually configuring validation rules is extremely labor-intensive, prone to omissions or errors, and fails to cover all possible validation scenarios.

[0003] For big data services involving related technologies, which involve multiple interfaces and fields, the large differences in voice information across different fields, the time-consuming and error-prone manual configuration of verification rules, and the resulting low verification efficiency, no effective solution has yet been proposed. Summary of the Invention

[0004] The main purpose of this application is to provide a data verification method, device, storage medium and electronic device based on a large model, in order to solve the problem that in the related technology, big data business involves multiple interfaces and fields, and due to the large difference in voice information of different fields, manual configuration of verification rules is time-consuming and has a high error rate, resulting in low verification efficiency.

[0005] To achieve the above objectives, according to one aspect of this application, a data verification method based on a large model is provided. The method includes: receiving text data input by a user, wherein the text data indicates the data content that the user needs to verify; inputting the text data and a set of verification rules into a large model, and determining the target verification rules required for data verification through the large model; sampling data from a database after a preset time interval to obtain data to be verified; and verifying the data to be verified using the target verification rules to obtain a verification result, wherein the verification result includes at least: abnormal data and anomaly type.

[0006] Further, the text data and the set of verification rules are input into a large model, and the target verification rules required for data verification are determined through the large model, including: matching the text data with the verification rules in the set of verification rules through the large model to obtain a first set of rules that match the text data; modifying the first set of rules according to the rule review opinions to obtain a second set of rules, wherein the rule review opinions are determined based on expert opinions; determining the multi-field association rules in the set of verification rules, and modifying the second set of rules according to the multi-field association rules to obtain the target verification rules.

[0007] Furthermore, the large model is used to match the text data with the verification rules in the verification rule set to obtain a first rule set that matches the text data. This includes: extracting keywords from the text data to obtain the content to be verified; performing semantic association analysis on the content to be verified and the verification rule set using the large model to output associated verification rules and the association relationship between the content to be verified and the associated verification rules; constructing a rule mapping table based on the association relationship, the content to be verified, and the associated verification rules; and determining the first rule set based on the rule mapping table.

[0008] Further, the second rule set is modified according to the multi-field association rule to obtain the target verification rule, including: determining the multiple fields involved in the multi-field association rule; determining the dependency conditions when there is an association relationship between the multiple fields; searching in the second rule set for whether there is a rule to be modified that satisfies the dependency conditions; and modifying the rule to be modified according to the multi-field association rule to obtain the target verification rule.

[0009] Furthermore, after receiving the text data input by the user, the method further includes: determining model prompt words based on the preset format of the verification rule set and the data verification result to be generated; inputting the model prompt words and the verification rule set into the large model to obtain a set of rule codes output by the model; and verifying the data to be verified using the target verification rule to obtain a verification result, including: determining the target rule code corresponding to the target verification rule in the rule code set; and verifying the data to be verified using the target rule code to obtain the verification result.

[0010] Furthermore, after verifying the data to be verified using the target verification rule and obtaining the verification result, the method further includes: extracting the abnormal features of the abnormal data from the verification result; inputting the verification result, the abnormal type, and the abnormal features into the large model to obtain the abnormal cause output by the large model; and inputting the abnormal cause and the verification result into the large model to obtain the verification rule improvement information output by the large model.

[0011] Furthermore, before inputting the text data and the set of verification rules into the large model, the method further includes: identifying multiple business scenarios involved in the financial institution, and determining a first type of verification rule based on the business operation requirements information of the multiple business scenarios; identifying multiple interfaces used by the financial institution to transmit data, collecting interface information of the multiple interfaces, and determining a second type of verification rule based on the interface information, wherein the interface information includes at least: data source information and field information; collecting historical abnormal data verified within a preset time period, and determining a third type of verification rule based on the historical abnormal data; and constructing a set of verification rules based on the first type of verification rule, the second type of verification rule, and the third type of verification rule.

[0012] To achieve the above objectives, according to another aspect of this application, a data verification device based on a large model is provided. The device includes: a receiving unit for receiving text data input by a user, wherein the text data indicates the data content that the user needs to verify; a first determining unit for inputting the text data and a set of verification rules into a large model, and determining the target verification rules required for data verification through the large model; a first acquisition unit for sampling data from a database every preset time interval to obtain data to be verified; and a verification unit for verifying the data to be verified using the target verification rules to obtain a verification result, wherein the verification result includes at least: abnormal data and abnormal type.

[0013] Further, the first determining unit includes: a matching subunit, used to match the text data with the verification rules in the verification rule set using the large model, to obtain a first rule set matching the text data; a first modification subunit, used to modify the first rule set according to the rule review opinions, to obtain a second rule set, wherein the rule review opinions are determined based on expert opinions; and a second modification subunit, used to determine the multi-field association rules in the verification rule set, and modify the second rule set according to the multi-field association rules, to obtain the target verification rule.

[0014] Further, the matching subunit includes: an extraction module, used to extract keywords from the text data to obtain the content to be verified; a processing module, used to perform semantic association analysis on the content to be verified and the set of verification rules through the large model, outputting the association verification rules and the association relationship between the content to be verified and the association verification rules; a construction module, used to construct a rule mapping table based on the association relationship, the content to be verified, and the association verification rules; and a first determination module, used to determine the first set of rules based on the rule mapping table.

[0015] Further, the second modification subunit includes: a second determining module, used to determine multiple fields involved in the multi-field association rule; a third determining module, used to determine the dependency conditions when there is an association relationship between the multiple fields; a retrieval module, used to retrieve whether there is a rule to be modified that satisfies the dependency conditions in the second rule set; and a modification module, used to modify the rule to be modified according to the multi-field association rule to obtain the target verification rule.

[0016] Further, the device includes: a first determining subunit, configured to determine model prompt words based on the preset format of the verification rule set and the data verification result to be generated after receiving text data input by the user; a processing subunit, configured to input the model prompt words and the verification rule set into the large model to obtain a set of rule codes output by the model; the verification unit 304 includes: a second determining subunit, configured to determine the target rule code corresponding to the target verification rule in the set of rule codes; and a verification subunit, configured to verify the data to be verified using the target rule code to obtain the verification result.

[0017] Further, the apparatus includes: an extraction unit, configured to extract abnormal features of the abnormal data from the verification result after verifying the data to be verified using the target verification rule and obtaining the verification result; a first processing unit, configured to input the verification result, the abnormal type, and the abnormal features into the large model to obtain the abnormal cause output by the large model; and a second processing unit, configured to input the abnormal cause and the verification result into the large model to obtain verification rule improvement information output by the large model.

[0018] Furthermore, the device further includes: a second determining unit, configured to determine multiple business scenarios involved in the financial institution before inputting the text data and the set of verification rules into the large model, and to determine a first type of verification rule based on the business operation requirements information of the multiple business scenarios; a second collecting unit, configured to determine multiple interfaces used by the financial institution to transmit data, collect the interface information of the multiple interfaces, and determine a second type of verification rule based on the interface information, wherein the interface information includes at least: data source information and field information; a third determining unit, configured to collect historical abnormal data verified within a preset time period, and to determine a third type of verification rule based on the historical abnormal data; and a constructing unit, configured to construct a set of verification rules based on the first type of verification rule, the second type of verification rule, and the third type of verification rule.

[0019] To achieve the above objectives, according to one aspect of this application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the data verification method based on a large model as described above, and the computer program, when executed by a processor, implements the steps of the data verification method based on a large model as described in various embodiments of this application.

[0020] To achieve the above objectives, according to one aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including stored computer instructions, wherein, when the computer instructions are executed by a processor, the data verification method based on any one of the above-described large models is implemented.

[0021] To achieve the above objectives, according to one aspect of this application, an electronic device is provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the large model-based data verification method described in any of the above claims.

[0022] This application employs the following steps: receiving text data input by a user, wherein the text data indicates the data content that the user needs to verify; inputting the text data and a set of verification rules into a large model, and determining the target verification rules required for data verification through the large model; sampling data from the database after a preset time interval to obtain data to be verified; verifying the data to be verified using the target verification rules to obtain a verification result, wherein the verification result includes at least: abnormal data and abnormal type. This addresses the problem in related technologies where big data services involve multiple interfaces and fields, and due to the significant differences in voice information between different fields, manual configuration of verification rules is time-consuming and has a high error rate, resulting in low verification efficiency.

[0023] By receiving text data input by users and inputting the parsed text data and a set of validation rules into a large language model, the model can accurately analyze the user's specific needs for data validation. This enables the model to determine the target validation rules based on semantic understanding, thus automating rule generation and matching. This achieves the technical effect of automatically identifying data validation content and further improves the efficiency of rule configuration and reduces human error.

[0024] In addition, after a preset time interval, samples are collected from a preset interface in the database, and the sampled data is verified using the target verification rules to obtain verification results that include abnormal data and abnormal types. This ensures the continuity and comprehensiveness of data verification, which not only improves the accuracy and efficiency of data quality detection, but also ensures the timely discovery and handling of data quality problems. This further achieves the technical effect of continuously optimizing data governance and ensuring the stability of data quality. Attached Figure Description

[0025] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0026] Figure 1 This is a flowchart of a data verification method based on a large model provided in Embodiment 1 of this application;

[0027] Figure 2 This is a flowchart illustrating the process of performing data verification and optimization operations according to the optional data quality verification system provided in Embodiment 1 of this application;

[0028] Figure 3 This is a schematic diagram of a data verification device based on a large model according to Embodiment 2 of this application;

[0029] Figure 4 This is a schematic diagram of a data verification electronic device based on a large model, according to Embodiment 5 of this application. Detailed Implementation

[0030] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, collected data, used data, generated data, processed data, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, collected information, used information, generated information, processed information, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, it needs to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information only after receiving consent from the aforementioned user or organization.

[0032] It should be noted that this application provides users with a corresponding entry point for choosing to agree to or reject the automated decision-making results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] Example 1

[0036] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of the data verification method based on a large model provided in Embodiment 1 of this application, as follows: Figure 1 As shown, the method includes the following steps:

[0037] Step S101: Receive text data input by the user, wherein the text data is used to indicate the data content that the user needs to verify.

[0038] The executing entity in this first embodiment can be a data quality verification system. This system integrates a large-scale language model to automatically generate and match data verification rules, thereby performing multi-dimensional quality verification on data originating from multiple data sources and transmitted through different interfaces. The system aims to improve the efficiency of data verification, reduce human error, ensure data quality meets business requirements, and form a closed-loop management system for data quality control through continuous optimization of verification rules.

[0039] In this first embodiment, the system receives natural language instructions or descriptions from the user terminal. These instructions or descriptions detail the specific requirements of the user (staff) regarding the data type, format, and business logic to be verified. The user inputs the characteristics of the data to be verified in text form, for example, "Mobile number verification: 11 digits, starting with 13 / 14 / 15 / 17 / 18 / 19". After receiving this text data, the system understands the verification requirements proposed by the user. By receiving and parsing the text data input by the user, the system can flexibly adapt to different verification requirements without requiring custom code development, thus improving the efficiency and accuracy of data quality verification.

[0040] Step S102: Input the text data and the set of verification rules into the large model, and determine the target verification rules required for data verification through the large model.

[0041] In this first embodiment, the data validation request provided by the user in text form, along with the existing set of validation rules, is submitted as input to a Large Language Model (LLM). The LLM analyzes the field information and business logic in the text data, as well as the pre-stored rules in the validation rule set. The data quality validation system then determines the specific rules that can meet the user's validation requirements, i.e., the aforementioned target validation rules. The LLM not only understands the user's intent but also identifies and matches the most relevant validation rules, ensuring the accuracy and effectiveness of data validation, thereby achieving effective validation of a specific dataset.

[0042] Step S103: Sample data is collected from the database after a preset time interval to obtain the data to be verified.

[0043] In this first embodiment, the data quality verification system periodically extracts a portion of data from the database as verification objects based on a configured time interval. For example, the system first defines a time period, such as once per hour or once per day, and then automatically triggers the data extraction mechanism. It accesses the corresponding data sources, such as source_01 and source_02, through preset interfaces in the database, such as interface A, data verification based on a large model, C, and D. The amount of data extracted, i.e., the sample size, is adjustable, such as being set to 10% of the total data volume. This proportion aims to balance the comprehensiveness and efficiency of the verification. The extracted sample data, i.e., the data to be verified, needs to be further verified to assess whether it conforms to established business rules and technical specifications.

[0044] Step S104: Verify the data to be verified using the target verification rules to obtain the verification result. The verification result includes at least: abnormal data and abnormal type.

[0045] In this first embodiment, target verification rules are used to verify the data to be verified. Specifically, the target verification rules determined by the large model are applied to the data sampled from the database preset interface to verify whether the data meets the specified quality standards.

[0046] The purpose of obtaining validation results is to identify data instances that do not conform to the rules, i.e., anomalous data, and the specific reasons for these non-compliance, i.e., the anomaly type. For example, if one of the target validation rules is to verify whether the "contact_info" field conforms to the mobile phone number format, the validation results will clearly indicate which data has format errors in this field, and the type of error (such as insufficient digits, incorrect starting digits, etc.). Similarly, when applying the association rule between "dept_code" and "dept_name", the validation results will identify data items where the department code and name do not match, as well as the mismatched department code value.

[0047] Therefore, the verification results not only include a list of identified abnormal data, but also detail the abnormality type of each abnormal data item, providing a data foundation for subsequent data quality improvement and verification system optimization.

[0048] In summary, the data verification method based on a large model provided in Embodiment 1 of this application receives text data input by the user, where the text data indicates the data content that the user needs to verify; inputs the text data and a set of verification rules into a large model, and determines the target verification rules required for data verification through the large model; samples are collected from the database after a preset time interval to obtain the data to be verified; and the target verification rules are used to verify the data to be verified to obtain the verification result, wherein the verification result includes at least: abnormal data and abnormal type. This method solves the problem in related technologies where big data business involves multiple interfaces and fields, and due to the large differences in voice information of different fields, manual configuration of verification rules is time-consuming and has a high error rate, resulting in low verification efficiency.

[0049] By receiving text data input by users and inputting the parsed text data and a set of validation rules into a large language model, the model can accurately analyze the user's specific needs for data validation. This enables the model to determine the target validation rules based on semantic understanding, thus automating rule generation and matching. This achieves the technical effect of automatically identifying data validation content and further improves the efficiency of rule configuration and reduces human error.

[0050] In addition, after a preset time interval, samples are collected from a preset interface in the database, and the sampled data is verified using target verification rules to obtain verification results that include abnormal data and abnormal types. This ensures the continuity and comprehensiveness of data verification, which not only improves the accuracy and efficiency of data quality detection, but also ensures the timely discovery and handling of data quality problems. This further achieves the technical effect of continuously optimizing data governance and ensuring the stability of data quality.

[0051] Optionally, in the data verification method based on a large model provided in Embodiment 1 of this application, text data and a set of verification rules are input into a large model, and the target verification rules required for data verification are determined through the large model. This includes: matching the text data with the verification rules in the set of verification rules through the large model to obtain a first set of rules that match the text data; modifying the first set of rules according to the rule review opinions to obtain a second set of rules, wherein the rule review opinions are determined based on expert opinions; determining the multi-field association rules in the set of verification rules, and modifying the second set of rules according to the multi-field association rules to obtain the target verification rules.

[0052] In this first embodiment, to construct highly accurate data validation rules that align with business needs, the user-provided text data and a pre-prepared set of validation rules are first input into a large-scale language model. Based on its powerful semantic understanding and logical inference capabilities, this model automatically analyzes the validation intent contained in the text data and the applicability of the existing rule set, thereby identifying validation rules that meet the user's needs and forming the first rule set.

[0053] Then, the first set of rules is passed to the rule review stage, where domain experts conduct a detailed review of the rules generated by the model based on their knowledge and experience. The experts assess the rationality and feasibility of the rules in light of the actual business situation and provide specific modification suggestions, i.e., rule review comments. This step ensures the consistency between the automatically generated rules and the actual business environment, reducing rule errors caused by model misunderstandings.

[0054] Secondly, based on expert review opinions, rules in the first rule set that do not conform to the business scenario are adjusted, inappropriate parts are corrected, necessary rule details are added, or redundant rules are removed, thus generating a second rule set, forming a data validation rule set that better reflects actual production conditions. For example, if the "contact_info" field only supports mobile phone numbers, then the "email verification rule" is deleted from the first rule set; if the "create_time" field needs to support the "YYYY / MM / DD" format, then rules are added to the first rule set, and corresponding rule codes are added to the rule code library.

[0055] Finally, considering the multi-field association rules frequently encountered in data quality verification, such as "the value of a certain field must match the values ​​of other fields" or "multiple fields should follow a specific logical relationship," these multi-field association rules may not be accurately identified by large models. Therefore, we can identify the multi-field association rules involved in the verification rule set and further optimize the multi-field association rules in the second rule set.

[0056] Through the above steps, the technical effect of efficiently generating and optimizing data verification rules is achieved, ensuring the accuracy and business adaptability of the verification rules, thereby effectively improving the accuracy and efficiency of data quality verification.

[0057] Optionally, in the data verification method based on a large model provided in Embodiment 1 of this application, the large model is used to match text data with verification rules in the verification rule set to obtain a first rule set that matches the text data. This includes: extracting keywords from the text data to obtain the content to be verified; performing semantic association analysis on the content to be verified and the verification rule set using the large model to output associated verification rules and the association relationship between the content to be verified and the associated verification rules; constructing a rule mapping table based on the association relationship, the content to be verified, and the associated verification rules; and determining the first rule set based on the rule mapping table.

[0058] In this first embodiment, to achieve automated and accurate data validation rule configuration, the data quality validation first performs in-depth analysis of the user-submitted text data to extract key information. This information reflects the data characteristics and requirements that the user wants to validate and is defined as the content to be validated. Keywords include, but are not limited to: the field list and business meaning of interface A, the rule name of the validation rule, and its applicable scenarios.

[0059] Subsequently, leveraging the powerful processing capabilities of a large-scale language model, a deep semantic association analysis is performed between the content to be verified and the existing verification rule base. This process aims to identify which existing verification rules in the rule base are applicable to the data verification requirements described by the user. The large model outputs a series of associated verification rules by comparing the semantic structure of the content to be verified with the rules in the rule base. For example, recognizing the business meaning of the "create_time" field (i.e., the content to be verified above) as "time," a "time verification rule" (i.e., the associated verification rule mentioned above) is obtained; recognizing the business meaning of the "contact_info" field as "contact information," a "phone number verification rule" and an "email verification rule" are matched (in this embodiment, multiple rule adaptation is supported, meaning the user-input text data matches multiple verification rules); recognizing the business meaning of the "source_code" field as "identifier code," a "non-empty verification rule" is matched.

[0060] Then, after clarifying the relationship between the associated validation rules and the content to be validated, a rule mapping table is constructed. This table records each piece of content to be validated and its corresponding associated validation rule, ensuring the accuracy and rationality of subsequent rule selection.

[0061] For example, the rule mapping table can be as shown in Table 1, which includes interface fields and matching rules that are associated with the interface fields.

[0062] Table 1

[0063]

[0064] Finally, based on the relationships recorded in the rule mapping table, the content to be verified, and the associated verification rules, the first rule set was formed. The determination of the first rule set signifies the initial completion of the transformation from user verification requirements to specific verification rules, providing a solid data foundation for the next stage of verification.

[0065] Through the above steps, the technical effects of automating the analysis of user needs and intelligently matching verification rules are achieved. This reduces the manual involvement in the traditional verification rule configuration, lowers the error rate, and speeds up the rule generation process, thereby significantly improving the efficiency and accuracy of data verification and laying a solid technical foundation for high-quality data processing.

[0066] Optionally, in the data verification method based on a large model provided in Embodiment 1 of this application, the second rule set is modified according to the multi-field association rule to obtain the target verification rule, including: determining the multiple fields involved in the multi-field association rule; determining the dependency relationship when there is an association relationship between the multiple fields; searching in the second rule set for whether there is a rule to be modified that satisfies the dependency relationship; and modifying the rule to be modified according to the multi-field association rule to obtain the target verification rule.

[0067] In this first embodiment, to accurately verify the business association between multiple fields, the data quality verification system first performs deep analysis on the user-input text data to accurately identify all fields involved in business association verification and clarify their dependencies. For example, a multi-field association rule is "If dept_code=001, then dept_name=Technology Department; if dept_code=002, then dept_name=Operations Department," where the rule involves multiple fields including dept_code and dept_name, with the dependency relationship being that the dept_name field depends on the dept_code field. This step is crucial for constructing accurate multi-field association rules that reflect business realities. The dependency relationship not only covers direct matching relationships between field values ​​but may also include calculation or inference logic based on field values, providing necessary information for accurate rule generation.

[0068] Secondly, based on the determined dependencies, the system performs a detailed search in the second rule set to look for rules that have the dependency or involve multiple fields, i.e., the aforementioned rules to be modified. This search process aims to locate existing rules that may conflict with or be incompletely covered by the second rule set. If no rule requiring modification is found, the system will automatically enter the rule verification phase.

[0069] Finally, based on the parsed multi-field association rules, the system adjusts the identified rules to be modified. This modification may involve expanding the rules to include new dependencies, or refining the rules to ensure that the validation logic fully covers all association scenarios between fields. The modified rules are integrated into the second rule set, forming the final target validation rule set, i.e., the target validation rules. This set not only includes basic single-field validation rules but also specifically includes multi-field association validation rules, enabling comprehensive and accurate data validation.

[0070] Through the above steps, the system achieves automated understanding and application of multi-field association rules, realizing the technical effect of accurately identifying and handling complex relationships between fields during data validation. This process ensures the comprehensiveness and accuracy of data validation rules, improves the efficiency of data quality assessment, and provides a powerful tool for subsequent data anomaly detection and correction, significantly enhancing the reliability and efficiency of the data processing system.

[0071] Optionally, in the data verification method based on a large model provided in Embodiment 1 of this application, before receiving the text data input by the user, the method further includes: determining model prompt words according to the preset format of the verification rule set and the data verification result to be generated; inputting the model prompt words and the verification rule set into the large model to obtain the rule code set output by the model; and verifying the data to be verified using the target verification rule to obtain the verification result, including: determining the target rule code corresponding to the target verification rule in the rule code set; and verifying the data to be verified using the target rule code to obtain the verification result.

[0072] In this first embodiment, in order to achieve automated data validation based on a large language model (LLM) and thereby improve the efficiency and accuracy of data quality validation, the preset format of the data validation result to be generated is first determined. The preset format can be pre-set or identified from historical text data input by historical users. For example, multiple historical users input "Please output whether the X field conforms to the date format YYYY-MM-DD HH:MM:SS".

[0073] Then, based on the received set of validation rules and the preset format of the validation results to be generated, the system determines the prompt words for the large model, ensuring that the large model can accurately capture the core requirements of the rules. For example, the prompt word "validate whether the date field conforms to the YYYY-MM-DD format" is transformed into "please generate a Python function that takes a string as input and outputs a boolean value to determine whether the string conforms to the YYYY-MM-DD date format." This step can generate prompt words using an LLM to process the set of validation rules and the preset format, or it can generate prompt words by simply concatenating historical text data input by users with the preset format.

[0074] Secondly, the system inputs the constructed prompt words and rule set into a large language model, triggering the large model to generate the corresponding rule code set. Based on its deep learning capabilities and semantic understanding technology, the large model transforms the rules described in natural language into specific executable code snippets, providing a foundation for automated execution of subsequent verification work. For example, the large model might output a Python function code that uses regular expressions to validate date formats.

[0075] Finally, the system uses the generated set of rule codes to validate the data collected from the database. This process consists of two steps: First, the system determines the target rule code corresponding to the target validation rule from the set of rule codes. For example, if the current target is to validate a date field, the system will search for the date format validation code from the set of rule codes. Then, the system validates the collected data by executing the target rule code to obtain the final validation result. For example, the system will iterate through all date fields to be validated, execute the date format validation function, and record all data entries and their fields that do not conform to the rules.

[0076] Through the above steps, the technical effect of automatically converting natural language-described verification rules into executable code and efficiently verifying data quality is achieved. This solution greatly reduces the workload of manually writing and maintaining verification rules. At the same time, through intelligent analysis of large models, it improves the accuracy and applicability of rule generation, ensuring the comprehensiveness and precision of data verification. It is suitable for handling large-scale data and complex and ever-changing business scenarios, providing a technical foundation for data quality verification.

[0077] Optionally, in the data verification method based on a large model provided in Embodiment 1 of this application, after verifying the data to be verified using the target verification rule and obtaining the verification result, the method further includes: extracting abnormal features of abnormal data from the verification result; inputting the verification result, abnormal type, and abnormal features into the large model to obtain the abnormal cause output by the large model; and inputting the abnormal cause and verification result into the large model to obtain the verification rule improvement information output by the large model.

[0078] In this first embodiment, in order to accurately identify and analyze the root cause of data anomalies, and then optimize the verification rules and improve the effectiveness of data quality control, the verification results are first analyzed in depth to screen and extract the unique abnormal features of the abnormal data. This feature extraction process focuses on the specific manifestation of abnormal data, including the type of abnormal fields, the pattern of abnormal values, and the pattern of abnormal distribution.

[0079] Then, the extracted anomaly features, anomalous data from the verification results, and anomaly types (i.e., the verification rule categories to which the anomalous data belongs) are integrated into anomaly analysis input and submitted to a large-scale language model for in-depth processing. Based on its powerful semantic understanding and logical reasoning capabilities, the large-scale language model can uncover potential causal relationships from complex data anomalies and analyze the specific reasons leading to the data anomalies. This causal analysis step not only reveals the surface symptoms of data verification failure.

[0080] Secondly, the causes of anomalies identified in the above analysis, along with the original validation results, are input into a large language model. The aim is to derive information for improving the validation rules through the model. This improvement information includes suggestions for adjusting existing validation rules, such as adding validation dimensions for specific fields, modifying validation thresholds, or even redesigning rule logic, with the goal of eliminating or reducing the generation of anomalous data.

[0081] Finally, based on the validation rule improvement information output by the model, the validation rule set is updated to provide a more refined rule system that better adapts to business needs, enabling subsequent data validation. This rule update process is essentially an iterative optimization of data quality control, ensuring that the validation system can continuously improve, more effectively identify and avoid data anomalies, and thus provide a dynamic, self-improving validation system.

[0082] Through the above steps, the technical effects of automated diagnosis of abnormal data causes and optimization of verification rules are achieved. This reduces subjective judgment by human intervention, enhances the intelligence and adaptability of the data verification system, and ensures the continuous improvement of data verification rules and the steady improvement of data quality. This process not only improves the efficiency of data anomaly handling but also promotes the maturity and perfection of the data quality control system through continuous analysis and rule iteration.

[0083] Optionally, in the data verification method based on a large model provided in Embodiment 1 of this application, before inputting the text data and the set of verification rules into the large model, the method further includes: determining multiple business scenarios involved in the financial institution, and determining a first type of verification rule based on the business operation requirements information of the multiple business scenarios; determining multiple interfaces used by the financial institution to transmit data, collecting interface information of the multiple interfaces, and determining a second type of verification rule based on the interface information, wherein the interface information includes at least: data source information and field information; collecting historical abnormal data verified within a preset time period, and determining a third type of verification rule based on the historical abnormal data; and constructing a set of verification rules based on the first type of verification rule, the second type of verification rule, and the third type of verification rule.

[0084] In this first embodiment, the aim is to construct a comprehensive and customized data verification rule system to meet the data quality control needs of financial institutions in different business scenarios. First, the business scope covered by the financial institution is systematically reviewed to extract the diversity and specificity of business scenarios. This process involves a detailed understanding and analysis of each business scenario to identify the unique data quality requirements in business operations. Based on a deep understanding of business needs, a first category of verification rules is established. These rules are directly related to business objectives, ensuring that data meets both business logic and technical specifications. For example, the first category of verification rules includes, but is not limited to: mobile phone number verification rules (mobile phone numbers are 11 digits, starting with 13 / 14 / 15 / 17 / 18 / 19), time verification rules (time format is YYYY-MM-DD HH:MM:SS, and the time is no later than the current time), email verification rules (email addresses include the @ symbol, and the suffix is ​​.com / .cn, etc.), and IP verification email addresses (IPv4 format is xxx.xxx.xxx.xxx, with each segment value ranging from 0 to 255).

[0085] Then, focusing on the operational characteristics of multiple interfaces used by financial institutions for data transmission, detailed interface information was collected, including data source attributes and specific field definitions. Through analysis of this interface information, the structural and format requirements of the data during transmission were determined, leading to the development of a second type of verification rules specifically targeting the accuracy and consistency of interface data. The construction of these rules, especially the precise understanding of field information, effectively prevents business risks caused by format errors or missing information during data transmission. For example, detailed descriptions of each interface (A, data verification based on a large model, C, D) were collected, including but not limited to: the data source corresponding to the interface (e.g., interface A associated with source_01 and source_03), the field list, and their business meanings (e.g., fields of interface A: source_code (data source identifier), create_time (creation time), dept_code (department code), dept_name (department name), work_content (work content), contact_info (contact information)).

[0086] Secondly, historical abnormal data is collected from the data verification records of financial institutions within a preset time period (e.g., the past week, the past three years, a specific time period, etc., which are not specifically limited in this embodiment). This step aims to learn from past data quality issues. By analyzing the characteristics and causes of historical abnormal data, a third type of verification rule is derived, which aims to specifically strengthen the verification of error-prone fields and conduct additional checks on specific data types or formats.

[0087] Finally, the three types of verification rules mentioned above are comprehensively considered and integrated to construct a set of verification rules. This set covers the specific needs of business scenarios, the operational specifications of data interfaces, and verification rules extracted from historical abnormal data. This synthesis process ensures the comprehensiveness and relevance of the rule set, enabling effective monitoring and verification of data quality from multiple dimensions.

[0088] Through the above steps, the technical effect of refined management and control of financial institutions' data quality is achieved. This not only improves the efficiency and accuracy of data verification but also enables rapid response and adaptation to changes in business scenarios. The construction of this rule set provides a solid technical foundation for financial institutions' data governance, helping to reduce business risks caused by data quality issues and ensuring smooth business operations and compliance. Furthermore, through continuous rule optimization and the application of historical data, this rule system possesses the ability to self-improve and evolve, dynamically adjusting to changes in the business environment to ensure that data verification rules are always in optimal condition, meeting the high standards of data quality required by financial institutions.

[0089] Define the validation rule requirements, generate validation code using LLM, and obtain a rule code library. The data collection interface receives user-input text data, matches validation rules to the large model, creates an initial rule set (i.e., the first rule set mentioned above), verifies the initial rules (correcting deviations), configures multi-field association rules, defines the target validation rule, samples data from the database, validates the sampled data by executing the rule code corresponding to the target validation rule, generates a data anomaly report, generates optimization suggestions for the large model, optimizes the validation rules or enterprise business processes, and continues the data validation process after optimization.

[0090] Optionally, in this first embodiment, Figure 2 This is a flowchart illustrating the data verification and optimization operations performed by the optional data quality verification system provided in Embodiment 1 of this application. Figure 2 As shown, firstly, the system determines the required data validation rules, and then uses a large language model (LLM) to transform these requirements into specific validation codes, thereby establishing a rule code library and laying the foundation for subsequent rule matching.

[0091] Then, the system collects detailed descriptions of each interface and receives the verification rule requirements input by the user in text form. It analyzes the relationship between the interface fields and the rule code library through a large language model to generate an initial rule set (i.e., the first rule set mentioned above), which contains the mapping relationship between the initially matched verification rules and the corresponding interface fields.

[0092] Secondly, the initial rule set is submitted to the manual verification stage, where business experts and technical personnel correct any possible deviations to ensure the accuracy of the rules. Based on this, complex multi-field association rules are configured to form the final target verification rule set to meet the needs of specific business scenarios.

[0093] Finally, the system samples data from the database, executes the verification process according to the rule code corresponding to the target verification rule, performs comprehensive verification on the sampled data, and outputs a data anomaly report (i.e., the verification result mentioned above). Subsequently, the report and the existing rule set are input into a large language model, which analyzes the causes of the anomalies and generates optimization suggestions. These suggestions can be used to further adjust the verification rules or optimize the enterprise's business processes to achieve better data quality control. The optimized rules and processes are then reapplied to data verification, forming a cycle of continuous improvement.

[0094] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0095] Example 2

[0096] Embodiment 2 of this application also provides a data verification device based on a large model. It should be noted that the data verification device based on a large model in Embodiment 2 of this application can be used to execute the data verification method based on a large model provided in Embodiment 1 of this application. The data verification device based on a large model provided in Embodiment 2 of this application will be described below.

[0097] Figure 3 This is a schematic diagram of a data verification device based on a large model provided in Embodiment 2 of this application. Figure 3 As shown, the device includes: a receiving unit 301, a first determining unit 302, a first acquiring unit 303, and a verification unit 304.

[0098] Specifically, the receiving unit 301 is used to receive text data input by the user, wherein the text data is used to indicate the data content that the user needs to verify.

[0099] The first determining unit 302 is used to input text data and a set of verification rules into a large model, and to determine the target verification rules required for data verification through the large model.

[0100] The first acquisition unit 303 is used to sample data from the database every preset time interval to obtain the data to be verified.

[0101] The verification unit 304 is used to verify the data to be verified using the target verification rules and obtain the verification result, wherein the verification result includes at least: abnormal data and abnormal type.

[0102] The data verification device based on a large model provided in Embodiment 2 of this application receives text data input by a user through a receiving unit 301, wherein the text data is used to indicate the data content that the user needs to verify; a first determining unit 302 inputs the text data and a set of verification rules into a large model, and determines the target verification rules required for data verification through the large model; a first acquisition unit 303 samples and acquires data to be verified from the database every preset time interval; a verification unit 304 verifies the data to be verified using the target verification rules and obtains the verification result, wherein the verification result includes at least: abnormal data and abnormal type. This solves the problem in related technologies where big data services involve multiple interfaces and fields, and due to the large differences in voice information of different fields, manual configuration of verification rules is time-consuming and has a high error rate, resulting in low verification efficiency.

[0103] By receiving text data input by users and inputting the parsed text data and a set of validation rules into a large language model, the model can accurately analyze the user's specific needs for data validation. This enables the model to determine the target validation rules based on semantic understanding, thus automating rule generation and matching. This achieves the technical effect of automatically identifying data validation content and further improves the efficiency of rule configuration and reduces human error.

[0104] In addition, after a preset time interval, samples are collected from a preset interface in the database, and the sampled data is verified using target verification rules to obtain verification results that include abnormal data and abnormal types. This ensures the continuity and comprehensiveness of data verification, which not only improves the accuracy and efficiency of data quality detection, but also ensures the timely discovery and handling of data quality problems. This further achieves the technical effect of continuously optimizing data governance and ensuring the stability of data quality.

[0105] Optionally, in the data verification device based on a large model provided in Embodiment 2 of this application, the first determining unit 302 includes: a matching subunit, used to match text data with verification rules in the verification rule set through the large model to obtain a first rule set that matches the text data; a first modification subunit, used to modify the first rule set according to the rule review opinions to obtain a second rule set, wherein the rule review opinions are determined based on expert opinions; and a second modification subunit, used to determine the multi-field association rules in the verification rule set and modify the second rule set according to the multi-field association rules to obtain the target verification rules.

[0106] Optionally, in the data verification device based on a large model provided in Embodiment 2 of this application, the matching subunit includes: an extraction module for extracting keywords from text data to obtain the content to be verified; a processing module for performing semantic association analysis on the content to be verified and the set of verification rules through the large model, outputting the associated verification rules, and the association relationship between the content to be verified and the associated verification rules; a construction module for constructing a rule mapping table based on the association relationship, the content to be verified, and the associated verification rules; and a first determination module for determining a first set of rules based on the rule mapping table.

[0107] Optionally, in the data verification device based on a large model provided in Embodiment 2 of this application, the second modification subunit includes: a second determining module, used to determine multiple fields involved in the multi-field association rule; a third determining module, used to determine the dependency conditions when there is an association relationship between multiple fields; a retrieval module, used to retrieve whether there is a rule to be modified that satisfies the dependency conditions in the second rule set; and a modification module, used to modify the rule to be modified according to the multi-field association rule to obtain the target verification rule.

[0108] Optionally, in the data verification device based on a large model provided in Embodiment 2 of this application, the device includes: a first determining subunit, used to determine model prompt words according to a set of verification rules and a preset format of the data verification result to be generated after receiving text data input by the user; a processing subunit, used to input the model prompt words and the set of verification rules into the large model to obtain a set of rule codes output by the model; and a verification unit 304 including: a second determining subunit, used to determine the target rule code corresponding to the target verification rule in the set of rule codes; and a verification subunit, used to verify the data to be verified using the target rule code to obtain a verification result.

[0109] Optionally, in the data verification device based on a large model provided in Embodiment 2 of this application, the device includes: an extraction unit, used to extract abnormal features of abnormal data from the verification results after verifying the data to be verified using the target verification rules and obtaining the verification results; a first processing unit, used to input the verification results, abnormal types, and abnormal features into the large model to obtain the abnormal causes output by the large model; and a second processing unit, used to input the abnormal causes and verification results into the large model to obtain the verification rule improvement information output by the large model.

[0110] Optionally, in the data verification device based on a large model provided in Embodiment 2 of this application, the device further includes: a second determining unit, used to determine multiple business scenarios involved in the financial institution before inputting text data and a set of verification rules into the large model, and to determine a first type of verification rule based on the business operation requirements information of the multiple business scenarios; a second collecting unit, used to determine multiple interfaces used by the financial institution to transmit data, collect interface information of the multiple interfaces, and determine a second type of verification rule based on the interface information, wherein the interface information includes at least: data source information and field information; a third determining unit, used to collect historical abnormal data verified within a preset time period, and to determine a third type of verification rule based on the historical abnormal data; and a construction unit, used to construct a set of verification rules based on the first type of verification rule, the second type of verification rule, and the third type of verification rule.

[0111] The data verification device based on the large model includes a processor and a memory. The receiving unit 301, the first determining unit 302, the first acquiring unit 303 and the verification unit 304 mentioned above are all stored in the memory as program units. The processor executes the program units stored in the memory to realize the corresponding functions.

[0112] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and data verification efficiency can be improved by adjusting kernel parameters.

[0113] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0114] Embodiment 3 of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a data verification method based on a large model.

[0115] Embodiment 4 of the present invention provides a processor for running a program, wherein the program executes a data verification method based on a large model during runtime.

[0116] Figure 4 This is a schematic diagram of a data verification electronic device based on a large model, according to Embodiment 5 of this application. Figure 4As shown, Embodiment 5 of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: receiving text data input by a user, wherein the text data is used to indicate the data content that the user needs to verify; inputting the text data and a set of verification rules into a large model, and determining the target verification rules required for data verification through the large model; sampling and collecting data from the database every preset time interval to obtain the data to be verified; and verifying the data to be verified using the target verification rules to obtain a verification result, wherein the verification result includes at least: abnormal data and abnormal type.

[0117] When the processor executes the program, it also performs the following steps: inputting text data and a set of validation rules into a large model, and determining the target validation rules required for data validation through the large model, including: matching the text data with the validation rules in the set of validation rules through the large model to obtain a first set of rules that match the text data; modifying the first set of rules based on the rule review comments to obtain a second set of rules, wherein the rule review comments are determined based on expert opinions; determining the multi-field association rules in the set of validation rules, and modifying the second set of rules based on the multi-field association rules to obtain the target validation rules.

[0118] When the processor executes the program, it also performs the following steps: matching the text data with the validation rules in the validation rule set using a large model to obtain a first rule set that matches the text data, including: extracting keywords from the text data to obtain the content to be validated; performing semantic association analysis on the content to be validated and the validation rule set using the large model to output associated validation rules and the association relationship between the content to be validated and the associated validation rules; constructing a rule mapping table based on the association relationship, the content to be validated, and the associated validation rules; and determining the first rule set based on the rule mapping table.

[0119] When the processor executes the program, it also performs the following steps: modifying the second rule set according to the multi-field association rule to obtain the target verification rule, including: determining the multiple fields involved in the multi-field association rule; determining the dependency conditions when there is an association relationship between the multiple fields; searching in the second rule set for whether there is a rule to be modified that satisfies the dependency conditions; and modifying the rule to be modified according to the multi-field association rule to obtain the target verification rule.

[0120] When the processor executes the program, it also performs the following steps: After receiving the text data input by the user, the above method further includes: determining the model prompt words based on the preset format of the verification rule set and the data verification result to be generated; inputting the model prompt words and the verification rule set into the large model to obtain the rule code set output by the model; verifying the data to be verified using the target verification rule to obtain the verification result, including: determining the target rule code corresponding to the target verification rule in the rule code set; verifying the data to be verified using the target rule code to obtain the verification result.

[0121] When the processor executes the program, it also performs the following steps: After verifying the data to be verified using the target verification rules and obtaining the verification results, the above method further includes: extracting the abnormal features of the abnormal data from the verification results; inputting the verification results, abnormal types, and abnormal features into the large model to obtain the abnormal causes output by the large model; and inputting the abnormal causes and verification results into the large model to obtain the verification rule improvement information output by the large model.

[0122] When the processor executes the program, it also performs the following steps: Before inputting the text data and the set of verification rules into the large model, the above method further includes: identifying multiple business scenarios involved in the financial institution, and determining the first type of verification rules based on the business operation requirements information of the multiple business scenarios; identifying multiple interfaces used by the financial institution to transmit data, collecting the interface information of the multiple interfaces, and determining the second type of verification rules based on the interface information, wherein the interface information includes at least: data source information and field information; collecting historical abnormal data verified within a preset time period, and determining the third type of verification rules based on the historical abnormal data; and constructing a set of verification rules based on the first type of verification rules, the second type of verification rules, and the third type of verification rules.

[0123] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0124] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: receiving text data input by a user, wherein the text data is used to indicate the data content that the user needs to verify; inputting the text data and a set of verification rules into a large model, and determining the target verification rules required for data verification through the large model; sampling data from a database every preset time interval to obtain data to be verified; and verifying the data to be verified using the target verification rules to obtain a verification result, wherein the verification result includes at least: abnormal data and abnormal type.

[0125] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: inputting text data and a set of validation rules into a large model, determining the target validation rules required for data validation through the large model, including: matching the text data with the validation rules in the set of validation rules through the large model to obtain a first set of rules that match the text data; modifying the first set of rules based on rule review opinions to obtain a second set of rules, wherein the rule review opinions are determined based on expert opinions; determining the multi-field association rules in the set of validation rules, and modifying the second set of rules based on the multi-field association rules to obtain the target validation rules.

[0126] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: matching text data with validation rules in a validation rule set using a large model to obtain a first rule set that matches the text data, including: extracting keywords from the text data to obtain the content to be validated; performing semantic association analysis on the content to be validated and the validation rule set using the large model to output associated validation rules and the association relationship between the content to be validated and the associated validation rules; constructing a rule mapping table based on the association relationship, the content to be validated, and the associated validation rules; and determining the first rule set based on the rule mapping table.

[0127] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: modifying a second rule set according to a multi-field association rule to obtain a target validation rule, including: determining the multiple fields involved in the multi-field association rule; determining the dependency conditions when there is an association relationship between the multiple fields; searching in the second rule set for a rule to be modified that satisfies the dependency conditions; and modifying the rule to be modified according to the multi-field association rule to obtain the target validation rule.

[0128] When executed on a data processing device, it is also suitable to execute an initialization program with the following method steps: after receiving text data input by the user, the above method further includes: determining model prompt words based on a set of validation rules and a preset format of the data validation result to be generated; inputting the model prompt words and the set of validation rules into a large model to obtain a set of rule codes output by the model; validating the data to be validated using the target validation rule to obtain a validation result, including: determining the target rule code corresponding to the target validation rule in the set of rule codes; validating the data to be validated using the target rule code to obtain a validation result.

[0129] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: after verifying the data to be verified using the target verification rules and obtaining the verification results, the above method further includes: extracting the abnormal features of the abnormal data from the verification results; inputting the verification results, abnormal types, and abnormal features into the large model to obtain the abnormal causes output by the large model; and inputting the abnormal causes and verification results into the large model to obtain the verification rule improvement information output by the large model.

[0130] When executed on a data processing device, it is also suitable to execute an initialization procedure with the following steps: Before inputting the text data and the set of validation rules into the large model, the above method further includes: identifying multiple business scenarios involved in the financial institution, and determining a first type of validation rule based on the business operation requirements information of the multiple business scenarios; identifying multiple interfaces used by the financial institution to transmit data, collecting interface information of the multiple interfaces, and determining a second type of validation rule based on the interface information, wherein the interface information includes at least: data source information and field information; collecting historical abnormal data verified within a preset time period, and determining a third type of validation rule based on the historical abnormal data; and constructing a set of validation rules based on the first type of validation rule, the second type of validation rule, and the third type of validation rule.

[0131] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0133] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0134] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0135] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0136] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0137] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0138] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0139] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0140] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data validation method based on a large model, characterized in that, include: Receive text data input by the user, wherein the text data is used to indicate the data content that the user needs to verify; The text data and the set of verification rules are input into a large model, and the target verification rules required for data verification are determined through the large model. After a preset time interval, data is sampled from the database to obtain data to be verified. The data to be verified is verified using the target verification rules to obtain a verification result, wherein the verification result includes at least: abnormal data and abnormal type.

2. The method according to claim 1, characterized in that, The text data and set of validation rules are input into a large model, and the target validation rules required for data validation are determined through the large model, including: The large model is used to match the text data with the verification rules in the verification rule set to obtain a first rule set that matches the text data. The first set of rules is modified based on the rule review comments to obtain the second set of rules, wherein the rule review comments are determined based on expert opinions; Determine the multi-field association rules in the set of verification rules, and modify the second set of rules according to the multi-field association rules to obtain the target verification rule.

3. The method according to claim 2, characterized in that, The large model is used to match the text data with the validation rules in the validation rule set to obtain a first rule set that matches the text data, including: Keywords are extracted from the text data to obtain the content to be verified; The large model is used to perform semantic association analysis on the content to be verified and the set of verification rules, and output the associated verification rules and the relationship between the content to be verified and the associated verification rules. A rule mapping table is constructed based on the association, the content to be verified, and the association verification rules; The first rule set is determined based on the rule mapping table.

4. The method according to claim 2, characterized in that, The second rule set is modified according to the multi-field association rule to obtain the target verification rule, including: Determine the multiple fields involved in the multi-field association rule; Determine the dependency conditions for establishing relationships between the multiple fields; Search the second rule set to see if there exists a rule to be modified that satisfies the dependency condition; The target validation rule is obtained by modifying the rule to be modified according to the multi-field association rule.

5. The method according to claim 1, characterized in that, After receiving text data input by the user, the method further includes: The model prompt words are determined based on the set of verification rules and the preset format of the data verification results to be generated; The model prompt words and the set of verification rules are input into the large model to obtain the set of rule codes output by the model; The data to be verified is verified using the target verification rules to obtain the verification result, including: Determine the target rule code corresponding to the target verification rule from the set of rule codes; The target rule code is used to verify the data to be verified, and the verification result is obtained.

6. The method according to claim 1, characterized in that, After verifying the data to be verified using the target verification rule and obtaining the verification result, the method further includes: Extract the abnormal features of the abnormal data from the verification results; The verification result, the anomaly type, and the anomaly characteristics are input into the large model to obtain the anomaly cause output by the large model; The cause of the anomaly and the verification result are input into the large model to obtain the verification rule improvement information output by the large model.

7. The method according to claim 1, characterized in that, Before inputting the text data and set of validation rules into the large model, the method further includes: Identify multiple business scenarios involved in financial institutions, and determine the first type of verification rules based on the business operation requirements information of the multiple business scenarios; The system identifies multiple interfaces used by the financial institution to transmit data, collects interface information of these multiple interfaces, and determines a second type of verification rule based on the interface information. The interface information includes at least: data source information and field information. Collect historical abnormal data verified within a preset time period, and determine the third type of verification rule based on the historical abnormal data; A set of verification rules is constructed based on the first type of verification rules, the second type of verification rules, and the third type of verification rules.

8. A data verification device based on a large model, characterized in that, include: A receiving unit is configured to receive text data input by a user, wherein the text data is used to indicate the data content that the user needs to verify; The first determining unit is used to input the text data and the set of verification rules into the large model, and determine the target verification rules required for data verification through the large model. The first data acquisition unit is used to sample data from the database every preset time interval to obtain the data to be verified. The verification unit is used to verify the data to be verified using the target verification rule and obtain a verification result, wherein the verification result includes at least: abnormal data and abnormal type.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes stored computer instructions, wherein the data verification method based on a large model as described in any one of claims 1 to 7 is implemented when the computer instructions are executed by a processor.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the data verification method based on a large model as described in any one of claims 1 to 7.

Images