Data verification method, device, equipment and readable storage medium

Abstract

The present application relates to the field of data verification, and discloses a data verification method, apparatus, device and readable storage medium. According to an acquired keyword set containing several target keywords and their corresponding target verification types, a verification identification list associated with the target verification type is indexed. For preset verification identifications to be verified, the data source is confirmed to be correct based on hash value comparison, and then the keyword sets are batch verified according to the verification identification list. There is no need to perform data verification of all identifications on the document, and the verification content and verification type can be specified. At the same time, the accuracy and consistency of the verification database are ensured, which solves the technical problem that existing data verification tools only support full database queries but not fixed-point queries, which may cause unnecessary data interference in the test environment and affect the test personnel's investigation of specific promissory notes or contracts.

Description

Technical Field

[0001] The present application relates to the fields of data verification and financial technology, and in particular to a data verification method, apparatus, device and readable storage medium. Background Art

[0002] In today's in-house testing environment, ensuring the accuracy of test data is crucial. This is especially true in the financial industry, where data accuracy is directly linked to risk management and business decision-making. In this context, manual verification of data for a specific IOU or contract is impractical. This is especially true when more than 300 verification items are involved. Manual verification is not only time-consuming and labor-intensive, but also prone to errors.

[0003] To address this issue, automated data verification tools can be used. These tools can handle large amounts of data verification tasks in a short period of time, significantly improving efficiency. However, the introduction of automated tools also brings new challenges. For example, while existing verification software can monitor production data in real time and verify its accuracy, it is expensive to deploy and maintain. Furthermore, current verification software only supports full database queries, not targeted queries. This can cause unnecessary data interference in testing environments, hindering testers' ability to verify specific IOUs or contracts. Summary of the Invention

[0004] The present application provides a data verification method, apparatus, device and readable storage medium, which solves the technical problem that existing data verification tools only support full-scale queries of the database but not fixed-point queries, which may cause unnecessary data interference in the test environment and affect the tester's work on checking specific promissory notes or contracts.

[0005] In view of this, the first aspect of the present application provides a data verification method, the method comprising:

[0006] Obtaining a keyword set to be verified and a target verification type, wherein the keyword set includes a plurality of target keywords;

[0007] Determine a corresponding verification identifier list according to the target verification type and a preset mapping relationship, wherein the verification identifier list includes a plurality of preset verification identifiers;

[0008] Calculate a first hash value of the preset verification identifier using a preset hash function;

[0009] Compare the first Hash value with the second Hash value pre-stored in the preset verification identifier. If the first Hash value is consistent with the second Hash value, the preset verification identifier is correct; otherwise, generate a data source error prompt;

[0010] If the preset verification identifier is correct, then the identifier of each target keyword in the keyword set is verified one by one according to the verification identifier list to determine the target keyword that hits the verification identifier list.

[0011] A second aspect of the present application provides a data verification device, the device comprising:

[0012] An acquisition unit, configured to acquire a keyword set to be verified and a target verification type, wherein the keyword set includes a plurality of target keywords;

[0013] An identifier determination unit, configured to determine a corresponding verification identifier list according to the target verification type and a preset mapping relationship, wherein the verification identifier list includes a plurality of preset verification identifiers;

[0014] A hash calculation unit, configured to calculate a first hash value of the preset verification identifier using a preset hash function;

[0015] a data source verification unit, configured to compare the first hash value with a second hash value pre-stored in the preset verification identifier; if the first hash value is consistent with the second hash value, the preset verification identifier is correct; otherwise, a data source error prompt is generated;

[0016] The keyword verification unit is used to perform identification verification on each target keyword in the keyword set according to the verification identification list if the preset verification identification is correct, and determine the target keyword that hits the verification identification list.

[0017] A third aspect of the present application provides a data verification device, the device comprising a processor and a memory:

[0018] The memory is used to store program code and transmit the program code to the processor;

[0019] The processor is used to execute the steps of the data verification method as described in the first aspect according to the instructions in the program code.

[0020] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the method described in the first aspect.

[0021] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0022] In the present application, a data verification method, apparatus, device and readable storage medium are provided. According to an acquired keyword set containing several target keywords and their corresponding target verification types, a verification identifier list associated with the target verification type is indexed. For the preset verification identifier to be verified, based on the hash value comparison, the data source is confirmed to be correct, and then the keyword set is batch verified according to the verification identifier list. There is no need to perform data verification of all identifiers on the document. The verification content and verification type can be specified, and the accuracy and consistency of the verification database can be ensured at the same time. This solves the technical problem that the existing data verification tools only support full query of the database but not fixed-point query, which may cause unnecessary data interference in the test environment and affect the tester's investigation of specific promissory notes or contracts. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of an application environment of a data verification method according to an embodiment of the present invention;

[0024] Figure 2 This is a flow chart of the data verification method in the embodiment of the present application;

[0025] Figure 3 This is a structural diagram of a data verification device in an embodiment of the present application;

[0026] Figure 4 This is a structural diagram of the data verification device in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0028] The data verification method provided by the embodiment of the present invention can be applied in Figure 1In an application environment, the client communicates with the server through a network. The server can receive a keyword set containing several target keywords and their corresponding target verification types through the client, index a verification identifier list associated with the target verification type, and for the preset verification identifier to be verified, confirm the correctness of the data source based on hash value comparison. Then, the keyword set is batch verified according to the verification identifier list to determine the target keywords that succeeded or failed the verification, and the server returns the result to the client as a prompt. The server does not need to perform data verification on all identifiers of the document, and can specify the verification content and verification type, while ensuring the accuracy and consistency of the verification database. This solves the technical problem that existing data verification tools only support full database queries but not fixed-point queries, which may cause unnecessary data interference in the test environment and affect the tester's troubleshooting work on specific promissory notes or contracts. The client can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server can be implemented using an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0029] This application designs a data verification method, device, equipment and readable storage medium. For ease of understanding, please refer to Figure 2 , Figure 2 This is a flow chart of the data verification method in the embodiment of the present application. Figure 2 As shown, specifically:

[0030] S101, obtaining a keyword set to be verified and a target verification type, wherein the keyword set includes several target keywords;

[0031] It should be noted that, upon receiving a set of keywords to be verified and a specified verification type, the system extracts these keywords and verification types and prepares to proceed to the next step.

[0032] Keywords can include but are not limited to document number, document name, user name and other different dimensions.

[0033] S102: Determine a corresponding verification identifier list according to the target verification type and the preset mapping relationship, where the verification identifier list includes several preset verification identifiers;

[0034] It should be noted that, according to the obtained verification type, a database or identification library is queried, and a verification identification list matching the verification type is obtained according to a preset mapping relationship.

[0035] S103, calculating a first Hash value of a preset verification identifier using a preset Hash function;

[0036] It should be noted that first, you should choose a suitable hash algorithm, such as SHA-256 or MD5. SHA-256 is generally recommended due to its higher security, but MD5 can still be used in some situations where security requirements are not extremely high.

[0037] Clearly identify the data for which hash values ​​need to be calculated. These identifiers can be file names, database fields, data block identifiers, etc.

[0038] Based on the determined validation flag, the corresponding data is collected. If it is a database field, a query operation may be required to obtain the data.

[0039] Hash the collected data using the selected hash algorithm. In SQL databases, you can use built-in hash functions such as MD5() or SHA2() in MySQL or HASHBYTES() in SQL Server.

[0040] S104: Compare the first Hash value with a second Hash value pre-stored in a preset verification identifier. If the first Hash value is consistent with the second Hash value, the preset verification identifier is correct; otherwise, a data source error prompt is generated.

[0041] S105: If the preset verification identifier is correct, perform identifier verification on each target keyword in the keyword set one by one according to the verification identifier list to determine the target keyword that hits the verification identifier list.

[0042] It should be noted that the keywords are matched and verified one by one through the verification identifier to identify which keywords have passed the verification identifier and which have not.

[0043] Suppose we have a data verification process for loan application documents. The keyword set may include loan application number, applicant name, application date, etc., and the target verification type is "loan review".

[0044] 1. Get the keyword set and target verification type:

[0045] Keyword set: {“loan application number 123456”, “Zhang San”, “2024-05-28”}

[0046] Target verification type: "Loan Review"

[0047] 2. Determine the calibration identification list based on the target calibration type:

[0048] The checklist of verification marks may include:

[0049] Mark 1: The loan application number must be 6 digits.

[0050] Mark 2: The applicant's name must be in Chinese characters.

[0051] Mark 3: The application date must be before the current date.

[0052] 3. Verify the identification of each target keyword in the keyword set one by one according to the verification identification list:

[0053] Loan application number 123456 meets identification 1.

[0054] Zhang San meets the identification 2.

[0055] 2024-05-28 does not match identifier 3 (assuming the current date is 2024-05-27).

[0056] 4. Determine the target keywords for the hit verification mark list:

[0057] Hit keywords: {“loan application number 123456”, “Zhang San”}

[0058] Missed keyword: "2024-05-28" (needs further processing or user input correction).

[0059] Furthermore, before determining the verification identification list according to the target verification type, the following steps are also included:

[0060] Get the target database table, target table field and target matching field value;

[0061] Get the identification filter conditions and identification verification dimensions;

[0062] Create preset verification identifiers based on identifier filtering conditions, identifier verification dimensions, target database table, target table fields, and target matching field values.

[0063] It should be noted that in the automated verification process, before determining the verification mark list, a series of preparatory work needs to be done to ensure the accuracy and applicability of the verification marks.

[0064] In order to determine the data source and specific fields to be verified, you first need to obtain the target database table, target table field, and target matching field value, and identify the table and field that need to be verified from the database.

[0065] To refine the application scope of verification flags, you need to obtain flag filtering conditions and flag verification dimensions. Specifically, you need to determine the data record scope to which the verification flag applies, such as a specific time range, user group, or business category. You also need to define verification dimensions, such as application number, loan type, and product category, to refine the application scenarios of the verification flag.

[0066] Finally, construct a specific verification identifier based on the filtering conditions and verification dimensions.

[0067] Suppose we need to verify a loan application table (loan_applications) in a loan system. The table contains the fields: application_id (application number), applicant_name (applicant name), application_date (application date), and loan_amount (loan amount).

[0068] 1. Get the target database table, target table field, and target matching field value:

[0069] Target database table: loan_applications

[0070] Target table fields: application_id, applicant_name, application_date, loan_amount

[0071] Target matching field value: Assume that we need to verify all applications with an application date in 2024.

[0072] 2. Get the identification filter conditions and identification verification dimensions:

[0073] Identify the filter condition: application_date is in 2024.

[0074] Identify verification dimensions: Verify by applicant_name (applicant name) and loan_amount (loan amount).

[0075] 3. Create a preset verification mark:

[0076] Flag 1: application_id must be non-empty and conform to a specific format (for example, 8 digits).

[0077] Flag 2: applicant_name must be non-empty and contain only letters.

[0078] Flag 3: application_date must be in 2024.

[0079] Flag 4: loan_amount must be a positive number and not exceed 1 million yuan.

[0080] The specific implementation process involves:

[0081] Database design: Create a table in the database to store validation identifiers. The fields may include rule_id (identifier ID), description (identifier description), condition (applicable condition), dimension (validation dimension), sql_statement (SQL validation statement), etc.

[0082] Front-end design: Design a user interface that allows users to select the validation type and enter relevant filter conditions and validation dimensions.

[0083] Backend logic: Implement a backend service to query and generate verification identifiers based on the information entered by the user.

[0084] Identification test: Before implementing the verification, test the generated verification identification to ensure that its logic is correct.

[0085] In this way, the flexibility and accuracy of verification marking can be ensured, while the automation level of the entire verification process can be improved.

[0086] Furthermore, after creating a preset verification identifier based on the identifier filtering condition, the identifier verification dimension, the target database table, the target table field, and the target matching field value, the following steps are also included:

[0087] Create target verification type;

[0088] Construct a preset mapping relationship between the target verification type and several preset verification identifiers.

[0089] It should be noted that in the automated verification process, after creating a preset verification identifier, it is necessary to further define the verification type and establish a mapping relationship between the verification type and the verification identifier so that the system can quickly select the appropriate verification identifier according to the verification type.

[0090] First, we need to define the types of validation operations so that we can apply different validation flags to different types of data. Specifically:

[0091] Define new verification types in the system, such as "loan application verification", "contract signing verification", etc.

[0092] Secondly, associate the verification type with a specific verification identifier to achieve fast retrieval and application. Specifically:

[0093] Create a mapping table in the system or use a configuration file to associate each calibration type with a set of preset calibration identifiers.

[0094] Assume that we already have a series of preset validation flags for the loan application form (loan_applications). Now we need to define validation types for these flags and establish mapping relationships.

[0095] 1. Create target verification type:

[0096] Verification Type: "Loan Application Completeness Verification"

[0097] 2. Build mapping relationship:

[0098] Verification Type: "Loan Application Completeness Verification"

[0099] Mapped preset validation flags:

[0100] Flag 1: application_id must be non-empty and conform to a specific format (for example, 8 digits).

[0101] Flag 2: applicant_name must be non-empty and contain only letters.

[0102] Flag 3: application_date must be in 2024.

[0103] Flag 4: loan_amount must be a positive number and not exceed 1 million yuan.

[0104] In this way, the flexibility and scalability of the verification process can be ensured, while the efficiency of the application of verification marks can be improved.

[0105] Furthermore, a verification identifier list is determined according to the target verification type. The verification identifier list includes several preset verification identifiers, specifically including:

[0106] Determine several preset verification identifiers corresponding to the target verification type based on the mapping relationship;

[0107] Collect SQL statements corresponding to several preset verification identifiers to generate a verification identifier list, and the SQL statements carry the target parameters to be replaced.

[0108] It should be noted that determining the specific checksum identifier list based on the target checksum type is a key step in the automated checksum process. This process involves retrieving the checksum identifiers based on the previously established mapping relationship and converting these identifiers into specific SQL statements to execute the checksum.

[0109] First, quickly find the applicable verification mark based on the verification type selected by the user, specifically:

[0110] Query the mapping table or configuration file to find the list of preset calibration identifiers associated with the target calibration type.

[0111] Secondly, convert the verification mark into an executable SQL query statement, specifically:

[0112] Generate a corresponding SQL statement for each verification identifier and ensure that the statement contains the target parameter to be replaced.

[0113] Assume we have a target check type "Loan Application Completeness Check" and we already have a preset check identifier associated with it.

[0114] 1. Determine the preset calibration identifier corresponding to the target calibration type:

[0115] Verification Type: "Loan Application Completeness Verification"

[0116] Corresponding logo:

[0117] Flag 1: application_id must be non-empty and conform to a specific format (for example, 8 digits).

[0118] Flag 2: applicant_name must be non-empty and contain only letters.

[0119] Flag 3: application_date must be in 2024.

[0120] Flag 4: loan_amount must be a positive number and not exceed 1 million yuan.

[0121] 2. Collect the SQL statements corresponding to the preset verification flags:

[0122] SQL statement for ID 1: SELECT * FROM loan_applications WHERE application_id IS NULL OR LEN(application_id)! = 8;

[0123] SQL statement for ID 2: SELECT * FROM loan_applications WHERE applicant_name ISNULL OR applicant_name NOT LIKE'ABCDEFGHIJKLMNOPQRSTUVWXYZ';

[0124] SQL statement for ID 3: SELECT * FROM loan_applications WHERE application_date < '2024-01-01' OR application_date > '2024-12-31';

[0125] SQL statement for ID 4: SELECT * FROM loan_applications WHERE loan_amount <= 0 OR loan_amount > 1000000;

[0126] Furthermore, the target keywords in the keyword set are checked one by one according to the check mark list, and the target keywords that hit the check mark list are specifically determined to include:

[0127] After substituting each target keyword in the keyword set into the target parameter corresponding to the SQL statement one by one, execute the SQL statement corresponding to the verification identification list;

[0128] After executing the SQL statement, when there is a hit record, the target keyword of the hit check identifier list is determined.

[0129] If there is no matching record, a successful verification prompt is generated.

[0130] It should be noted that in the automated verification process, the step of executing the SQL statement corresponding to the verification identification list is the core link, which directly determines the accuracy of the verification results.

[0131] First, replace the placeholders in the SQL statement with specific keywords to facilitate verification. Specifically:

[0132] Traverse the keyword set and substitute each keyword into the corresponding position in the SQL statement.

[0133] Secondly, the database query is used to check whether each keyword meets the verification mark, specifically:

[0134] Execute SQL statements in the database and obtain query results.

[0135] Finally, identify which keywords satisfy the validation flags, specifically:

[0136] Analyze the execution results of the SQL statement to determine which keywords triggered the verification flag.

[0137] Assume that we have the following preset validation flags and their corresponding SQL statements, and we already have a keyword set.

[0138] The SQL statement corresponding to the preset verification flag is:

[0139] Identification 1: SELECT*FROM loan_applications WHERE application_id=? ;

[0140] Identification 2: SELECT*FROM loan_applications WHERE applicant_name=? ;

[0141] Keyword set: {"application number 123456","Zhang San"}

[0142] 1. Import target parameters one by one:

[0143] For identifier 1, substitute "application number 123456" into the SQL statement to obtain: SELECT * FROM loan_applications WHERE application_id = 'application number 123456';

[0144] For identifier 2, substitute "Zhang San" into the SQL statement to obtain: SELECT * FROM loan_applications WHERE applicant_name = 'Zhang San';

[0145] 2. Execute SQL statements:

[0146] After executing the above two SQL statements, the database will return the records that meet the conditions.

[0147] 3. Determine the target keywords:

[0148] If the SQL statement with ID 1 is executed and a record is returned, it means that "application number 123456" has hit verification ID 1.

[0149] If the SQL statement with verification identifier 2 returns a record, it means that "Zhang San" hits verification identifier 2.

[0150] In this way, the automated verification process can efficiently process large amounts of data and ensure its accuracy and completeness.

[0151] Furthermore, after determining the target keyword that hits the verification identification list, the following steps are also included:

[0152] Determine the preset verification identifier corresponding to the SQL statement according to the corresponding SQL statement in the hit verification identifier list;

[0153] Generates a verification failure prompt based on the preset verification identifier and target keyword.

[0154] It should be noted that in the automated verification process, once the target keywords are determined to have hit the verification flag list, the next step is to determine which specific verification flags are triggered and generate a verification failure prompt based on these flags and the target keywords. Otherwise, the verification passes and a verification success prompt is generated.

[0155] Suppose we have the following preset verification identifiers and their corresponding SQL statements, and these SQL statements have been executed.

[0156] 1. SQL statements corresponding to the preset verification identifiers:

[0157] Identifier 1: The application number must be 8 digits. SELECT * FROM loan_applications WHERE application_id!=? AND LEN(application_id)!= 8;

[0158] Identifier 2: The applicant's name must be all letters. SELECT * FROM loan_applications WHERE applicant_name!=? AND applicant_name REGEXP '[^a-zA-Z]';

[0159] Keyword set: {"application number 123", "Zhang San 123"}

[0160] 2. Determine the preset verification identifiers corresponding to the SQL statements:

[0161] After executing the SQL statement for Identifier 1, it is found that "application number 123" hits the verification identifier.

[0162] After executing the SQL statement for Identifier 2, it is found that "Zhang San 123" hits the verification identifier.

[0163] 3. Generate a verification failure prompt:

[0164] For "application number 123", the prompt message may be: "Verification failed: The application number 'application number 123' does not meet the requirement of 8 digits."

[0165] For "Zhang San 123", the prompt message may be: "Verification failed: The applicant's name 'Zhang San 123' contains non-alphabetic characters."

[0166] In this way, the automated verification process can not only identify data problems but also provide clear feedback to users to help them quickly locate and solve problems.

[0167] Furthermore, creating preset verification identifiers based on the identifier filtering conditions, identifier verification dimensions, target database tables, target table fields, and target matching field values specifically includes:

[0168] Construct the logical structure of the preset verification identifier according to the identifier filtering conditions and the identifier verification dimension;

[0169] Fill the target database table, target table fields, and target matching field values ​​into the logical structure and generate the corresponding SQL statement for verification.

[0170] It should be noted that in the automated verification process, creating a preset verification identifier is a key step. It involves building a logical structure based on the identifier filtering conditions, identifier verification dimensions, target database table, target table fields, and target matching field values, and generating corresponding SQL statements.

[0171] First, define the basic logic and structure of the verification markup to adapt to different verification requirements. Specifically:

[0172] Design the logical expression for the identifier based on the identifier filter conditions and validation dimensions. For example, if the identifier filter condition is "application date is in 2024," the logical structure may involve comparison operations on date fields.

[0173] Secondly, integrate the specific database table and field information into the logical structure to form a complete verification mark, specifically:

[0174] Fill the target database table name, field name and expected matching value into the logical structure to generate an SQL statement that can be used for database query.

[0175] Suppose we need to validate a loan application table (loan_applications) in a loan application system. The table contains the fields: application_id (application number), applicant_name (applicant name), application_date (application date), and loan_amount (loan amount). Our goal is to create some pre-set validation flags to ensure data accuracy.

[0176] 1. Construct the logical structure of the preset verification mark:

[0177] ID 1: The application number must be 8 digits.

[0178] Mark 2: The applicant's name must be entirely in letters.

[0179] Mark 3: The application date must be in 2024.

[0180] Mark 4: The loan amount must be a positive number and not exceed RMB 1 million.

[0181] 2. Fill in the target database table, target table field, and target matching field value:

[0182] SQL statement for ID 1:

[0183] SELECT * FROM loan_applications

[0184] WHERE application_id IS NOT NULL

[0185] AND LEN(application_id)=8

[0186] AND application_id REGEXP'^[0-9]{8}$';

[0187] SQL statement for ID 2:

[0188] SELECT * FROM loan_applications

[0189] WHERE applicant_name IS NOT NULL

[0190] AND applicant_name REGEXP'^[A-Za-z]+$';

[0191] SQL statement for ID 3:

[0192] SELECT * FROM loan_applications

[0193] WHERE application_date BETWEEN'2024-01-01'AND'2024-12-31';

[0194] SQL statement for ID 4:

[0195] SELECT * FROM loan_applications

[0196] WHERE loan_amount>0

[0197] AND loan_amount<=1000000;

[0198] Database design: Ensure that the database table structure can support the execution of the above SQL statements.

[0199] Logical structure design: Design a logical structure template that can be adjusted according to different identification filtering conditions and verification dimensions.

[0200] Identification test: Before implementing verification, test the generated SQL statements to ensure that they can be executed correctly and return expected results.

[0201] This approach ensures the flexibility and accuracy of the verification mark, while increasing the automation level of the entire verification process. Furthermore, this logical structure-based approach can be easily extended to other verification marks, improving the maintainability and scalability of the system.

[0202] In another embodiment of the present invention, data preprocessing is an important step to improve the accuracy of verification before verification. The process may include the following sub-steps:

[0203] Data cleaning: Remove noise and irrelevant information from the data, such as removing spaces and special characters from string fields.

[0204] Data formatting: Convert data into a unified format for easy verification, such as unifying date fields into the ISO standard format.

[0205] Data conversion: Convert data into a type suitable for verification, such as converting a string type number into a numeric type.

[0206] In another embodiment of the present invention, in order to improve the flexibility and accuracy of verification, an advanced verification identification engine may be designed, which may include:

[0207] Tags editor: allows users to customize validation tags, including regular expressions, custom functions, etc.

[0208] Identity versioning: Manages the versions of checksum identities and allows users to roll back to older versions of identities.

[0209] Marker Optimizer: Analyzes the execution efficiency of verification marks and provides optimization suggestions.

[0210] In another embodiment of the present invention, to further improve the automation and accuracy of the verification process, a machine learning algorithm can be introduced to optimize the generation and selection of verification markers. By analyzing historical verification data, the machine learning model can learn which markers are more likely to trigger verification failures and adjust the marker priority or automatically generate new verification markers accordingly.

[0211] Machine learning optimization verification flag:

[0212] Data collection: Collect historical verification data, including verification identification, verification results, keyword features, etc.

[0213] Feature extraction: Extract features from historical data that can help predict calibration results.

[0214] Model training: Use the extracted features to train a machine learning model such as a decision tree, random forest, or neural network.

[0215] Mark optimization: Based on the model prediction results, optimize the existing verification marks or generate new verification marks.

[0216] Model evaluation: Regularly assess the accuracy and efficiency of the model and make necessary adjustments.

[0217] Suppose we have a data verification process for loan applications. We can use machine learning to optimize verification flags. First, we collect loan application verification data from the past year, including application number, applicant name, application date, loan amount, and verification results. Then, we extract the following features:

[0218] The numeric length and format of the application number.

[0219] The length and character type of the applicant's name.

[0220] Whether the application date falls on a working day.

[0221] Whether the loan amount is within the range of common loan amounts.

[0222] Using these features, we train a random forest model to predict the probability of validation failure. Once the model is trained, we use it to evaluate existing validation flags and adjust the priority of the flags based on the model's predictions. For example, if the model predicts that loan applications with weekend application dates are more likely to fail validation, we can increase the priority of the corresponding validation flags.

[0223] Additionally, we can incorporate natural language processing (NLP) technology to analyze and validate text fields, such as an applicant’s job description or loan purpose. NLP can help identify unusual words or phrases that could indicate a validation failure.

[0224] NLP technology applications:

[0225] Text preprocessing: cleaning and standardizing text data.

[0226] Feature engineering: Extracting text features such as term frequency-inverse document frequency (TF-IDF).

[0227] Sentiment analysis: Analyze the emotional tendencies in text to identify potential risks.

[0228] Entity recognition: Identify entities in text, such as names of people, places, and organizations.

[0229] Text classification: Classify text into predefined categories to assist in verification decisions.

[0230] By combining machine learning and NLP technologies, our verification system can not only more accurately predict verification results, but also better understand the context of the data, thereby providing more comprehensive verification services.

[0231] In another embodiment of the present invention, security and privacy are also crucial considerations when implementing automated verification processes. We need to ensure that all transmitted data is encrypted and that only authorized users can access sensitive information. Furthermore, we need to comply with relevant data protection regulations, such as the EU's General Data Protection Regulation (GDPR).

[0232] Safety measures:

[0233] Data encryption: Protects stored and transmitted data using strong encryption standards.

[0234] Access Control: Implement strict access control policies to ensure that only authorized users can access sensitive data.

[0235] Audit log: Records all data access and verification activities for auditing purposes when necessary.

[0236] Data desensitization: Data desensitization technology is used when processing data to protect personal privacy.

[0237] Privacy protection measures:

[0238] Minimize data collection: Collect only the data necessary to complete the verification task.

[0239] Data anonymization: Where possible, data is anonymized.

[0240] User consent: Obtain explicit consent from users before collecting and processing their data.

[0241] Data Retention Policy: Have a clear data retention policy in place and delete data when it is no longer needed.

[0242] By implementing these security and privacy measures, we can ensure the security and compliance of our automated verification process.

[0243] In another embodiment of the present invention, to improve user satisfaction and the efficiency of the verification process, we also need to design an intuitive and easy-to-use user interface. The user interface should provide clear instructions, allowing users to easily enter keyword sets and select verification types. In addition, the user interface should provide real-time feedback so that users can understand the verification progress and results.

[0244] User Interface Design:

[0245] Dashboard: Provides a dashboard that displays the verification progress, success rate, and failure reasons.

[0246] Form Input: Design a simple form that allows users to easily enter keyword sets and select validation types.

[0247] Real-time feedback: Provide real-time feedback through progress bars and notification messages.

[0248] Results presentation: Present the calibration results in an easy-to-understand format, such as a table or chart.

[0249] Error prompts: Provide clear error prompts to help users quickly locate and solve problems.

[0250] Through a well-designed user interface, we can improve the user experience and enable them to use the calibration system more efficiently.

[0251] In another embodiment of the present invention, when designing the verification system, we also considered the system's architecture and scalability. The system should be based on a modular architecture, allowing for easy addition of new features or improvement of existing ones when needed. Furthermore, the system should be able to handle a large number of concurrent requests to support large-scale verification tasks.

[0252] System architecture design:

[0253] Modular design: decompose the system into independent modules, such as input processing, identification verification, result processing, etc.

[0254] Microservice architecture: Adopt microservice architecture to improve the scalability and maintainability of the system.

[0255] Load balancing: Use load balancing technology to ensure that the system can handle a large number of concurrent requests.

[0256] Database optimization: Optimize database design to improve query efficiency and data consistency.

[0257] Cache mechanism: Introduce a cache mechanism to reduce the load on the database and improve response speed.

[0258] Through these system architecture designs, we can ensure the high performance and scalability of the verification system.

[0259] The data verification method, device, equipment, and readable storage medium provided by the present invention have the following beneficial effects:

[0260] Improve verification efficiency: By automating the verification process, the efficiency of data verification can be significantly improved. Traditional manual verification methods are time-consuming and labor-intensive, while the present invention uses automated tools to handle a large number of data verification tasks in a short period of time.

[0261] Reduced Errors: Automated verification reduces the possibility of human error. Manual verification is labor-intensive and inevitably leads to omissions. Automated verification tools can accurately execute pre-set verification marks, thus reducing the error rate.

[0262] Flexibility and Customizability: This invention allows users to specify the verification content and verification type, providing a high degree of flexibility and customizability. Users can choose different verification identifiers and dimensions as needed to meet different verification requirements.

[0263] Avoid unnecessary data interference: This invention supports fixed-point queries, avoiding the data interference that may be caused by full-data queries. When verifying a specific IOU or contract, it can avoid interference with other irrelevant data, thereby improving the efficiency of testers in troubleshooting specific issues.

[0264] Easy to maintain and expand: The automated verification tool of the present invention is based on a modular design and is easy to maintain and expand. As the business grows, new verification identifiers and types can be easily added to meet new verification needs.

[0265] Improve data accuracy: Automated verification can ensure data accuracy and completeness. This is particularly important in the financial industry, as data accuracy is directly related to risk management and business decision-making.

[0266] Reduced costs: Although it may require some initial investment to develop and deploy automated verification tools, in the long run, it can significantly reduce the cost of verification because it reduces the reliance on manual verification.

[0267] Improve user experience: The present invention provides an intuitive and easy-to-use user interface, where users can easily input keyword sets and select verification types and obtain real-time feedback, thereby improving user experience.

[0268] Security and privacy protection: The invention is designed with security and privacy protection in mind, ensuring that all transmitted data is encrypted and only authorized users can access sensitive information, in compliance with data protection regulations.

[0269] Support for large-scale processing: The system architecture design of the present invention takes into account the needs of large-scale processing, can handle a large number of concurrent requests, and support large-scale verification tasks.

[0270] Facilitate business decision-making: Accurate data verification results can support business decisions, help enterprises better understand business conditions, and formulate more effective business strategies.

[0271] Improve the reliability of the test environment: The present invention improves the reliability of the test environment by reducing data interference in the test environment, thereby ensuring the accuracy of the test results.

[0272] Facilitates regulatory compliance: Regulatory compliance is a key issue in the financial industry. The automated verification tool of the present invention can help companies more easily meet regulatory requirements by ensuring the accuracy and integrity of data.

[0273] Enhanced system robustness: Through automated verification, the system can handle various data verification tasks more robustly and maintain stable operation even when the data volume is large or complex.

[0274] Promoting technological innovation: The implementation of this invention encourages technological innovation in the field of data verification, laying the foundation for more efficient and intelligent verification tools that may appear in the future.

[0275] In summary, the present invention has significant beneficial effects in improving verification efficiency, reducing costs, enhancing data accuracy, and improving user experience, and plays an important role in promoting the development of the financial technology field.

[0276] See also Figure 3 , Figure 3 Schematic diagram of the structure of the data verification device in the embodiment of the present application. Figure 3 As shown, specifically:

[0277] An acquisition unit 201 is configured to acquire a keyword set to be verified and a target verification type, wherein the keyword set includes a plurality of target keywords;

[0278] The identifier determination unit 202 is used to determine a corresponding verification identifier list according to the target verification type and the preset mapping relationship, where the verification identifier list includes several preset verification identifiers;

[0279] A hash calculation unit 203 is configured to calculate a first hash value of the preset verification identifier using a preset hash function;

[0280] The data source verification unit 204 is configured to compare the first hash value with a second hash value pre-stored in the preset verification identifier. If the first hash value is consistent with the second hash value, the preset verification identifier is correct; otherwise, a data source error prompt is generated.

[0281] The keyword verification unit 205 is configured to perform verification on each target keyword in the keyword set according to the verification identifier list if the preset verification identifier is correct, and determine the target keyword that hits the verification identifier list.

[0282] Furthermore, it also includes an identification creation unit, which is used to:

[0283] Get the target database table, target table field and target matching field value;

[0284] Get the identification filter conditions and identification verification dimensions;

[0285] Create preset verification identifiers based on identifier filtering conditions, identifier verification dimensions, target database table, target table fields, and target matching field values.

[0286] Furthermore, it also includes an identification mapping unit, which is used to:

[0287] Create target verification type;

[0288] Construct a preset mapping relationship between the target verification type and several preset verification identifiers.

[0289] Furthermore, the identification determination unit 202 is specifically configured to:

[0290] Determine several preset verification identifiers corresponding to the target verification type based on the mapping relationship;

[0291] Collect SQL statements corresponding to several preset verification identifiers to generate a verification identifier list, and the SQL statements carry the target parameters to be replaced.

[0292] Furthermore, the verification unit 205 is specifically configured to:

[0293] After substituting each target keyword in the keyword set into the target parameter corresponding to the SQL statement one by one, execute the SQL statement corresponding to the verification identification list;

[0294] After executing the SQL statement, when there is a hit record, the target keyword of the hit check identifier list is determined.

[0295] Furthermore, it also includes a prompt unit for:

[0296] Determine the preset verification identifier corresponding to the SQL statement according to the corresponding SQL statement in the hit verification identifier list;

[0297] Generates a verification failure prompt based on the preset verification identifier and target keyword.

[0298] Furthermore, the identification creation unit is specifically configured to:

[0299] Construct the logical structure of the preset verification mark according to the mark filtering conditions and mark verification dimensions;

[0300] Fill the target database table, target table fields, and target matching field values ​​into the logical structure and generate the corresponding SQL statement for verification.

[0301] Another embodiment of the present invention provides a data verification device, such as Figure 4 As shown, the device 10 includes:

[0302] One or more processors 110 and memory 120, Figure 4 In the description, a processor 110 is used as an example. The processor 110 and the memory 120 may be connected via a bus or other means. Figure 4 The bus connection is taken as an example.

[0303] The processor 110 is used to implement various control logics of the device 10. It can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. In addition, the processor 110 can also be any traditional processor, microprocessor, or state machine. The processor 110 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP, and / or any other such configuration.

[0304] Memory 120, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions corresponding to the method for constructing a multilingual phoneme representation model in the embodiments of the present invention. Processor 110 executes the non-volatile software programs, instructions, and modules stored in memory 120 to execute various functional applications and data processing of device 10, thereby implementing the method for constructing a multilingual phoneme representation model in the aforementioned method embodiment.

[0305] The memory 120 may include a program storage area and a data storage area. The program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the device 10. Furthermore, the memory 120 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some embodiments, the memory 120 may optionally include a memory remotely located relative to the processor 110, and such remote memory may be connected to the device 10 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0306] One or more units are stored in the memory 120, and when executed by one or more processors 110, implement the following steps:

[0307] Obtain the keyword set to be verified and the target verification type, where the keyword set contains several target keywords;

[0308] Determine the corresponding verification identifier list according to the target verification type and the preset mapping relationship, and the verification identifier list includes several preset verification identifiers;

[0309] Calculate a first hash value of a preset verification identifier using a preset hash function;

[0310] Compare the first hash value with the second hash value pre-stored by the preset verification identifier. If the first hash value and the second hash value are consistent, the preset verification identifier is correct; otherwise, a data source error prompt is generated;

[0311] If the preset verification identifier is correct, then the identifier of each target keyword in the keyword set is verified one by one according to the verification identifier list to determine the target keyword that hits the verification identifier list.

[0312] An embodiment of the present invention provides a non-volatile computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the following steps are implemented:

[0313] Obtain the keyword set to be verified and the target verification type, where the keyword set contains several target keywords;

[0314] Determine the corresponding verification identifier list according to the target verification type and the preset mapping relationship, and the verification identifier list includes several preset verification identifiers;

[0315] Calculate a first hash value of a preset verification identifier using a preset hash function;

[0316] Compare the first hash value with the second hash value pre-stored by the preset verification identifier. If the first hash value and the second hash value are consistent, the preset verification identifier is correct; otherwise, a data source error prompt is generated;

[0317] If the preset verification identifier is correct, then the identifier of each target keyword in the keyword set is verified one by one according to the verification identifier list to determine the target keyword that hits the verification identifier list.

[0318] As examples, non-volatile storage media can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) as external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM, (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0319] In an embodiment of the present application, a data verification method, apparatus, device and readable storage medium are provided. According to an acquired keyword set containing several target keywords and their corresponding target verification types, a verification identifier list associated with the target verification type is indexed. For the preset verification identifier to be verified, the data source is confirmed to be correct based on the hash value comparison, and then the keyword set is batch verified according to the verification identifier list. There is no need to perform data verification of all identifiers on the document. The verification content and verification type can be specified, and the accuracy and consistency of the verification database are ensured at the same time. This solves the technical problem that the existing data verification tools only support full query of the database but not fixed-point query, which may cause unnecessary data interference in the test environment and affect the test personnel's investigation of specific promissory notes or contracts.

[0320] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0321] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0322] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0323] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0324] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0325] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0326] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), random access memory (English full name: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program code.

[0327] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0328] It should be noted that if software tools or components other than those of our company appear in the embodiments of this application, they are only used for illustration and do not represent actual use.

Claims

1. A data verification method, characterized in that: include: Obtaining a keyword set to be verified and a target verification type, wherein the keyword set includes a plurality of target keywords; Determine a corresponding verification identifier list according to the target verification type and a preset mapping relationship, wherein the verification identifier list includes a plurality of preset verification identifiers; Calculate a first hash value of the preset verification identifier using a preset hash function; Compare the first Hash value with the second Hash value pre-stored in the preset verification identifier. If the first Hash value is consistent with the second Hash value, the preset verification identifier is correct; otherwise, generate a data source error prompt; If the preset verification identifier is correct, then the identifier of each target keyword in the keyword set is verified one by one according to the verification identifier list to determine the target keyword that hits the verification identifier list.

2. The data verification method according to claim 1, wherein: Before determining the corresponding verification identifier list according to the target verification type and the preset mapping relationship, the following step is further included: Get the target database table, target table field and target matching field value; Get the identification filter conditions and identification verification dimensions; A preset verification identifier is created based on the identifier filtering condition, identifier verification dimension, target database table, target table field, and target matching field value.

3. The data verification method according to claim 2, characterized in that: After creating a preset verification identifier according to the identifier filtering condition, the identifier verification dimension, the target database table, the target table field, and the target matching field value, the following further comprises: Create target verification type; A preset mapping relationship between the target verification type and a plurality of the preset verification identifiers is established.

4. The data verification method according to claim 3, wherein: The verification identifier list is determined according to the target verification type, and the verification identifier list includes several preset verification identifiers, specifically including: Determine, based on the mapping relationship, a number of the preset verification identifiers corresponding to the target verification type; A plurality of SQL statements corresponding to the preset verification identifiers are collected to generate a verification identifier list, wherein the SQL statements carry target parameters to be replaced.

5. The data verification method according to claim 1, wherein: If the preset verification identifier is correct, then performing identifier verification on each target keyword in the keyword set one by one according to the verification identifier list, and determining the target keyword that hits the verification identifier list specifically includes: After substituting each target keyword in the keyword set into the target parameter corresponding to the SQL statement one by one, executing the SQL statement corresponding to the verification identifier list; After executing the SQL statement, when there is a hit record, the target keyword that hits the verification identifier list is determined.

6. The data verification method according to claim 5, characterized in that: After determining that the target keyword hits the verification identifier list, the following step is further included: Determine the preset verification identifier corresponding to the SQL statement according to the SQL statement corresponding to the verification identifier list; A verification failure prompt is generated based on the preset verification identifier and the target keyword.

7. The data verification method according to claim 2, characterized in that: The step of creating a preset verification identifier based on the identifier filtering condition, the identifier verification dimension, the target database table, the target table field, and the target matching field value specifically includes: Constructing a logical structure of a preset verification identifier according to the identifier filtering condition and the identifier verification dimension; The target database table, the target table fields and the target matching field values ​​are filled into the logical structure, and a corresponding SQL statement for verification is generated.

8. A data verification device, characterized in that: include: An acquisition unit, configured to acquire a keyword set to be verified and a target verification type, wherein the keyword set includes a plurality of target keywords; An identifier determination unit, configured to determine a corresponding verification identifier list according to the target verification type and a preset mapping relationship, wherein the verification identifier list includes a plurality of preset verification identifiers; A hash calculation unit, configured to calculate a first hash value of the preset verification identifier using a preset hash function; a data source verification unit, configured to compare the first hash value with a second hash value pre-stored in the preset verification identifier; if the first hash value is consistent with the second hash value, the preset verification identifier is correct; otherwise, a data source error prompt is generated; The keyword verification unit is used to perform identification verification on each target keyword in the keyword set according to the verification identification list if the preset verification identification is correct, and determine the target keyword that hits the verification identification list.

9. A data verification device, characterized in that: The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the data verification method according to any one of claims 1 to 7 according to instructions in the program code.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the data verification method according to any one of claims 1 to 7.

Images