File verification method, storage medium, electronic device, and program product
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INDUSTRIAL AND COMMERCIAL BANK OF CHINA
- Filing Date
- 2024-12-31
- Publication Date
- 2026-06-12
Smart Images
Figure CN119808173B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology and can also be applied to the field of financial technology. Specifically, it relates to a file verification method, storage medium, electronic device, and program product. Background Technology
[0002] Data migration is a crucial step in the process of modern enterprise IT upgrades, system integration, or migration, involving the transfer of large amounts of data from the source system to the target system. During this process, ensuring data integrity and accuracy is essential to avoid business interruptions and maintain data security and system stability.
[0003] Traditional document verification methods, such as record count comparison, sample data comparison, and data structure checks, can initially verify data consistency, but they have limitations in handling logical missing or duplicate accounting information. They often fail to effectively detect and repair accounting defects in the original documents, increasing the risks and potential problems after data migration, thus resulting in low document verification accuracy in related technologies.
[0004] There is currently no effective solution to the problem of low accuracy in document verification in related technologies. Summary of the Invention
[0005] The main objective of this application is to provide a file verification method, storage medium, electronic device, and program product to solve the problem of low accuracy in file verification in related technologies.
[0006] To achieve the above objectives, according to one aspect of this application, a file verification method is provided. The method includes: obtaining file verification rules for a file to be verified, wherein the file verification rules guide the verification process of the file to be verified; determining associated files of the file to be verified based on the file verification rules, wherein the associated files represent files that have a data interaction relationship with the file to be verified; determining association information of the file to be verified based on the association relationship between the associated files and the file to be verified, wherein the association information represents information involved in the data interaction process between the file to be verified and the associated files; and verifying the file to be verified based on the association information to obtain a verification result, wherein the verification result indicates whether the file to be verified has any anomalies.
[0007] Furthermore, the verification of the file to be verified is performed based on the association information to obtain the verification result, including: comparing the association information and the file to be verified to obtain the difference data between the association information and the file to be verified; in response to the difference data meeting the preset conditions, determining the verification result as the file to be verified having an anomaly, wherein the preset conditions are pre-set conditions used to indicate whether the file to be verified has an anomaly; in response to the difference data not meeting the preset conditions, determining the verification result as the file to be verified not having an anomaly.
[0008] Furthermore, the associated files include upstream associated files and downstream associated files. Based on the association relationship between the associated files and the file to be verified, the association information of the file to be verified is determined, including: determining the output data of the upstream associated file and the reference data of the downstream associated file based on the association relationship. The output data of the upstream associated file is applied to the file to be verified, and the output data of the file to be verified is applied to the downstream associated file. The output data represents the data to be input into the file to be verified after data processing in the upstream associated file, and the reference data represents the data to be input into the file to be verified. The output data and reference data are then input into the project processing logic corresponding to the file to be verified to obtain the association information. The project processing logic represents the operations required to implement the project corresponding to the file to be verified.
[0009] Furthermore, the method also includes: obtaining the project type corresponding to the project information contained in the file to be verified; and determining the project processing logic corresponding to the project information based on the project type.
[0010] Furthermore, the project type corresponding to the project information contained in the file to be verified is obtained, including: determining the project type based on the identification information of the project information; or, determining historical project information with a similarity greater than a preset similarity with the project information, and determining the historical project type corresponding to the historical project information as the project type.
[0011] Further, determining the associated files of the file to be verified based on the file verification rules includes: determining the project data chain corresponding to the file verification rules, wherein the project data chain is used to represent a data chain containing at least one project node, and different project nodes are used to achieve different project purposes; determining the project node position of the file to be verified according to the project data chain, wherein the project node position is used to represent the position of the project node corresponding to the file to be verified in the project data chain; and determining the associated files according to the project node position and the project data chain.
[0012] Further, determining the project data chain corresponding to the file verification rule includes: determining at least one data processing logic involved in the file verification rule; in response to the fact that the at least one data processing logic contains multiple data processing logics, concatenating the multiple data processing logics based on the logical order of the multiple data processing logics to obtain the project data chain; in response to the fact that the at least one data processing logic contains a single data processing logic, determining the single data processing logic as the project data chain.
[0013] To achieve the above objectives, according to another aspect of this application, a file verification device is provided. The device includes: a first acquisition module, configured to acquire file verification rules for a file to be verified, wherein the file verification rules guide the verification process of the file to be verified; a first determination module, configured to determine associated files of the file to be verified based on the file verification rules, wherein the associated files represent files that have a data interaction relationship with the file to be verified; a second determination module, configured to determine association information of the file to be verified based on the association relationship between the associated files and the file to be verified, wherein the association information represents information involved in the data interaction process between the file to be verified and the associated files; and a verification module, configured to verify the file to be verified based on the association information and obtain a verification result, wherein the verification result indicates whether the file to be verified is abnormal.
[0014] To achieve the above objectives, according to another aspect of this application, a computer-readable storage medium is provided, which includes a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to execute the above-described file verification method.
[0015] To achieve the above objectives, according to another aspect of this application, an electronic device is provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the above-described file verification method during runtime.
[0016] To achieve the above objectives, according to another aspect of this application, a computer program product is provided, including a computer program that implements the above-described file verification method when executed by a processor.
[0017] In this embodiment, the file verification rules for the file to be verified are first obtained; then, the associated files of the file to be verified are determined based on the file verification rules; next, the association information of the file to be verified is determined based on the association relationship between the associated files and the file to be verified; finally, the file to be verified is verified based on the association information to obtain the verification result. It is noteworthy that this application obtains the file verification rules for the file to be verified, then determines the associated files that have a data interaction relationship with the file to be verified based on the file verification rules, and then determines the association information of the file to be verified based on the data interaction relationship between the associated files and the file to be verified. The file verification process is then guided by the association information and associated files to obtain the verification result. This process, by obtaining the associated files and association information of the file to be verified to guide the verification, provides powerful guiding data information for the file verification process, while ensuring that all relevant files are included in the verification scope, thereby avoiding omissions and errors, reducing the risk of data migration, improving the integrity and accuracy of data migration, and thus solving the technical problem of low file verification accuracy in related technologies. Attached Figure Description
[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:
[0019] Figure 1 A hardware structure block diagram of a computer terminal for implementing a file verification method is shown.
[0020] Figure 2 This is a flowchart of a file verification method according to an embodiment of this application;
[0021] Figure 3 This is a schematic diagram of a document verification device according to an embodiment of this application;
[0022] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation
[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0025] Example 1
[0026] According to an embodiment of this application, a method embodiment for file verification is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0027] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing a file verification method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.
[0028] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).
[0029] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the file verification method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-mentioned file verification method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0030] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.
[0031] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).
[0032] Under the aforementioned operating environment, this application provides the following: Figure 2 The file verification method shown. Figure 2 This is a flowchart of a file verification method according to an embodiment of this application. Figure 2 As shown, the method may include the following steps:
[0033] Step S202: Obtain the file verification rules for the file to be verified, wherein the file verification rules are used to guide the verification process of the file to be verified.
[0034] The aforementioned files to be verified can refer to data files that need to be transferred from the old system to the new system. The types of files to be verified can include, but are not limited to, various key information in enterprise operations, such as transaction records, customer data, and financial data. The specific files to be verified need to be determined based on the actual situation, and are not limited here.
[0035] The aforementioned file verification rules can refer to a series of standards and processes used to guide and standardize the verification process of files to be verified before and after migration. The types of file verification rules can include, but are not limited to, data consistency verification, data integrity verification, business logic verification, data correlation verification, and abnormal data handling. The specific file verification rules need to be determined according to the type of file to be verified and the actual situation, and are not limited here. File verification rules can be used to determine the accuracy and consistency of data during the migration process.
[0036] In one optional embodiment, during the data migration process, the file verification rules for the migrated data, i.e., the files to be verified, are first obtained. These file verification rules can be generated based on the data type of the files to be verified, and are used to ensure the accuracy and consistency of the data during the migration process. This process not only effectively detects fundamental defects in the files, such as logical omissions, duplications, and data formatting, but also further analyzes the impact of these defects on business continuity and data consistency. It ensures that only files that have undergone rigorous verification and meet the standards can be migrated to the new system, thereby guaranteeing the comprehensiveness and efficiency of the verification, significantly reducing errors and anomalies during the data migration process, and thus providing a guarantee for the smooth launch and stable operation of the new system.
[0037] Step S204: Determine the associated files of the file to be verified based on the file verification rules, wherein the associated files are used to represent files that have a data interaction relationship with the file to be verified.
[0038] The aforementioned data interaction relationship refers to the interaction and information transfer between different files through data fields or data records during data processing. This data interaction relationship can be direct, such as the output of one file directly serving as the input of another; or it can be indirect, where multiple files exchange and process data through intermediate tables or logical processing links.
[0039] The types of associated files mentioned above may include, but are not limited to, upstream associated files, downstream associated files, and cross-associated files. The specific associated files need to be determined according to actual needs, and there are no restrictions here. Associated files can be used to ensure data continuity, discover potential errors in the data processing process, and perform targeted verification to avoid unnecessary full data checks, thereby improving the efficiency of data verification.
[0040] In one optional embodiment, the file verification mechanism during data migration is enhanced by determining the associated files of the file to be verified based on file verification rules. This method not only ensures the accuracy and consistency of data migration but also significantly improves the overall performance and reliability of the data processing system by identifying and correcting data interaction errors between files. Compared to traditional verification methods that only focus on the integrity of a single file, the file verification method proposed in this application can more comprehensively assess the integrity of data migration, effectively avoiding business interruptions and data security issues caused by broken data links or data inconsistencies. Furthermore, by focusing on associated files, this application can perform targeted verification, reducing unnecessary data checks, thereby greatly improving the efficiency of data verification and reducing the cost of data migration.
[0041] Step S206: Determine the association information of the file to be verified based on the association relationship between the associated file and the file to be verified. The association information is used to represent the information involved in the data interaction between the file to be verified and the associated file.
[0042] The aforementioned relationship refers to the logical connection and dependency formed between the file to be verified and the associated file during data interaction in the process of data migration and processing. The relationship is reflected in the mapping between data fields, the reference of data records, and the connection of data processing logic. It can be used to ensure the consistency, integrity, and validity of data when it is transferred between different files.
[0043] The aforementioned associated information may refer to the key data and processing details involved in the data interaction between the file to be verified and the associated files. The types of associated information may include, but are not limited to, field mapping information, data processing logic, data transformation rules, and data verification conditions. The specific associated information needs to be determined according to actual needs and is not limited here. The associated information can be used for data consistency verification, data integrity confirmation, data processing logic verification, and abnormal data location.
[0044] In one optional embodiment, by determining the association between the file to be verified and related files, and then determining the association information based on the association, a deeper level of support is provided for file verification during the data migration process. This method not only ensures the accuracy and integrity of data migration, but also provides a more comprehensive data verification perspective through detailed data interaction analysis, effectively preventing data errors and inconsistencies, and ensuring business continuity and data security. Compared with traditional verification methods that rely solely on field value matching, the technical solution of this application can gain a deeper understanding of the data transmission process, promptly identify and resolve potential data processing anomalies, and greatly improve the efficiency and quality of data verification.
[0045] Step S208: Verify the file to be verified based on the association information to obtain the verification result, wherein the verification result is used to indicate whether there is any abnormality in the file to be verified.
[0046] The aforementioned verification results can refer to the conclusions drawn after checking the file to be verified based on associated information, through verification algorithms and logical analysis. The types of verification results can include, but are not limited to, consistency verification results, integrity verification results, logical verification results, and format verification results. In addition to the above types, verification results can be quantitative, such as the number of erroneous records in the file; or qualitative, such as whether data conversion rules are correctly implemented or whether data field mapping is accurate. The specific verification results need to be determined according to the verification process, and are not limited here. Verification results can be used for data quality confirmation, problem location and correction, and to provide a basis for final decision-making.
[0047] In one optional embodiment, the verification scheme based on associated information can perform comprehensive and in-depth verification of files during the data migration process. The verification results can not only efficiently confirm the accuracy, integrity, and logic of the data, but also provide detailed verification at the data format level. Compared with traditional verification methods that rely solely on simple field comparisons, the file verification method of this application can perform more accurate anomaly detection based on richer information dimensions, such as data field mapping, data processing logic, and data reference relationships, thereby ensuring the high-quality completion of data migration. Simultaneously, this application reduces the need for manual intervention through an automated verification process, improves verification efficiency, reduces the error rate, and provides strong technical support for enterprise data migration projects.
[0048] In this embodiment, the file verification rules for the file to be verified are first obtained; then, the associated files of the file to be verified are determined based on the file verification rules; next, the association information of the file to be verified is determined based on the association relationship between the associated files and the file to be verified; finally, the file to be verified is verified based on the association information to obtain the verification result. It is noteworthy that this application obtains the file verification rules for the file to be verified, then determines the associated files that have a data interaction relationship with the file to be verified based on the file verification rules, and then determines the association information of the file to be verified based on the data interaction relationship between the associated files and the file to be verified. The file verification process is then guided by the association information and associated files to obtain the verification result. This process, by obtaining the associated files and association information of the file to be verified to guide the verification, provides powerful guiding data information for the file verification process, while ensuring that all relevant files are included in the verification scope, thereby avoiding omissions and errors, reducing the risk of data migration, improving the integrity and accuracy of data migration, and thus solving the technical problem of low file verification accuracy in related technologies.
[0049] Optionally, in the file verification method provided in this application embodiment, the verification of the file to be verified based on the association information to obtain the verification result includes: comparing the association information and the file to be verified to obtain the difference data between the association information and the file to be verified; in response to the difference data meeting a preset condition, determining that the verification result is that the file to be verified has an anomaly, wherein the preset condition is a pre-set condition used to indicate whether the file to be verified has an anomaly; in response to the difference data not meeting the preset condition, determining that the verification result is that the file to be verified does not have an anomaly.
[0050] The aforementioned discrepancy data can refer to data that is inconsistent or does not conform to preset rules, discovered by comparing the association information between the file to be verified and the associated file. The types of discrepancy data can include, but are not limited to, differences in field values, differences in data structure, differences in data format, and differences in data logic. The specific discrepancy data needs to be determined according to the actual situation and is not limited here. The discrepancy data can be used for problem identification, data quality assurance, business process verification, and decision support.
[0051] The aforementioned preset conditions can refer to a series of standards or rules set in advance when performing file verification, used to determine whether the differences between the associated information and the file to be verified constitute an anomaly. The types of preset conditions may include, but are not limited to, consistency thresholds, integrity checks, and logical rules. The specific preset conditions need to be determined according to the requirements of data quality and business continuity, and are not limited here. Preset conditions can be used to ensure the accuracy and integrity of data migration and processing.
[0052] In one optional embodiment, preset conditions are set according to data quality and business continuity requirements. Then, the associated information is compared with the file to be verified to obtain difference data. Next, it is determined whether the difference data meets the preset conditions. If it does, the verification result is determined that there is an anomaly in the file to be verified; otherwise, the verification result is determined that there is no anomaly in the file to be verified. This file verification method based on associated information, by accurately comparing difference data and combining it with preset conditions for anomaly analysis, effectively improves the quality and efficiency of file verification during data migration. It can not only quickly identify key issues in data processing but also accurately define the scope of anomalies under complex business logic, avoiding potential misjudgments or omissions in traditional verification methods.
[0053] Optionally, in the file verification method provided in this application embodiment, the associated files include upstream associated files and downstream associated files. Determining the association information of the file to be verified based on the association relationship between the associated files and the file to be verified includes: determining the output data of the upstream associated file and the reference data of the downstream associated file based on the association relationship, wherein the data output by the upstream associated file is applied to the file to be verified, and the data output by the file to be verified is applied to the downstream associated file. The output data represents the data to be input into the file to be verified after data processing in the upstream associated file, and the reference data represents the data to be input into the file to be verified. The output data and reference data are input into the project processing logic corresponding to the file to be verified to obtain the association information, wherein the project processing logic represents the operations required to implement the project corresponding to the file to be verified.
[0054] The aforementioned upstream associated files can refer to files located before the file to be verified in the data processing flow. The upstream associated files contain the original data or preliminary processed data required by the file to be verified. The upstream associated files can provide data input for the file to be verified, ensuring the continuity and integrity of the data in the processing chain.
[0055] The aforementioned upstream associated file can refer to a file located after the file to be verified in the data processing flow. The upstream associated file receives the processing result or output data of the file to be verified as the input for its subsequent processing. The downstream associated file can be used to verify the correctness of the output data of the file to be verified and ensure that the data can continue to flow according to the established business logic and data processing flow.
[0056] The output data mentioned above refers to the data that needs to be input into the file to be verified after specific data processing of the upstream related files. The output data can be used to ensure that the data input of the file to be verified meets expectations, satisfies the requirements of data processing, and maintains the continuity of the data processing chain.
[0057] The aforementioned referenced data may refer to upstream data that the file to be verified needs to reference during the processing. This data can be used to verify the data processing results of the file to be verified itself, ensuring the accuracy and consistency of the data during the processing.
[0058] The aforementioned project processing logic refers to the specific operational procedures required to implement the corresponding project of the file to be verified. The project processing logic may include, but is not limited to, data processing algorithms, data transformation rules, data verification conditions, data merging and distribution strategies, etc. The specific project processing logic needs to be determined according to actual needs, and is not limited here. The project processing logic can be used to guide and standardize the data processing process to ensure the correctness, integrity and efficiency of the data.
[0059] In one optional embodiment, the output data of the upstream associated file is first determined. This data, after processing, is received as input by the file to be verified. Next, the reference data of the downstream associated file is determined; this is the data that the file to be verified needs to provide to the downstream file after processing. By combining the output data and reference data with the project processing logic of the file to be verified, the association information of the file to be verified is obtained. This process, by analyzing the data dependencies between upstream and downstream files, can identify potential logical errors and data inconsistencies during data migration, ensuring the continuity of the data processing chain and the integrity of the data. Secondly, this method can effectively locate the source of data problems. Once data anomalies are detected, they can be quickly traced back to upstream or downstream associated files, facilitating rapid problem resolution. Thirdly, through automated analysis of output and reference data, and its integration with project processing logic, this application can significantly improve the efficiency of file verification, reduce the error rate of manual verification, and lower the cost and risk of data migration projects.
[0060] Optionally, in the file verification method provided in this application embodiment, the method further includes: obtaining the project type corresponding to the project information contained in the file to be verified; and determining the project processing logic corresponding to the project information based on the project type.
[0061] The project information mentioned above can refer to various details and attributes related to the data migration project. Project information may include, but is not limited to, the source of the data, the target of the data, the structure of the data, the processing requirements of the data, the time range of the data, and the business background of the data. The specific project information needs to be determined based on the file to be verified, and is not limited here. Project information can be used to help understand the background, purpose and expected effect of data migration, determine the project type, and provide a basis for formulating effective data migration strategies and project processing logic.
[0062] The project types mentioned above refer to the categories to which data migration projects belong after classification. Project types can be divided into multiple categories based on different dimensions such as the purpose of data migration, the nature of the data, and the complexity of data processing. Project types may include, but are not limited to, full data migration, incremental data migration, data structure transformation, data quality improvement, and data model reconstruction. Each type has its specific processing logic. The specific project type needs to be determined based on the actual file to be tested, and is not limited here. Project types can be used to clarify project categories and project processing logic, thereby ensuring that data processing is carried out according to the corresponding project type and processing logic.
[0063] In one optional embodiment, by acquiring the project information contained in the file to be verified and determining its project type, and further determining the project processing logic based on the project type, a more accurate and efficient strategy is provided for file verification during the data migration process. The above process can formulate a verification scheme that matches the characteristics of the data migration project, ensuring the targeting and effectiveness of the verification process. For example, for data structure transformation projects, the accuracy of data field mapping can be verified; for data quality improvement projects, the verification of data cleaning effects needs to be strengthened. By deeply integrating project information with project type, this application can achieve comprehensive analysis and refined management of data migration projects, significantly improving the accuracy and efficiency of data migration, thereby reducing risks during the data migration process and ensuring business continuity and data security.
[0064] Optionally, in the file verification method provided in this application embodiment, obtaining the project type corresponding to the project information contained in the file to be verified includes: determining the project type based on the identification information of the project information; or, determining historical project information with a similarity greater than a preset similarity with the project information, and determining the historical project type corresponding to the historical project information as the project type.
[0065] The aforementioned identification information can refer to a set of key data used to distinguish the characteristics of different projects or files. Identification information may include, but is not limited to, key metadata such as the name, version, type, creation date, modification history, business domain, data source, and data target of the project or file. The specific identification information needs to be determined according to the actual situation and is not limited here. Identification information can be used to connect the file to be verified with the project type to quickly determine the project type to which the file belongs.
[0066] The aforementioned preset similarity can refer to a pre-defined similarity threshold, used to determine the degree of similarity between the project information of the file to be verified and historical project information. When the similarity between the two exceeds the preset similarity, they can be considered to be similar in project type, thus allowing for the application of relevant processing experience from historical project types.
[0067] The aforementioned historical project information can refer to the project details and data processing information recorded in past data migration projects. Historical project information may include, but is not limited to, the project's objectives, scope, process, and results. The specific historical project information needs to be determined based on the system's records, and is not limited here. Historical project information can serve as an important part of enterprise knowledge management, helping to accumulate and pass on management experience and technical knowledge from data migration projects.
[0068] The aforementioned historical project types refer to categories summarized based on historical project information. These historical project types reflect the characteristics and processing logic of different data migration tasks, such as data structure transformation, data quality improvement, and full data migration. Historical project types can be used as templates for new projects to guide the planning and execution of new projects, thereby improving the consistency and efficiency of project management.
[0069] In one optional embodiment, the project type can be determined in two ways: first, based on the project information's identifier; second, by comparing the similarity between the project information and historical project information with a preset similarity, and determining the historical project type corresponding to the historical project information with a similarity greater than the preset similarity as the project type. This process not only simplifies the project classification process and improves the accuracy of project type determination, enhancing the system's adaptability to unknown or non-standard project types, but also provides strong guidance and assurance for data processing of new projects through knowledge reuse and risk assessment.
[0070] Optionally, in the file verification method provided in this application embodiment, determining the associated file of the file to be verified based on the file verification rules includes: determining the project data chain corresponding to the file verification rules, wherein the project data chain is used to represent a data chain containing at least one project node, and different project nodes are used to achieve different project purposes; determining the project node position of the file to be verified according to the project data chain, wherein the project node position is used to represent the position of the project node corresponding to the file to be verified in the project data chain; and determining the associated file according to the project node position and the project data chain.
[0071] The aforementioned project data chain can refer to a set of data processing steps and technical nodes that the data goes through from its original generation point to its final use point during data migration or data processing. The types of project data chains can include, but are not limited to, full data migration chains, incremental data migration chains, data quality improvement chains, and data structure transformation chains. The specific project data chain needs to be determined according to the project type and project purpose, and is not limited here. Project data chains can be used to track every step of data processing, facilitate auditing and problem localization, perform process control, and improve the efficiency and accuracy of data migration.
[0072] The aforementioned project node can refer to a specific data processing unit in the project data chain. A project node represents a key step or function in the data processing process, such as reading the data source, preprocessing the data, validating the data, or writing the data target.
[0073] The project node positions mentioned above refer to the arrangement order and position of project nodes in the project data chain. The project node positions reflect the flow order of data processing, that is, the flow path of data from one node to another in the project data chain. The project node positions can be used to control the flow and order of data in the processing process, ensure the continuity of data processing, clarify the dependencies of data processing, and help manage the dependencies between data processing steps.
[0074] In one optional embodiment, the project data chain is determined based on file verification rules, and the position of the file to be verified within the project node of the project data chain is determined accordingly. Furthermore, the associated files of the file to be verified are identified, achieving refined management and automated control of file verification during data migration. This process effectively addresses the neglect of data processing flow and node dependencies in traditional file verification. By clearly defining the project data chain, not only can the data processing flow be comprehensively tracked, but the position of the file within the data processing process can also be accurately located. This allows for more accurate and efficient identification of upstream and downstream associated files of the file to be verified, ensuring the continuity of data processing and the targeted nature of data verification.
[0075] Optionally, in the file verification method provided in this application embodiment, determining the project data chain corresponding to the file verification rule includes: determining at least one data processing logic involved in the file verification rule; in response to the fact that the at least one data processing logic contains multiple data processing logics, concatenating the multiple data processing logics based on the logical order of the multiple data processing logics to obtain the project data chain; and in response to the fact that the at least one data processing logic contains a single data processing logic, determining the single data processing logic as the project data chain.
[0076] The aforementioned logical order refers to the sequential and dependent relationships between various data processing logics or steps in a data processing flow. Types of logical order include, but are not limited to, linear order, branching order, looping order, and parallel order. The specific logical order needs to be determined based on the actual situation and is not limited here. Logical order ensures the coherence and consistency of data processing, reflecting the entire processing path of data from input to output and from the source system to the target system.
[0077] The aforementioned chaining refers to connecting multiple data processing logics in their logical order to form a complete project data chain. Chaining types can include, but are not limited to, linear chaining, branching chaining, looping chaining, and parallel chaining. The specific chaining type needs to be determined based on the logical order and is not limited here. The chaining process ensures seamless connection between data processing logics, allowing data to be passed from one processing logic to the next in a predetermined flow order until the end of the data processing.
[0078] In one optional embodiment, the project data chain is determined based on at least one data processing logic in the file verification rules, including the following two cases: For cases involving multiple data processing logics, they are sequentially linked according to their inherent relationships and execution order to construct a complex but ordered project data chain. This process not only ensures the continuity of data processing but also effectively avoids errors and omissions in the data processing process through explicit dependency management, improving the comprehensiveness and accuracy of file verification. When the file verification rule involves only a single data processing logic, it is directly determined as the project data chain, thereby simplifying the generation process of the project data chain and improving the efficiency and response speed of file verification. Especially in scenarios where data migration requirements are relatively simple and data processing logic is relatively simple, this method can quickly locate and verify files, reduce unnecessary processing steps, and save time and computing resources.
[0079] The file verification method provided in this application provides powerful guidance data for the file verification process by obtaining the associated files and associated information of the file to be verified. At the same time, it ensures that all related files are included in the verification scope, thereby avoiding omissions and errors, reducing the risk of data migration, improving the integrity and accuracy of data migration, and thus solving the technical problem of low file verification accuracy in related technologies.
[0080] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0081] Example 2
[0082] This application also provides a file verification device. It should be noted that the file verification device of this application can be used to execute the file verification method provided in this application. The file verification device provided in this application is described below.
[0083] According to an embodiment of this application, an apparatus for implementing the above-described document verification method is also provided. Figure 3 This is a schematic diagram of a document verification device according to an embodiment of this application, such as... Figure 3As shown, the device includes: a first acquisition module 302, used to acquire file verification rules for the file to be verified, wherein the file verification rules are used to guide the verification process of the file to be verified; a first determination module 304, used to determine the associated files of the file to be verified based on the file verification rules, wherein the associated files are used to represent files that have a data interaction relationship with the file to be verified; a second determination module 306, used to determine the association information of the file to be verified according to the association relationship between the associated files and the file to be verified, wherein the association information is used to represent the information involved in the data interaction process between the file to be verified and the associated files; and a verification module 308, used to verify the file to be verified based on the association information and obtain a verification result, wherein the verification result is used to indicate whether there is an anomaly in the file to be verified.
[0084] The file verification device and method provided in this application embodiment guide the verification process by obtaining the associated files and associated information of the file to be verified. This provides powerful guidance data information for the file verification process and ensures that all related files are included in the verification scope, thereby avoiding omissions and errors, reducing the risk of data migration, improving the integrity and accuracy of data migration, and thus solving the technical problem of low file verification accuracy in related technologies.
[0085] Optionally, in the file verification device provided in this application embodiment, the verification module includes: a comparison unit, used to compare the associated information and the file to be verified to obtain the difference data between the associated information and the file to be verified; a first determining unit, used to determine the verification result as the file to be verified having an anomaly in response to the difference data meeting a preset condition, wherein the preset condition is a pre-set condition for indicating whether the file to be verified has an anomaly; and a second determining unit, used to determine the verification result as the file to be verified not having an anomaly in response to the difference data not meeting the preset condition.
[0086] Optionally, in the file verification device provided in this application embodiment, the associated files include an upstream associated file and a downstream associated file. The second determining module includes: a third determining unit, used to determine the output data of the upstream associated file and the reference data of the downstream associated file according to the association relationship, wherein the data output by the upstream associated file is applied to the file to be verified, and the data output by the file to be verified is applied to the downstream associated file. The output data is used to represent the data to be input to the file to be verified after data processing in the upstream associated file, and the reference data is used to represent the data to be input to the file to be verified; and an input unit, used to input the output data and the reference data into the project processing logic corresponding to the file to be verified to obtain the association information, wherein the project processing logic is used to represent the operations required to implement the project corresponding to the file to be verified.
[0087] Optionally, in the file verification device provided in this application embodiment, the device further includes: a second acquisition module, used to acquire the project type corresponding to the project information contained in the file to be verified; and a third determination module, used to determine the project processing logic corresponding to the project information based on the project type.
[0088] Optionally, in the document verification device provided in this application embodiment, the second acquisition module includes: a fourth determining unit, used to determine the project type based on the identification information of the project information; or, a fifth determining unit, used to determine historical project information with a similarity greater than a preset similarity with the project information, and determine the historical project type corresponding to the historical project information as the project type.
[0089] Optionally, in the file verification device provided in this application embodiment, the first determining module includes: a sixth determining unit, used to determine the project data chain corresponding to the file verification rule, wherein the project data chain is used to represent a data chain containing at least one project node, and different project nodes are used to achieve different project purposes; a seventh determining unit, used to determine the project node position of the file to be verified according to the project data chain, wherein the project node position is used to represent the position of the project node corresponding to the file to be verified in the project data chain; and an eighth determining unit, used to determine the associated file according to the project node position and the project data chain.
[0090] Optionally, in the document verification device provided in this application embodiment, the sixth determining unit includes: determining at least one data processing logic involved in the document verification rule; in response to the fact that the at least one data processing logic contains multiple data processing logics, concatenating the multiple data processing logics based on the logical order of the multiple data processing logics to obtain a project data chain; and in response to the fact that the at least one data processing logic contains a single data processing logic, determining the single data processing logic as a project data chain.
[0091] It should be noted that the above-mentioned modules or units may be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above-mentioned modules may also be part of the device and may run in the computer terminal 10 provided in Embodiment 1.
[0092] Example 3
[0093] Embodiments of this application may provide an electronic device. Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 4 As shown, the electronic device may include: one or more ( Figure 4Only one of the following is shown: processor 402, memory 404, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.
[0094] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0095] The processor can invoke information and application programs stored in the memory via a transmission device to perform the following steps: obtaining file verification rules for the file to be verified, wherein the file verification rules are used to guide the verification process of the file to be verified; determining associated files of the file to be verified based on the file verification rules, wherein associated files are used to represent files that have a data interaction relationship with the file to be verified; determining the association information of the file to be verified based on the association relationship between the associated files and the file to be verified, wherein the association information is used to represent the information involved in the data interaction process between the file to be verified and the associated files; verifying the file to be verified based on the association information to obtain a verification result, wherein the verification result is used to indicate whether the file to be verified has any anomalies.
[0096] The processor can also invoke information and application programs stored in the memory via the transmission device to perform the following steps: compare the associated information and the file to be verified to obtain the difference data between the associated information and the file to be verified; in response to the difference data meeting the preset conditions, determine that the verification result is that the file to be verified has an anomaly, wherein the preset conditions are pre-set conditions used to indicate whether the file to be verified has an anomaly; in response to the difference data not meeting the preset conditions, determine that the verification result is that the file to be verified does not have an anomaly.
[0097] The processor can also invoke information and applications stored in the memory via the transmission device to perform the following steps: determining the output data of the upstream associated file and the reference data of the downstream associated file based on the association relationship, wherein the output data of the upstream associated file is applied to the file to be verified, and the output data of the file to be verified is applied to the downstream associated file, the output data is used to represent the data to be input to the file to be verified after data processing of the upstream associated file, and the reference data is used to represent the data to be input to the file to be verified; inputting the output data and reference data into the project processing logic corresponding to the file to be verified to obtain the association information, wherein the project processing logic is used to represent the operations required to implement the project corresponding to the file to be verified.
[0098] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: obtain the project type corresponding to the project information contained in the file to be verified; determine the project processing logic corresponding to the project information based on the project type.
[0099] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: determine the project type based on the identification information of the project information; or, determine the historical project information with a similarity greater than a preset similarity with the project information, and determine the historical project type corresponding to the historical project information as the project type.
[0100] The processor can also invoke information and applications stored in the memory via the transmission device to perform the following steps: determining the project data chain corresponding to the file verification rule, wherein the project data chain is used to represent a data chain containing at least one project node, and different project nodes are used to achieve different project purposes; determining the project node position of the file to be verified based on the project data chain, wherein the project node position is used to represent the position of the project node corresponding to the file to be verified in the project data chain; and determining the associated file based on the project node position and the project data chain.
[0101] The processor can also invoke information and applications stored in the memory via the transmission device to perform the following steps: determining at least one data processing logic involved in the file verification rule; in response to the fact that the at least one data processing logic contains multiple data processing logics, concatenating the multiple data processing logics based on the logical order of the multiple data processing logics to obtain a project data chain; in response to the fact that the at least one data processing logic contains a single data processing logic, determining the single data processing logic as a project data chain.
[0102] This application provides a file verification scheme. The file verification method provided in this application guides the verification process by obtaining associated files and related information of the file to be verified. This provides strong guidance data for the file verification process and ensures that all related files are included in the verification scope, thereby avoiding omissions and errors, reducing the risk of data migration, and improving the integrity and accuracy of data migration. This solves the technical problem of low file verification accuracy in related technologies.
[0103] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.
[0104] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0105] Example 4
[0106] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the file verification method provided in Embodiment 1.
[0107] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.
[0108] This application also provides a computer program product that, when executed on a data processing device, is suitable for performing file verification method steps.
[0109] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0110] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0111] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.
[0112] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0113] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0114] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0115] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A file verification method, characterized in that, include: Obtain the file verification rules for the file to be verified, wherein the file verification rules are used to guide the verification process of the file to be verified; The associated files of the file to be verified are determined based on the file verification rules, wherein the associated files are used to represent files that have a data interaction relationship with the file to be verified; Based on the association relationship between the associated file and the file to be verified, the association information of the file to be verified is determined, wherein the association information is used to represent the information involved in the data interaction between the file to be verified and the associated file; The file to be verified is verified based on the association information to obtain a verification result, wherein the verification result is used to indicate whether the file to be verified is abnormal; The associated files include upstream associated files and downstream associated files. Based on the association relationship between the associated files and the file to be verified, the association information of the file to be verified is determined, including: The output data of the upstream associated file is determined based on the association relationship, and the reference data of the downstream associated file is determined. The data output by the upstream associated file is applied to the file to be verified, and the data output by the file to be verified is applied to the downstream associated file. The output data is used to represent the data of the upstream associated file to be input to the file to be verified after data processing, and the reference data is used to represent the data to be input to the file to be verified. The output data and the reference data are input into the project processing logic corresponding to the file to be verified to obtain the association information. The project processing logic is used to represent the operations required to implement the project corresponding to the file to be verified.
2. The method according to claim 1, characterized in that, The file to be verified is verified based on the associated information to obtain the verification result, including: The association information and the file to be verified are compared to obtain the difference data between the association information and the file to be verified; In response to the difference data satisfying a preset condition, the verification result is determined to be that the file to be verified has the anomaly, wherein the preset condition is a pre-set condition used to indicate whether the file to be verified has the anomaly; In response to the discrepancy data not meeting the preset conditions, the verification result is determined to be that the file to be verified does not contain the anomaly.
3. The method according to claim 1, characterized in that, The method further includes: Obtain the project type corresponding to the project information contained in the file to be verified; The project processing logic corresponding to the project information is determined based on the project type.
4. The method according to claim 3, characterized in that, Retrieve the project type corresponding to the project information contained in the file to be verified, including: The project type is determined based on the identification information of the project information; Alternatively, identify historical project information with a similarity greater than a preset similarity to the project information, and determine the historical project type corresponding to the historical project information as the project type.
5. The method according to claim 1, characterized in that, Based on the file verification rules, the associated files of the file to be verified are determined, including: Determine the project data chain corresponding to the file verification rule, wherein the project data chain is used to represent a data chain containing at least one project node, and different project nodes are used to achieve different project purposes; The project node position of the file to be verified is determined according to the project data chain, wherein the project node position is used to indicate the position of the project node corresponding to the file to be verified in the project data chain; The associated file is determined based on the project node location and the project data chain.
6. The method according to claim 5, characterized in that, Determining the project data chain corresponding to the file verification rule includes: Determine at least one data processing logic involved in the file verification rule; In response to the fact that the at least one data processing logic contains multiple data processing logics, the multiple data processing logics are connected in series based on their logical order to obtain a project data chain; In response to the fact that the at least one data processing logic includes a single data processing logic, the single data processing logic is determined to be the project data chain.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 6.
8. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 6.
9. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 6.