Data comparison method and system, electronic equipment and storage medium
Through Java reflection mechanism and annotation technology, dynamic comparison of multi-system data is solved, and the problem of complicated repetition and poor scalability of comparison logic is achieved, flexible and automated data comparison is achieved, and development efficiency and scalability are improved.
Patent Information
- Application Number
- CN202411033357.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-12
AI Technical Summary
In the prior art, the comparison logic of multi-system data consistency verification is duplicated and complicated and difficult to expand, resulting in increased maintenance difficulty and violation of the opening and closing principle, and poor scalability.
Using Java reflection mechanism and annotation technology, dynamically traverse attributes in the data model, judge whether comparison and associated business systems are needed based on the annotation, realize automated comparison logic, reduce code duplication related to specific business systems and fields, and core comparison codes Curing.
It greatly simplifies the writing and maintenance of comparison logic, improves development efficiency and scalability, enhances the flexibility of data comparison, and can automatically adapt to business scenarios and demand changes.
Smart Images

Figure CN118939671A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data comparison method, system, electronic device and storage medium. Background Art
[0002] When verifying the consistency of data in multiple business systems, data comparison is involved. The current practice is to compare each system and each field one by one. For example, field a of system one is compared with field a of system two, field b of system one is compared with field b of system two, and so on, until the data comparison of all fields between system one and system two is completed. Similarly, other systems (such as system one and system three, system one and system four, etc.) are also compared in this way. Therefore, the current comparison logic involves a large amount of code related to specific business systems and specific fields. The comparison logic is repetitive and complicated and difficult to expand, and the amount of comparison code is large.
[0003] Assuming there are m business systems, each of which has n fields to be compared, the amount of comparison code is (m-1)×n, such as Figure 1 As shown in the figure, there are 6 business systems, each of which has 20 fields to be compared, and each field needs to be compared 5 times, so the amount of comparison code is 100. When the data comparison requirements change, such as adding a new business system or a new field for comparison, the original comparison code must be modified, which not only increases the difficulty of maintenance, but also violates the open-closed principle and has poor scalability. Summary of the invention
[0004] The present application provides a data comparison method, system, electronic device and storage medium to at least solve the problem in the prior art that the comparison logic of multi-system data consistency verification is repetitive and complicated and difficult to expand.
[0005] In a first aspect, the present application provides a data comparison method, comprising:
[0006] Receive data comparison instructions;
[0007] Using the Java reflection mechanism to traverse the attributes in the data model, wherein the data model includes at least one attribute and an annotation of the attribute, the attributes in the data model correspond to the fields in the multiple business systems involved in the comparison, and the annotation is used to at least mark whether the corresponding attribute needs to be compared and the business system associated with the attribute;
[0008] Each time an attribute is traversed, according to the annotation corresponding to the attribute, if the attribute needs to be compared, the Java reflection mechanism is used to obtain the attribute value of the attribute in each business system associated with it for comparison.
[0009] In a possible implementation manner, before receiving the data comparison instruction, the method further includes:
[0010] Receive a custom operation and customize the annotation;
[0011] receiving construction information, obtaining at least one field from the construction information, and storing the at least one field as different attributes according to a unique identifier of the field, wherein the field obtained from the construction information is a field in the multiple business systems involved in the comparison;
[0012] At least one field feature corresponding to the at least one field is obtained from the construction information, and annotations are added to each attribute according to the at least one field feature to complete the construction of the data model, wherein the field feature includes at least: whether the field needs to be compared, and the business system associated with the field.
[0013] In a possible implementation, receiving a data comparison instruction includes:
[0014] Receiving a data change message from any business system among the multiple business systems to trigger data comparison; and / or,
[0015] Receiving information about a newly added field in an existing business system, updating the data model according to the information about the newly added field, and triggering data comparison; and / or,
[0016] The information of the newly added business system is received, and after the data model is updated according to the information of the newly added business system, data comparison is triggered.
[0017] In a possible implementation, updating the data model according to the information of the newly added field includes:
[0018] Acquire a unique identifier of the newly added field from the information of the newly added field, and store the newly added field as a new attribute according to the unique identifier of the newly added field;
[0019] At least whether the newly added field needs to be compared and the business system associated with the newly added field are obtained from the information of the newly added field, and annotations are added to the new attributes.
[0020] In a possible implementation, updating the data model according to the information of the newly added business system includes:
[0021] Determining, according to the information of the newly added business system, whether each field in the newly added business system is a newly added field or an existing field in the existing business system;
[0022] For the newly added field in the newly added business system, store the newly added field as a new attribute according to the unique identifier of the newly added field in the newly added business system; and obtain from the information of the newly added business system whether the newly added field needs to be compared and the business system associated with the newly added field, add annotations to the new attribute, so as to update the data model;
[0023] For the existing fields in the newly added business system, the attribute corresponding to the existing field is searched in the data model according to the unique identifier of the existing field, and the newly added business system is added as the associated system of the attribute to the annotation corresponding to the attribute to update the data model.
[0024] In a possible implementation manner, the data change message is received after a preset delay time after data change occurs in any business system among the multiple business systems.
[0025] In a possible implementation, each time an attribute is traversed, according to the annotation corresponding to the attribute, when the attribute needs to be compared, the attribute value of the attribute in each business system associated with the attribute is obtained by using the Java reflection mechanism for comparison, including:
[0026] Each time an attribute is traversed, the annotation corresponding to the attribute is obtained from the data model;
[0027] According to the annotation corresponding to the attribute, determine whether the attribute needs to be compared;
[0028] If the attribute needs to be compared, determine the business system associated with the attribute based on the annotation corresponding to the attribute;
[0029] Use Java reflection mechanism to obtain the attribute value from the business system associated with the attribute;
[0030] Compare the obtained attribute values.
[0031] In a possible implementation, the data comparison method of the present application further includes:
[0032] When constructing a data model, the query interface of each business system is called to perform data query. If the data types of the same fields in different business systems are different, the data types of the same fields are normalized in the data model.
[0033] When updating the data model, when the newly added field is stored as a new attribute, the query interface of the business system associated with the newly added field is called to perform data query on the newly added field. If the data types of the newly added field are different in different business systems, the data types of the newly added field are normalized in the data model.
[0034] In a second aspect, the present application provides a data comparison system, comprising:
[0035] A receiving module, used for receiving a data comparison instruction;
[0036] A traversal module, used to traverse the attributes in the data model using a Java reflection mechanism, wherein the data model includes at least one attribute and an annotation of the attribute, the attributes in the data model correspond to fields in multiple business systems involved in the comparison, and the annotation is used to at least mark whether the corresponding attribute needs to be compared and the business system associated with the attribute;
[0037] The comparison module is used to traverse an attribute each time, and according to the annotation corresponding to the attribute, when the attribute needs to be compared, use the Java reflection mechanism to obtain the attribute value of the attribute in each business system associated with it for comparison.
[0038] In a third aspect, the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data comparison method described in the embodiment of the present application when executing the computer program.
[0039] In a fourth aspect, the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the data comparison method described in the embodiments of the present application is implemented.
[0040] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:
[0041] The data comparison method provided in the embodiment of the present application utilizes the Java reflection mechanism to traverse the attributes in the data model, analyzes the annotations of the attributes, determines whether the attributes need to be compared and the business system to which they are associated, dynamically collects data and compares the data of the relevant business system. With the data model as the core, the comparison logic is abstracted, and the attributes can be flexibly marked by annotations to mark the comparison range (i.e., the attributes that need to be compared) and the actual business system to be compared. The Java reflection mechanism dynamically obtains information so that the comparison logic can be automatically performed, thereby eliminating the need to manually write a large amount of comparison codes related to specific business systems and specific fields, greatly reducing repetitive and templated codes, and the core comparison code is solidified. Even if a new business system or field is added, there is no need to modify the core comparison code. It is only necessary to add the corresponding attributes and annotations to the data model, which greatly simplifies the writing and maintenance of the comparison logic, not only improving the development efficiency, but also enhancing the flexibility and extensibility of the data comparison, so that the data comparison system can better adapt to different business scenarios and demand changes, and solves the problem that the comparison logic of multi-system data consistency verification in the prior art is repeated and complicated and difficult to expand. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0044] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0045] Figure 1 It is a schematic diagram of data comparison of the prior art;
[0046] Figure 2 A schematic diagram of a data comparison method provided in an embodiment of the present application;
[0047] Figure 3 A schematic diagram of data comparison provided for the embodiments of the present application;
[0048] Figure 4 A schematic diagram of an overall solution for monitoring data consistency during system reconstruction provided in an embodiment of the present application;
[0049] Figure 5 A structural block diagram of a data comparison system provided in an embodiment of the present application;
[0050] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0052] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.
[0053] In order to solve the technical problems in the prior art that the comparison logic of multi-system data consistency verification is repetitive and complicated, lacks flexibility and scalability, the present application provides a data comparison method based on annotations and Java reflection mechanism. The comparison logic does not contain code related to specific business systems or specific fields, and the core comparison code is solidified. Even if a new business system or field is added, there is no need to modify the core comparison code, which greatly simplifies the writing and maintenance of the comparison logic, improves development efficiency, and enhances scalability.
[0054] Figure 2 A flow chart of a data comparison method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method comprises the following steps:
[0055] S201, receiving a data comparison instruction. That is, when there is a data comparison requirement, the data comparison is triggered and the core comparison code is executed.
[0056] S202, using the Java reflection mechanism to traverse the attributes in the data model, wherein the data model includes at least one attribute and an annotation of the attribute, the attributes in the data model correspond to fields in multiple business systems involved in the comparison, and the annotation is at least used to mark whether the corresponding attribute needs to be compared and the business system associated with the attribute.
[0057] S203, each time an attribute is traversed, according to the annotation corresponding to the attribute, if the attribute needs to be compared, the attribute value of the attribute in each business system associated with it is obtained by using the Java reflection mechanism for comparison.
[0058] This application will initially construct a data model based on the business systems involved in the comparison, so as to ensure that the data comparison can be performed with the help of the data model in the future. For example, there are three business systems, respectively recorded as system 1, system 2 and system 3. System 1 and system 2 both include fields a, field b and field c, and system 3 only includes fields a and field b. Correspondingly, the constructed data model includes: three attributes corresponding to field a, field b and field c (respectively recorded as attribute a, attribute b and attribute c), and annotations of the attributes. Through the annotations in the data model, at least it can be known whether each attribute needs to be compared and the business system associated with the attribute. The business system associated with the attribute refers to the business system to which the field corresponding to the attribute belongs. For example, attribute a needs to be compared and the business systems associated with attribute a are system 1, system 2 and system 3, attribute b does not need to be compared and the business systems associated with attribute b are system 1, system 2 and system 3, attribute c needs to be compared and the business systems associated with attribute c are system 1 and system 2. For example, the business system is a commodity system, and the fields include: commodity name, commodity price, commodity inventory, etc. Each field in the business system has a corresponding value. The fields in the business system correspond to the attributes in the data model. The values of the fields in the business system can also be called attribute values or field values. Java reflection mechanism is used to dynamically obtain information.
[0059] The data comparison method provided in the embodiment of the present application utilizes the Java reflection mechanism to traverse the attributes in the data model, analyzes the annotations of the attributes, determines whether the attributes need to be compared and the business system to which they are associated, dynamically collects data and compares the data of the relevant business system. With the data model as the core, the comparison logic is abstracted, and the attributes can be flexibly marked by annotations to mark the comparison range (i.e., the attributes that need to be compared) and the actual business system to be compared. The Java reflection mechanism dynamically obtains information so that the comparison logic can be automatically performed, thereby eliminating the need to manually write a large amount of comparison codes related to specific business systems and specific fields, greatly reducing repetitive and templated codes, and the core comparison code is solidified. Even if a new business system or field is added, there is no need to modify the core comparison code. It is only necessary to add the corresponding attributes and annotations to the data model, which greatly simplifies the writing and maintenance of the comparison logic, not only improving the development efficiency, but also enhancing the flexibility and extensibility of the data comparison, so that the data comparison system can better adapt to different business scenarios and demand changes, and solves the problem that the comparison logic of multi-system data consistency verification in the prior art is repeated and complicated and difficult to expand.
[0060] Before the data comparison system performs data comparison normally, it is necessary to construct a data model according to the business systems involved in the comparison. Specifically, in one embodiment of the present application, before receiving the data comparison instruction, it also includes: receiving a custom operation to customize the annotation; receiving construction information, obtaining at least one field from the construction information, and storing the at least one field as different attributes according to the unique identifier of the field, wherein the field obtained from the construction information is a field in the multiple business systems involved in the comparison; obtaining at least one field feature corresponding to the at least one field from the construction information, and adding annotations to each attribute according to the at least one field feature to complete the construction of the data model, wherein the field feature includes at least: whether the field needs to be compared, and the business system associated with the field.
[0061] Among them, the custom operation is used to customize the annotation, mainly to customize the items and item meanings contained in the annotation. If the items and item meanings contained in the annotation do not change, you only need to perform customization once when constructing the data model. Through custom annotations, you can flexibly specify the attributes that need to be compared and their associated business systems to meet the requirements of different business scenarios and demand changes, and realize the flexibility and automation of comparison logic.
[0062] The construction information can be manually input by the staff, and the construction information includes at least: at least one field, and at least one field feature corresponding to the at least one field. Based on the construction information, the attributes in the data model can be configured and annotations can be added to the attributes to complete the construction of the data model. The data model can be updated later according to actual conditions.
[0063] This embodiment does not limit the execution order of the step "receiving a custom operation and customizing the annotation" and the step "receiving construction information, obtaining at least one field from the construction information, and storing the at least one field as different attributes according to the unique identifier of the field". The annotation can be customized first and then the attribute can be stored, or the attribute can be stored first and then the annotation can be customized, as long as the customized annotation is performed before adding an annotation to the attribute.
[0064] Specifically, annotations can be added to each attribute, and whether the attribute needs to be compared can be marked in the annotation; annotations can also be added only for attributes that need to be compared, and for attributes that do not have annotations (it can also be understood that the annotation of the attribute is empty), it is directly considered that the attribute does not need to be compared; all attributes that need to be compared and / or all attributes that do not need to be compared can also be listed in the annotation. For example, the number of attributes that do not need to be compared is small, then the attributes that do not need to be compared can be listed in the annotation, which is equivalent to exclusion. Except for the listed attributes, the remaining attributes need to be compared. The specific form of adding annotations can be determined according to actual conditions. This embodiment does not limit this, as long as the attributes that need to be compared and the business system associated with the attributes can be known through the annotations.
[0065] This embodiment constructs a data model for all business systems involved in the comparison. Through annotations, it can flexibly specify the attributes that need to be compared and their associated business systems to meet the requirements of different business scenarios and changes in demand. If the fields are expanded later, it is only necessary to update the data model without modifying the core comparison code. The processing is more flexible and convenient, and the comparison logic can be automated, thereby improving the flexibility of the comparison.
[0066] Attributes can carry their own data types, or the data types of attributes can be annotated in annotations.
[0067] Considering that the data types of the same field in different business systems may be different, the data types can be normalized. Specifically, when constructing the data model, the query interface of each business system is called to query the data. If the data types of the same field in different business systems are different, the data types of the same field are normalized in the data model. Normalization can be achieved manually or automatically. For example, 5 business systems all include field e (i.e., the same field mentioned above). The data type of field e in 4 business systems is A, and the data type in another business system is B. The data type A can be automatically determined as the normalization result of field e.
[0068] This embodiment can ensure data comparability by normalizing the data types of fields (or attributes) when constructing a data model.
[0069] A change in the value of an existing field in any business system, a new field, or a new business system will trigger data comparison. In an optional embodiment, receiving a data comparison instruction includes:
[0070] Receiving a data change message from any business system among the multiple business systems to trigger data comparison; and / or,
[0071] Receiving information about a newly added field in an existing business system, updating the data model according to the information about the newly added field, and triggering data comparison; and / or,
[0072] The information of the newly added business system is received, and after the data model is updated according to the information of the newly added business system, data comparison is triggered.
[0073] Among them, the data change message can carry the unique identifier of the field whose value has changed. The information of the newly added field can be entered by the staff, and the information of the newly added field can include: the unique identifier of the newly added field, whether the newly added field needs to be compared, the business system associated with the newly added field, etc. Since it involves the newly added field, the data model needs to be updated. The information of the newly added business system can be entered by the staff, and the newly added business system can include existing fields and / or newly added fields. The information of the newly added business system can include: the unique identifier of each field in the newly added business system, whether the newly added field needs to be compared, the business system associated with the newly added field, etc. Since it involves the newly added business system, the data model needs to be updated.
[0074] In the case of a change in the value of an existing field in any business system, this embodiment triggers data comparison, which can verify whether the value of the field in its associated business system has been synchronized, thereby ensuring data consistency of related business systems. In the case of a new field, first update the data model, then trigger data comparison, thereby ensuring that the data comparison of the newly added field is included when executing the core comparison code, thereby ensuring data consistency of related business systems; in the case of a new business system, first update the data model, then trigger data comparison, thereby ensuring that the data comparison of the newly added field and the existing field is included when executing the core comparison code, thereby ensuring data consistency of related business systems; only the data model needs to be updated, and there is no need to modify the core comparison code, which makes the processing more flexible and convenient, and enhances the flexibility and extensibility of data comparison.
[0075] The update of the data model in different situations is explained below.
[0076] (1) Updating the data model according to the information of the newly added field, including:
[0077] Acquire a unique identifier of the newly added field from the information of the newly added field, and store the newly added field as a new attribute according to the unique identifier of the newly added field;
[0078] At least whether the newly added field needs to be compared and the business system associated with the newly added field are obtained from the information of the newly added field, and annotations are added to the new attributes.
[0079] In the case of adding new fields, this embodiment adds new attributes and annotations to the data model to update the data model, thereby ensuring that the data comparison of the new fields is included when executing the core comparison code, thereby ensuring data consistency of related business systems.
[0080] (2) Updating the data model according to the information of the newly added business system, including:
[0081] Determining, according to the information of the newly added business system, whether each field in the newly added business system is a newly added field or an existing field in the existing business system;
[0082] For the newly added field in the newly added business system, store the newly added field as a new attribute according to the unique identifier of the newly added field in the newly added business system; and obtain from the information of the newly added business system whether the newly added field needs to be compared and the business system associated with the newly added field, add annotations to the new attribute, so as to update the data model;
[0083] For the existing fields in the newly added business system, the attribute corresponding to the existing field is searched in the data model according to the unique identifier of the existing field, and the newly added business system is added as the associated system of the attribute to the annotation corresponding to the attribute to update the data model.
[0084] In the case of a new business system, this embodiment adds new attributes and annotations to the data model for the newly added fields, and modifies the existing related annotations in the data model for the existing fields, so as to update the data model, thereby ensuring that the data comparison of the newly added fields and the existing fields is included when the core comparison code is executed, thereby ensuring the data consistency of the related business systems.
[0085] If any of the multiple business systems makes a data change, the data comparison system immediately receives a data change message. At this time, other business systems may not have time to modify the value of the corresponding field, resulting in inconsistent comparison and wasting resources. In order to solve this problem, in one embodiment of the present application, the data change message is received after a preset time delay after the data change occurs in any of the multiple business systems. For example, any of the multiple business systems sends a data change message after a preset time delay after the data change occurs, or any of the multiple business systems sends a data change message with a delay after the data change occurs (that is, the data comparison system will receive the data change message only after the preset time delay). The preset time can be flexibly adjusted according to actual needs, for example, 1 minute.
[0086] This embodiment introduces a delayed comparison mechanism, which delays a certain period of time after the source system data is changed before informing the data comparison system to trigger the data comparison. This can ensure that the relevant business system has enough time to complete data writing, avoid inconsistent data comparison due to different data writing timings, and improve the efficiency and accuracy of data comparison.
[0087] Similar to constructing a data model, updating a data model may also involve differences in the data types of the same fields in different business systems. Specifically, when updating a data model, when a newly added field is stored as a new attribute, the query interface of the business system associated with the newly added field is called to perform data query on the newly added field. If the data types of the newly added field in different business systems are different, the data types of the newly added field are normalized in the data model. This embodiment can ensure data comparability by normalizing the data types of newly added fields (or newly added attributes) when updating the data model.
[0088] Specifically, each time an attribute is traversed, according to the annotation corresponding to the attribute, when the attribute needs to be compared, the attribute value of the attribute in each business system associated with it is obtained by using the Java reflection mechanism for comparison, including: each time an attribute is traversed, the annotation corresponding to the attribute is obtained from the data model; according to the annotation corresponding to the attribute, whether the attribute needs to be compared is determined; if the attribute needs to be compared, the business system associated with the attribute is determined according to the annotation corresponding to the attribute; the attribute value is obtained from the business system associated with the attribute by using the Java reflection mechanism; and the obtained attribute value is compared. This embodiment uses annotations and Java reflection mechanisms, and there is no need to manually write a large amount of comparison code related to specific business systems and specific fields. The core comparison code is solidified, which greatly simplifies the writing and maintenance of the comparison logic, reduces the workload of developers and the possibility of errors, and thus reduces the cost of development and maintenance.
[0089] Exemplarily, determining whether the attribute needs to be compared based on the annotation corresponding to the attribute includes: judging whether there is a corresponding annotation for the attribute; if there is no corresponding annotation for the attribute, determining that the attribute does not need to be compared; if there is a corresponding annotation for the attribute, determining whether the attribute needs to be compared based on the content in the annotation corresponding to the attribute.
[0090] If the comparison result of step S203 is inconsistent, the inconsistent attributes and the attribute values of the attributes in the associated business systems are stored for page display, and an alarm message is output to notify relevant personnel in time to repair the data according to specific business needs.
[0091] The data comparison method is described below in conjunction with a specific embodiment. However, it is worth noting that the specific embodiment is only for better illustrating the present application and does not constitute an improper limitation on the present application. The explanation of terms that are the same or corresponding to the above embodiment will not be repeated in this embodiment.
[0092] The data comparison method can be executed by a data comparison system. The data comparison system has the functions of constructing data models, updating data models, comparing data, and feedback on comparison results. Among them, the functions of constructing data models and updating data models can be executed in combination with the information input by the staff. The data comparison function involves the core comparison code written in advance by the developer. The main comparison logic of the core comparison code is as follows: Use the Java reflection mechanism to traverse the attributes in the data model, determine whether the attribute needs to be compared and the business system associated with the attribute based on the annotation of the attribute, and for the attributes that need to be compared (called the attributes to be compared), use the Java reflection mechanism to obtain the value of the attribute to be compared in its associated system, and compare the obtained values.
[0093] refer to Figure 3 , take 6 business systems as an example, respectively denoted as system 1 to system 6, construct a data model according to the fields in these 6 business systems, and specifically call the query interface of each business system to query data. If the data type of the same field in different business systems is different, the data type of the field is normalized. The constructed data model includes: at least one attribute (denoted as attribute 1 to attribute n) and the annotation of the attribute. The attributes in the data model correspond to the fields in the business system. The annotation of the attribute is at least used to mark whether the corresponding attribute needs to be compared and the business system associated with the attribute. When the data comparison is triggered, the core comparison logic is executed: the Java reflection mechanism is used to loop through the attributes in the data model, and the annotation of the attribute is obtained to determine whether the attribute needs to be compared and the business system associated with the attribute. The Java reflection mechanism is used to dynamically obtain the value of the attribute to be compared in its associated system, and the obtained value is compared to determine whether the data is consistent.
[0094] Combine the following Figure 4 The data comparison process is explained using data change as an example.
[0095] refer to Figure 4During the system reconstruction, data changes involved 7 business systems. In order to ensure the data consistency of each business system, a delayed comparison mechanism was introduced. The source system (that is, any business system among the 7 business systems, taking system 1 as an example) sent a delayed message after the data was changed. The message contained the unique identifier of the field where the data changed. The data comparison system will receive the message after a preset time, thereby ensuring that all business systems participating in the comparison have enough time to complete data writing, avoiding inconsistent comparisons caused by different data writing timings.
[0096] After receiving the delayed data change message, the data comparison system triggers the data comparison and starts to execute the core comparison logic: using the Java reflection mechanism to traverse the attributes in the data model, each time an attribute is traversed, it is determined whether the attribute needs to be compared according to the annotation corresponding to the attribute. If the attribute needs to be compared, the business system associated with the attribute is determined according to the annotation corresponding to the attribute and the value of the attribute is obtained from the business system associated with the attribute using the Java reflection mechanism. The obtained value is compared to determine whether the data is consistent, and the inconsistent data is stored in the database. After that, the page can be displayed according to the information stored in the warehouse. At the same time, an alarm message is sent when the data comparison is inconsistent, and the relevant personnel are notified in time to repair the data according to specific business needs.
[0097] This specific embodiment is based on Java reflection and annotation, which simplifies the data comparison of multiple business systems. With the data model as the core, the comparison logic is abstracted so that the comparison logic does not contain comparison codes related to specific business systems and specific fields, thereby improving development efficiency. The core comparison code is solidified, and even if a new business system or field is added, it is only necessary to add corresponding attributes and annotations to the data model without modifying the core logic of the comparison data, thereby enhancing scalability and enabling the data comparison system to better adapt to different business scenarios and demand changes. Compared with the existing traditional data comparison methods, the solution of the present application greatly reduces repetitive and templated codes, reducing the amount of comparison code related to specific business systems and specific fields to zero, making the processing of later expanded fields more flexible and convenient, and solving the problems of repeated and complicated comparison logic for multi-system data consistency verification in the prior art, and lack of flexibility and scalability.
[0098] Through the annotations in the data model, the comparison scope and the actual business system to be compared can be flexibly specified to meet the requirements of different business scenarios and demand changes. The Java reflection mechanism is used to dynamically obtain attribute values so that the comparison logic can be automated, which greatly simplifies the writing and maintenance of the comparison code, thereby achieving flexible and automated comparison logic based on annotations and Java reflection mechanisms. In addition, it can reduce the workload of developers and the possibility of errors, thereby reducing the cost of development and maintenance. By introducing a delayed comparison mechanism and an automated comparison process, the efficiency and accuracy of the comparison can be improved, and manual intervention and human errors can be reduced.
[0099] The present application also provides a data comparison system, which can be used to implement the data comparison method described in the above embodiment. The data comparison system can be implemented by software and / or hardware.
[0100] Figure 5 The structural block diagram of the data comparison system provided in the embodiment of the present application is as follows: Figure 5 As shown, the data comparison system includes:
[0101] Receiving module 501, used for receiving data comparison instructions;
[0102] A traversal module 502 is used to traverse the attributes in the data model using a Java reflection mechanism, wherein the data model includes at least one attribute and an annotation of the attribute, the attributes in the data model correspond to fields in multiple business systems involved in the comparison, and the annotation is used to at least mark whether the corresponding attribute needs to be compared and the business system associated with the attribute;
[0103] The comparison module 503 is used to traverse an attribute each time, and according to the annotation corresponding to the attribute, when the attribute needs to be compared, obtain the attribute value of the attribute in each business system associated with it by using the Java reflection mechanism for comparison.
[0104] In a possible implementation, the data comparison system further includes:
[0105] A custom module, used for receiving a custom operation and customizing the annotation before receiving a data comparison instruction;
[0106] A storage module, used to receive construction information, obtain at least one field from the construction information, and store the at least one field as different attributes according to a unique identifier of the field, wherein the field obtained from the construction information is a field in the multiple business systems involved in the comparison;
[0107] An adding module is used to obtain at least one field feature corresponding to the at least one field from the construction information, and to add annotations to each attribute according to the at least one field feature to complete the construction of the data model, wherein the field feature includes at least: whether the field needs to be compared, and the business system associated with the field.
[0108] In a possible implementation, the receiving module 501 includes:
[0109] A first trigger unit is configured to receive a data change message sent by any business system among the multiple business systems to trigger data comparison; and / or,
[0110] A second trigger unit is configured to receive information about a newly added field in an existing business system, and trigger data comparison after updating the data model according to the information about the newly added field; and / or,
[0111] The third trigger unit is used to receive information about a newly added business system, and trigger data comparison after updating the data model according to the information about the newly added business system.
[0112] In a possible implementation, the second trigger unit includes:
[0113] The first update sub-unit is used to obtain the unique identifier of the newly added field from the information of the newly added field, and store the newly added field as a new attribute according to the unique identifier of the newly added field; and to obtain at least whether the newly added field needs to be compared and the business system associated with the newly added field from the information of the newly added field, and add annotations to the new attributes.
[0114] In a possible implementation manner, the third trigger unit includes:
[0115] A determination subunit, configured to determine, based on the information of the newly added business system, whether each field in the newly added business system is a newly added field or an existing field in the existing business system;
[0116] The second updating subunit is configured to store the newly added field in the newly added business system as a new attribute according to the unique identifier of the newly added field in the newly added business system; and obtain from the information of the newly added business system whether the newly added field needs to be compared and the business system associated with the newly added field, and add annotations to the new attributes to update the data model;
[0117] The third updating sub-unit is used to search for the attribute corresponding to the existing field in the newly added business system in the data model according to the unique identifier of the existing field, and add the newly added business system as the associated system of the attribute to the annotation corresponding to the attribute to update the data model.
[0118] In a possible implementation manner, the data change message is received after a preset delay time after data change occurs in any business system among the multiple business systems.
[0119] In a possible implementation, the comparison module 503 includes:
[0120] A first acquisition unit, configured to acquire an annotation corresponding to an attribute from the data model each time an attribute is traversed;
[0121] A first determining unit, configured to determine whether the attribute needs to be compared according to an annotation corresponding to the attribute;
[0122] A second determining unit is used to determine the business system associated with the attribute according to the annotation corresponding to the attribute if the attribute needs to be compared;
[0123] A second acquisition unit is used to acquire the attribute value from the business system associated with the attribute by using a Java reflection mechanism;
[0124] The comparison unit is used to compare the obtained attribute values.
[0125] In a possible implementation, the data comparison system further includes:
[0126] A normalization module is used to call the query interface of each business system to perform data query when constructing a data model. If the data type of the same field in different business systems is different, the data type of the same field is normalized in the data model; and when updating the data model, when a newly added field is stored as a new attribute, call the query interface of the business system associated with the newly added field to perform data query on the newly added field. If the data type of the newly added field in different business systems is different, the data type of the newly added field is normalized in the data model.
[0127] The data comparison system can execute the data comparison method provided in the above embodiment, and has the corresponding functional modules and beneficial effects of executing the data comparison method. For technical details not described in detail in this embodiment, please refer to the data comparison method provided in the above embodiment.
[0128] The present application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data comparison method described in the above embodiment of the present application when executing the computer program.
[0129] like Figure 6 , which is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, the electronic device includes a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.
[0130] In some embodiments, the processor 111 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 111 is generally used to control the overall operation of the electronic device, such as performing control and processing related to data interaction or communication. In this embodiment, the processor 111 is used to run the program code stored in the memory 113 or process data, such as running a data comparison program code.
[0131] The communication interface 112 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and the communication interface 112 may also be used to establish a communication connection between the electronic device and other electronic devices.
[0132] The memory 113 is used to store computer programs, such as data comparison programs. The memory 113 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 113 can be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. In other embodiments, the memory 113 can also be an external storage device of the electronic device, such as a plug-in hard disk equipped with the electronic device, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. Of course, the memory 113 can also include both the internal storage unit of the electronic device and its external storage device. In this embodiment, the memory 113 is generally used to store the operating system and various computer programs installed in the electronic device, such as data comparison program code, etc. In addition, the memory 113 can also be used to temporarily store various types of data that have been output or are to be output.
[0133] Figure 6 Only an electronic device having components 111 - 114 is shown, but it is understood that implementing all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.
[0134] In one embodiment of the present application, the processor 111 is used to execute the program stored in the memory 113 to implement the data comparison method provided by any of the aforementioned method embodiments, including: receiving a data comparison instruction; using a Java reflection mechanism to traverse the attributes in the data model, wherein the data model includes at least one attribute and an annotation of the attribute, and the attributes in the data model correspond to fields in multiple business systems participating in the comparison, and the annotation is at least used to mark whether the corresponding attribute needs to be compared and the business system associated with the attribute; each time an attribute is traversed, according to the annotation corresponding to the attribute, if the attribute needs to be compared, the Java reflection mechanism is used to obtain the attribute value of the attribute in each business system associated with it for comparison.
[0135] For a detailed description of the above steps, please refer to the specific description of the aforementioned data comparison method embodiment.
[0136] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the data comparison method described in the embodiment of the present application is implemented.
[0137] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0139] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.
[0140] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A data comparison method, characterized in that: include: Receive data comparison instructions; Using the Java reflection mechanism to traverse the attributes in the data model, wherein the data model includes at least one attribute and an annotation of the attribute, the attributes in the data model correspond to the fields in the multiple business systems involved in the comparison, and the annotation is used to at least mark whether the corresponding attribute needs to be compared and the business system associated with the attribute; Each time an attribute is traversed, according to the annotation corresponding to the attribute, if the attribute needs to be compared, the Java reflection mechanism is used to obtain the attribute value of the attribute in each business system associated with it for comparison.
2. The method according to claim 1, characterized in that Before receiving the data comparison instruction, it also includes: Receive a custom operation and customize the annotation; receiving construction information, obtaining at least one field from the construction information, and storing the at least one field as different attributes according to a unique identifier of the field, wherein the field obtained from the construction information is a field in the multiple business systems involved in the comparison; At least one field feature corresponding to the at least one field is obtained from the construction information, and annotations are added to each attribute according to the at least one field feature to complete the construction of the data model, wherein the field feature includes at least: whether the field needs to be compared, and the business system associated with the field.
3. The method according to claim 1, characterized in that Receive data comparison instructions, including: Receiving a data change message from any business system among the multiple business systems to trigger data comparison; and / or, Receiving information about a newly added field in an existing business system, updating the data model according to the information about the newly added field, and triggering data comparison; and / or, The information of the newly added business system is received, and after the data model is updated according to the information of the newly added business system, data comparison is triggered.
4. The method according to claim 3, characterized in that Updating the data model according to the information of the newly added field includes: Acquire a unique identifier of the newly added field from the information of the newly added field, and store the newly added field as a new attribute according to the unique identifier of the newly added field; At least whether the newly added field needs to be compared and the business system associated with the newly added field are obtained from the information of the newly added field, and annotations are added to the new attributes.
5. The method according to claim 3, characterized in that: Updating the data model according to the information of the newly added business system includes: Determining, according to the information of the newly added business system, whether each field in the newly added business system is a newly added field or an existing field in the existing business system; For the newly added field in the newly added business system, store the newly added field as a new attribute according to the unique identifier of the newly added field in the newly added business system; and obtain from the information of the newly added business system whether the newly added field needs to be compared and the business system associated with the newly added field, add annotations to the new attribute, so as to update the data model; For the existing fields in the newly added business system, the attribute corresponding to the existing field is searched in the data model according to the unique identifier of the existing field, and the newly added business system is added as the associated system of the attribute to the annotation corresponding to the attribute to update the data model.
6. The method according to claim 3, characterized in that The data change message is received after a preset time delay after data change occurs in any business system among the multiple business systems.
7. The method according to claim 1, characterized in that Each time an attribute is traversed, according to the annotation corresponding to the attribute, if the attribute needs to be compared, the Java reflection mechanism is used to obtain the attribute value of the attribute in each business system associated with it for comparison, including: Each time an attribute is traversed, the annotation corresponding to the attribute is obtained from the data model; According to the annotation corresponding to the attribute, determine whether the attribute needs to be compared; If the attribute needs to be compared, determine the business system associated with the attribute based on the annotation corresponding to the attribute; Use Java reflection mechanism to obtain the attribute value from the business system associated with the attribute; Compare the obtained attribute values.
8. The method according to claim 2 or 3, characterized in that: Also includes: When constructing a data model, the query interface of each business system is called to perform data query. If the data types of the same fields in different business systems are different, the data types of the same fields are normalized in the data model. When updating the data model, when the newly added field is stored as a new attribute, the query interface of the business system associated with the newly added field is called to perform data query on the newly added field. If the data types of the newly added field are different in different business systems, the data types of the newly added field are normalized in the data model.
9. A data comparison system, characterized in that: include: A receiving module, used for receiving a data comparison instruction; A traversal module, used to traverse the attributes in the data model using a Java reflection mechanism, wherein the data model includes at least one attribute and an annotation of the attribute, the attributes in the data model correspond to fields in multiple business systems involved in the comparison, and the annotation is used to at least mark whether the corresponding attribute needs to be compared and the business system associated with the attribute; The comparison module is used to traverse an attribute each time, and according to the annotation corresponding to the attribute, when the attribute needs to be compared, use the Java reflection mechanism to obtain the attribute value of the attribute in each business system associated with it for comparison.
10. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data comparison method according to any one of claims 1 to 8 when executing the computer program.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data comparison method according to any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
A method and apparatus for changing association of object attributes
CN109284101A
Universal configurable cross-system data comparison system and method
CN110275904A
Table data verification method and device
CN110347999A
Method for realizing data verification based on Java reflection technology
CN113805863A
Object data comparison method and device and electronic equipment
CN117149791A
Cited By
Code generation method and device, vehicle and storage medium
CN119806496A
