Log recording method and device during data synchronization between systems, equipment and medium

Abstract

The invention discloses a log recording method and device during data synchronization between systems, equipment and a medium, and relates to the technical field of JAVA back-ends, and the method comprises the following steps: obtaining request parameters in a synchronization request message, and carrying out log recording by using a preset relational database. And after converting the request parameter data, performing log recording on the to-be-synchronized information in a data conversion result. Judging whether the to-be-synchronized information meets conditions or not in the data synchronization process, if yes, analyzing important attribute change conditions and performing log recording after synchronization operation is completed, and if not, determining synchronization failure information and performing log recording. Therefore, the complete change in the data synchronization process between the systems is flexibly and accurately recorded, the abnormal information, the synchronization failure reason and the change of the important attribute information in the data synchronization process are conveniently checked, and the accuracy and the reliability of data synchronization are ensured.

Description

Technical Field

[0001] The present invention relates to the field of JAVA backend technology, and in particular to a log recording method, device, equipment and medium for data synchronization between systems. Background Art

[0002] In today's digital age, data synchronization between systems has become a key link in data interaction and sharing. However, the current data synchronization between systems is insufficient in terms of log recording. The lack of a complete log recording mechanism makes it difficult to quickly locate and solve problems when they occur during the data synchronization process, which brings difficulties to subsequent data analysis and troubleshooting, and the accuracy and reliability of data synchronization are difficult to guarantee.

[0003] In summary, how to implement a complete log recording mechanism during data synchronization is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a log recording method, device, equipment and medium for data synchronization between systems, which can realize a perfect log recording mechanism during the data synchronization process. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a log recording method for data synchronization between systems, including:

[0006] Obtaining a request parameter in the current synchronization request message based on a preset string data type, and logging the request parameter based on a preset relational database to complete a first logging operation;

[0007] Performing data conversion on the request parameters based on a preset statement, and using the preset relational database to log each piece of information to be synchronized in the obtained data conversion result, so as to complete a second log recording operation;

[0008] During the data synchronization process, it is determined whether each piece of information to be synchronized meets the preset data synchronization condition to obtain a corresponding determination result;

[0009] If the judgment result indicates that the condition is satisfied, after triggering and completing the data synchronization operation of the corresponding information to be synchronized, the important attribute information and the changes of the information to be synchronized are analyzed and logged based on the preset relational database, the preset important attribute categories and the obtained post-synchronization information, so as to complete the third log recording operation;

[0010] If the judgment result indicates that it is not satisfied, the synchronization failure information of the corresponding information to be synchronized is determined based on the judgment result, and the synchronization failure information is logged using the preset relational database to complete the fourth log recording operation.

[0011] Optionally, obtaining a request parameter in the current synchronization request message based on a preset string data type, and logging the request parameter based on a preset relational database includes:

[0012] Get the request parameters in the current synchronization request message based on the string data type;

[0013] Generate log record information corresponding to the request parameter based on a first log table in a preset relational database to obtain a corresponding first system log;

[0014] The first system log includes a batch information synchronization identifier as a primary key, the request parameters, the synchronization type, the first log generation time, the number of batch information synchronization items, the number of successful information synchronization items, and the number of failed information synchronization items.

[0015] Optionally, the log recording method for data synchronization between the systems further includes:

[0016] After all the data synchronization operations are completed based on the judgment results corresponding to the respective pieces of the information to be synchronized, the number of successful data synchronization operations and the number of failed data synchronization operations are counted to obtain the corresponding number of successful information synchronizations and the number of failed information synchronizations;

[0017] Determine the number of batch synchronization items of the information based on the number of successful synchronization items and the number of failed synchronization items of the information;

[0018] The first system log in the first log table is updated using the number of successful information synchronizations, the number of failed information synchronizations, and the number of batch information synchronizations.

[0019] Optionally, performing data conversion on the request parameter based on a preset statement, and using the preset relational database to log each piece of information to be synchronized in the obtained data conversion result, includes:

[0020] Based on the preset statement, each piece of information to be synchronized in the request parameter is converted into a preset object, and a corresponding list to be synchronized is determined according to each converted information obtained;

[0021] Based on the second log table in the preset relational database, log each of the converted information in the to-be-synchronized list one by one to obtain a second system log corresponding to each of the converted information;

[0022] The second system log includes corresponding identity information, second log generation time, corresponding character string information of the converted information, and batch information synchronization identifier.

[0023] Optionally, after triggering and completing the data synchronization operation of the corresponding information to be synchronized, analyzing the changes of the important attribute information and the information to be synchronized based on the preset relational database, the preset important attribute categories and the obtained post-synchronization information and recording them in a log, including:

[0024] triggering a data synchronization operation of the corresponding information to be synchronized, and after the data synchronization operation is completed, analyzing changes in the important attribute information based on the preset relational database, the preset important attribute category and the obtained post-synchronization information to obtain a first analysis result;

[0025] Based on the first analysis result, log record information corresponding to the first analysis result is generated in a third log table in the preset relational database to obtain a corresponding third system log;

[0026] The third system log includes the identity information, the batch information synchronization identifier, the third log generation time, the target important attribute name, and the corresponding initial target important attribute value and updated target important attribute value.

[0027] Optionally, after triggering and completing the data synchronization operation of the corresponding information to be synchronized, analyzing the changes of the important attribute information and the information to be synchronized based on the preset relational database, the preset important attribute categories and the obtained post-synchronization information and recording them in a log, including:

[0028] After the corresponding data synchronization operation is completed, analyzing the change of the information to be synchronized based on the preset relational database and the obtained post-synchronization information to obtain a second analysis result;

[0029] Based on the second analysis result, generating log record information corresponding to the second analysis result in a fourth log table in the preset relational database to obtain a corresponding fourth system log;

[0030] The fourth system log includes the corresponding batch information synchronization identifier, the identity information, and the initial information and changed information of the information to be synchronized.

[0031] Optionally, determining the synchronization failure information of the corresponding information to be synchronized based on the judgment result, and using the preset relational database to log the synchronization failure information includes:

[0032] The synchronization failure information of the corresponding information to be synchronized is determined based on the judgment result, and the log record information corresponding to the synchronization failure information is generated by using the fifth log table in the preset relational database to obtain the corresponding fifth system log.

[0033] In a second aspect, the present application provides a log recording device for data synchronization between systems, including:

[0034] A first log recording module, used for obtaining a request parameter in a current synchronization request message based on a preset string data type, and logging the request parameter based on a preset relational database to complete a first log recording operation;

[0035] A second log recording module, used to perform data conversion on the request parameters based on a preset statement, and log each piece of information to be synchronized in the obtained data conversion result using the preset relational database, so as to complete a second log recording operation;

[0036] The data judgment module is used to judge whether each piece of information to be synchronized meets the preset data synchronization condition during the data synchronization process to obtain a corresponding judgment result;

[0037] A third log recording module is used for analyzing the changes of the important attribute information and the information to be synchronized based on the preset relational database, the preset important attribute category and the obtained post-synchronization information and performing log recording if the judgment result indicates that the conditions are met, after triggering and completing the data synchronization operation of the corresponding information to be synchronized, so as to complete the third log recording operation;

[0038] The fourth log recording module is used to determine the synchronization failure information of the corresponding information to be synchronized based on the judgment result if the judgment result indicates that it is not satisfied, and use the preset relational database to log the synchronization failure information to complete the fourth log recording operation.

[0039] In a third aspect, the present application provides an electronic device, including:

[0040] Memory, used to store computer programs;

[0041] The processor is used to execute the computer program to implement the log recording method for data synchronization between the aforementioned systems.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the log recording method for data synchronization between the aforementioned systems is implemented.

[0043] In the present application, the request parameters in the current synchronization request message are obtained based on the preset string data type, and the request parameters are logged based on the preset relational database to complete the first log recording operation; the request parameters are converted based on the preset statement, and the preset relational database is used to log each piece of information to be synchronized in the obtained data conversion result to complete the second log recording operation; in the process of data synchronization, it is judged whether each piece of information to be synchronized meets the preset data synchronization condition to obtain the corresponding judgment result; if the judgment result shows that it meets, then after triggering and completing the data synchronization operation of the corresponding information to be synchronized, the important attribute information and the change of the information to be synchronized are analyzed based on the preset relational database, the preset important attribute category and the obtained post-synchronization information, and the log is recorded to complete the third log recording operation; if the judgment result shows that it does not meet, then the synchronization failure information of the corresponding information to be synchronized is determined based on the judgment result, and the synchronization failure information is logged using the preset relational database to complete the fourth log recording operation. As can be seen from the above, the present application obtains the request parameters in the synchronization request message and logs with the preset relational database. After the request parameter data is converted, the information to be synchronized in the data conversion result is logged. During the data synchronization process, it is determined whether the information to be synchronized meets the conditions. If it meets the conditions, the important attribute changes are analyzed and logged after the synchronization operation is completed. If it does not meet the conditions, the synchronization failure information is determined and logged. In this way, the application flexibly and accurately records the complete changes in the data synchronization process between systems, making it convenient to check abnormal information in the data synchronization process, the cause of synchronization failure, and the changes in important attribute information. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0045] Figure 1 A flow chart of a log recording method for data synchronization between systems disclosed in the present application;

[0046] Figure 2 A log recording flow chart for synchronizing user data disclosed in this application;

[0047] Figure 3 A schematic diagram of the structure of a log recording device for data synchronization between systems disclosed in the present application;

[0048] Figure 4 This is a structural diagram of a log recording device for data synchronization between systems disclosed in this application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] At present, data synchronization between systems has deficiencies in log recording and lacks a perfect log recording mechanism, which makes it difficult to quickly locate and solve problems when problems occur during data synchronization, which brings difficulties to subsequent data analysis and troubleshooting, and the accuracy and reliability of data synchronization are difficult to guarantee. To this end, this application provides a log recording solution for data synchronization between systems to implement a perfect log recording mechanism during data synchronization.

[0051] See also Figure 1 As shown, an embodiment of the present invention discloses a log recording method for data synchronization between systems, including:

[0052] Step S11: obtaining request parameters in the current synchronization request message based on a preset character string data type, and logging the request parameters based on a preset relational database to complete a first logging operation.

[0053] In this embodiment, it is necessary to obtain the request parameters in the current synchronization request message based on the string data type. In the data synchronization scenario, the request parameters are the relevant information sent to the server to clarify the specific operation and the data involved when performing data synchronization related operations. It can be understood that in the data synchronization process, the formats of the request parameters may vary, and obtaining the request parameters with the string data type can completely save the synchronization request message. Among them, the request parameter message is generally JSON (JavaScript Object Notation) array format data. If the request parameters are not received as a string data type, but are directly parsed into a specific data structure, some additional information may be lost. Because in this direct parsing process, only the attributes defined in the object can be received, while using the string type to receive can save the complete request message, including those non-standard attributes that may be needed in subsequent analysis.

[0054] It should be noted that after obtaining the request parameters, log record information corresponding to the request parameters is generated based on the first log table in the preset relational database to obtain the corresponding first system log; in this embodiment, the preset relational database can be a MySQL (My Structured Query Language, relational database management system) database, which can efficiently store and manage a large amount of log data and provide reliable support for log records in the data synchronization process. Among them, the first system log includes a batch information synchronization identifier as a primary key, request parameters, this synchronization type, the first record generation time, the number of batch information synchronization items, the number of successful information synchronization items, and the number of failed information synchronization items.

[0055] Specifically, in this embodiment, the batch information synchronization identifier as the primary key is unique, which can clearly distinguish different batches of data synchronization operations, so that the synchronization records of a specific batch can be quickly and accurately located and retrieved in a large number of system logs; the synchronization type of this time helps managers understand the purpose of data synchronization operations, and the synchronization types of this time include but are not limited to types such as addition, update, and deletion; the first record time provides a reference for the time dimension of the data synchronization process; the number of information batch synchronization items is the sum of the number of successful information synchronization items and the number of failed information synchronization items, which are key indicators for measuring the quality and effect of synchronization operations. They directly reflect the execution status of this synchronization task, and are crucial for timely discovery and resolution of problems in the synchronization process.

[0056] Step S12: performing data conversion on the request parameters based on a preset statement, and using the preset relational database to log each piece of information to be synchronized in the obtained data conversion result, so as to complete a second log recording operation.

[0057] In this embodiment, each piece of information to be synchronized in the request parameter is converted into a preset object based on a preset statement, and a corresponding list to be synchronized is determined based on each converted information. The preset object may be a specific data structure defined according to system design and business requirements, and the preset object may store various attribute information in a structured manner. For example, if the data to be synchronized is user data, the preset object may contain attributes such as user id (identity document), user name, user age, etc., to facilitate subsequent operations and management.

[0058] After determining the list to be synchronized, based on the second log table in the preset relational database, log each of the converted information in the list to be synchronized one by one to obtain a second system log corresponding to each of the converted information; wherein the second system log includes the corresponding identity information, the second record generation time, the corresponding string information of the converted information and the batch information synchronization identifier. Specifically, the identity information can be a code for uniquely identifying each piece of information to be synchronized; the string information of the converted information saves the detailed content of the converted information in the form of a string, which is convenient for directly viewing the specific content of the converted information when there is a specific need;

[0059] Step S13: During the data synchronization process, it is determined whether each piece of information to be synchronized meets a preset data synchronization condition to obtain a corresponding determination result.

[0060] In this embodiment, it is necessary to perform conditional judgment on each piece of information to be synchronized to obtain a judgment result on whether each piece of information to be synchronized can be synchronized. For example, whether identity information exists, whether necessary information is missing in the request parameters, and whether the format of certain request parameters meets the requirements can all be used as preset data synchronization conditions.

[0061] Step S14: If the judgment result indicates that it is satisfied, after triggering and completing the corresponding data synchronization operation of the information to be synchronized, analyze the changes of the important attribute information and the information to be synchronized based on the preset relational database, the preset important attribute category and the obtained post-synchronization information and perform log recording to complete the third log recording operation.

[0062] In this embodiment, it is necessary to analyze the changes in important attribute information. Therefore, if each piece of information to be synchronized meets the preset data synchronization conditions, the data synchronization operation of the corresponding information to be synchronized is triggered, and after the data synchronization operation is completed, the changes in the important attribute information are analyzed based on the preset relational database, the preset important attribute categories and the obtained post-synchronization information to obtain the first analysis result. Among them, the preset important attribute categories are pre-set during the data synchronization process between systems according to business needs, data characteristics and key factors of system operation, and are used to identify and distinguish attribute categories with important significance and influence.

[0063] After obtaining the first analysis result, log record information corresponding to the first analysis result is generated in the third log table in the preset relational database based on the first analysis result to obtain the corresponding third system log; wherein the third system log includes identity information, batch information synchronization identifier, third log generation time, target important attribute name, and corresponding initial target important attribute value and updated target important attribute value. Among them, recording the initial target important attribute value and the updated target important attribute value of the target important attribute in the third system log can clearly present the changes of the target important attribute during the data synchronization process, which is convenient for subsequent data comparison and analysis. For example, in business data synchronization, you can intuitively understand the changes before and after a key indicator, which is helpful to evaluate the impact of data synchronization on the business.

[0064] In this embodiment, a comprehensive analysis is also required for the overall changes in the information to be synchronized. Therefore, if each piece of information to be synchronized meets the preset data synchronization conditions, the data synchronization operation of the corresponding information to be synchronized is triggered, and after the data synchronization operation is completed, the changes in the information to be synchronized are analyzed based on the preset relational database and the obtained post-synchronization information to obtain a second analysis result. That is, based on the preset relational database and the obtained post-synchronization information, a comprehensive comparison is performed on the entire information to be synchronized to determine all changes that have occurred during the data synchronization process. This change analysis not only covers important attributes, but also includes other information that may affect data integrity and consistency, and the second analysis result is obtained through comprehensive analysis.

[0065] After obtaining the second analysis result, log record information corresponding to the second analysis result is generated in a fourth log table in a preset relational database based on the second analysis result to obtain a corresponding fourth system log; wherein the fourth system log includes a corresponding batch information synchronization identifier, the identity identification information, and the initial information and changed information of the information to be synchronized.

[0066] The functions of batch information synchronization identification and identity identification information are as described above, while the initial information of the information to be synchronized records the original state of the data before synchronization, and the changed information describes in detail the final state of the data after the synchronization operation. Through this complete recording method, the data change process can be fully recorded.

[0067] Step S15: If the judgment result indicates that the condition is not satisfied, the synchronization failure information of the corresponding information to be synchronized is determined based on the judgment result, and the synchronization failure information is logged using the preset relational database to complete the fourth log recording operation.

[0068] In this embodiment, if a piece of information to be synchronized does not meet the preset data synchronization conditions, the synchronization failure information of the corresponding information to be synchronized is determined based on the judgment result, and the fifth log table in the preset relational database is used to generate log record information corresponding to the synchronization failure information to obtain the corresponding fifth system log. The preset data synchronization conditions and the synchronization failure information are closely linked. For example, the absence of identity information, the lack of necessary information in the request parameters, and the format of certain request parameters that do not meet the requirements can all be used as synchronization failure information. Recording the synchronization failure information in the fifth log table is of great guiding significance for subsequent analysis of the root cause of the problem and formulation of solutions. The administrator can make targeted data adjustments and corrections to the synchronization information based on the synchronization failure information to improve the success rate of data synchronization.

[0069] In this embodiment, in order to count and feedback the results of batch data synchronization and monitor and manage the execution of data synchronization operations from an overall perspective, after completing all the data synchronization operations based on the judgment results corresponding to each piece of information to be synchronized, the number of successful data synchronization operations and the number of failed data synchronization operations can be counted to obtain the corresponding number of successful information synchronizations and the number of failed information synchronizations.

[0070] Specifically, in this embodiment, the number of successful and failed data synchronization operations must first be counted to obtain the corresponding number of successful information synchronizations and the number of failed information synchronizations. Then, the number of batch information synchronizations is determined based on the number of successful information synchronizations and the number of failed information synchronizations. Finally, the first system log in the first log table is updated using the number of successful information synchronizations, the number of failed information synchronizations, and the number of batch information synchronizations. Specifically, the updated first system log can display the overall situation of this batch synchronization, which not only facilitates the subsequent tracing and analysis of the data synchronization process, but also provides timely and accurate information to system administrators, so that administrators can make corresponding decisions based on the data synchronization results, such as troubleshooting and repairing failed synchronization operations.

[0071] As can be seen from the above, the present application obtains the request parameters in the synchronization request message and uses a preset relational database for log recording. After the request parameter data is converted, the information to be synchronized in the data conversion result is logged. During the data synchronization process, it is determined whether the information to be synchronized meets the conditions. If it meets the conditions, the important attribute changes are analyzed and logged after the synchronization operation is completed. If it does not meet the conditions, the synchronization failure information is determined and logged. In this way, the present application flexibly and accurately records the complete changes in the data synchronization process between systems, which is convenient for troubleshooting abnormal information in the data synchronization process, causes of synchronization failure, and changes in important attribute information.

[0072] Combine the following Figure 2 As shown, taking synchronization of user data as an example, the technical solution of the embodiment of the present application is specifically described.

[0073] In this example, first, you need to obtain the request parameters in the current synchronization request message. In order to speed up the synchronization of data, data synchronization is performed in batches. The interface provider generally provides three interfaces, including batch adding user interface, batch updating user interface, and batch deleting user interface. The specific code segment can be as follows:

[0074] @PostMapping(" / addUserList")

[0075] public ResponseBean addUserList(@RequestBody String param){

[0076] ResponseBean responseBean = syncUserService.addUserList(param);

[0077] return responseBean;

[0078] }

[0079] @PostMapping(" / updateUserList")

[0080] public ResponseBean updateUserList(@RequestBody String param){

[0081] ResponseBean responseBean = syncUserService.updateUserList(param);

[0082] return responseBean;

[0083] }

[0084] @PostMapping(" / deleteUserList")

[0085] public ResponseBean deleteUserList(@RequestBody String param){

[0086] ResponseBean responseBean = syncUserService.deleteUserList(param);

[0087] return responseBean;

[0088] }

[0089] In this embodiment, after obtaining the request parameters, the request parameters need to be recorded in the first log table, that is, the batch data request parameter log table sync_data_batch_log. The table mainly stores the following information: the randomly generated serial number batch_id (i.e., batch information synchronization identifier) ​​as the primary key, the request parameter content (i.e., the request parameter), the synchronization type, the time when the log was created (i.e., the first log generation time), the number of batch data synchronizations (i.e., the number of information batch synchronizations), the number of successful data synchronizations (i.e., the number of successful information synchronizations), and the number of failed data synchronizations (i.e., the number of failed information synchronizations). Among them, the synchronization type can include adding, updating, and deleting, and if the amount of data of the request parameters is relatively large, it can be stored in text. The specific code segment can be as follows:

[0090] SyncBatchBean syncBatchBean = new SyncBatchBean();

[0091] syncBatchBean.setId(batchId);

[0092] syncBatchBean.setContent(param);

[0093] syncBatchBean.setType(type);

[0094] syncBatchBean.setErrorType(0);

[0095] syncBatchMapper.insertSyncBatch(syncBatchBean);

[0096] In this embodiment, next, each piece of user information in the request parameter is converted into a UserBean object to obtain a user list userList containing multiple UserBean objects. By traversing the user list userList, the converted information is recorded one by one in the second log table, that is, the single record log table sync_data_single_log. The information saved in the sync_data_single_log table includes: user code (that is, identity information), string information of the converted information, batch information id (that is, batch information synchronization identifier), time of creating the log (that is, second log generation time) and other information. It should be emphasized that when adding data to the single record log table, exception handling should be added to ensure that the exception of a single piece of data does not affect the processing of other data. The specific code segment can be as follows:

[0097] List <userbeanbean>userList = JSONArray.parseArray(param,UserBeanBean.class);

[0098] for(UserBean userBean : userList){

[0099] try{

[0100] SyncSingleRecordBean syncSingleRecordBean = new SyncSingleRecordBean();

[0101] syncSingleRecordBean.setId(id);

[0102] syncSingleRecordBean.setType(type);

[0103] syncSingleRecordBean.setBatchId(batchId);

[0104] syncSingleRecordBean.setUserCode(userBean.getUserCode());

[0105] syncSingleRecordBean.setContent(JSON.toJSONString(userBean));

[0106] syncSingleRecordMapper.insertSyncSingleRecord(syncSingleRecordBean);

[0107] }catch (Exception e) {

[0108] log.error(e);

[0109] / / Other processing

[0110] …

[0111] }

[0112] }

[0113] Taking the update interface as an example, write the updateUser method to update the user information to the system's business table user table. The specific code segment can be as follows:

[0114] for(UserBean userBean : userList){

[0115] ResponseBean responseBean = null;

[0116] try{

[0117] responseBean = updateUser(userBean);

[0118] }catch (Exception e){

[0119] log.error("Failed to update user information", e);

[0120] try{

[0121] SyncErrorLogBean logBean = new SyncErrorLogBean();

[0122] logBean.setId(UUID.randomUUID().toString());

[0123] logBean.setUserCode(userBean.getUserCode());

[0124] logBean.setSyncType(syncType);

[0125] logBean.setContent(JSON.toJSONString(userBean));

[0126] logBean.setBatchId(batchId);

[0127] logBean.setErrorMsg(errorMsg);

[0128] syncErrorLogMapper.addSyncErrorLog(logBean);

[0129] }catch (Exception e) {

[0130] log.error("failed to save synchronization user log error", e);

[0131] }

[0132] continue;

[0133] }

[0134] }

[0135] In this embodiment, during the data synchronization process, it will be checked whether the data meets the requirements. For example, the user code is used to query whether there is a user in the user table. If not, an exception needs to be thrown. The exception information clearly states that the user corresponding to a certain user code does not exist, and the required information is not filled in the parameters. Some parameter formats do not meet the requirements, etc., which can all throw exceptions and capture them outside the call to the updateUser method, and save the exception information to the fifth log table, that is, the sync_error_log table in the synchronization failure list. The specific code segment can be as follows:

[0136] @Transactional(rollbackFor = Exception.class, propagation =Propagation.REQUIRES_NEW)

[0137] public ResponseBean updateUser(UserBean newUserBean){

[0138] / / Get the original user information

[0139] UserBean oldUserBean = userMapper.getUserByUserCode(newUserBean.getUserCode());

[0140] / / Other processing ......

[0142] / / Check the data of newUserBean

[0143] checkParam(newUserBean);

[0144] / / After verifying the parameters, update the data in the business table

[0145] userMapper.update(newUserBean);

[0146] / / Save the value changes of important attributes and report errors

[0147] addAttrChanged(newUserBean, oldUserBean);

[0148] / / Save complete record information before and after the change

[0149] addRecordChanged(newUserBean, oldUserBean);

[0150] / / Other processing ......

[0152] }

[0153] public void checkParam(UserBean userBean){

[0154] List <string>errorList = new ArrayList<>();

[0155] if(StringUtils.isEmpty(userBean.getUserName())){

[0156] errorList.add("User name cannot be empty");

[0157] }

[0158] / / Other checks ......

[0160] if(!CollectionUtils.isEmpty(errorList)){

[0161] / / Throw the detected problem as an exception

[0162] throw new RuntimeException(listToStr(errorList));

[0163] }

[0164] }

[0165] It is understandable that in this embodiment, after the data verification is passed, it is sometimes necessary to log the changes of important information. For example, when updating a user's information, the user's job title in the original user table is a, and it becomes b when updating. When comparing the value of the attribute, it is found that there is a change. At this time, it is necessary to make a record in the third log table, that is, the important attribute value change log table sync_attr_changed_log. The important attribute information change log table mainly contains the following information: user code (that is, identity information), the name of the important attribute (that is, the target important attribute name), the original value of the important attribute (that is, the initial target important attribute value), the synchronized value of the important attribute (that is, the updated target important attribute value), batch_id (that is, the batch information synchronization identifier), the time of creating the log (that is, the third log generation time), etc. The specific code segment can be shown as follows:

[0166] public void addAttrChanged(UserBean newUserBean, UserBeanoldUserBean){

[0167] / / Compare job titles

[0168] if (!StringUtils.equals(oldUserBean.getTitle(), newUserBean.getTitle())) {

[0169] SyncAttrChangedBean syncAttrChangedBean = new SyncAttrChangedBean();

[0170] syncAttrChangedBean.setId(id);

[0171] syncAttrChangedBean.setUserCode(oldUserBean.getUserCode());

[0172] syncAttrChangedBean.setBatchId(batchId);

[0173] syncAttrChangedBean.setAttrKey("title");

[0174] syncAttrChangedBean.setAttrName("Title");

[0175] syncAttrChangedBean.setAttrValueOld(oldUserBean.getTitle());

[0176] syncAttrChangedBean.setAtrrValueNew(newUserBean.getTitle());

[0177] syncAttrChangedMapper.insertSyncAttrChanged(attrChangedBean);

[0178] }

[0179] / / Compare other attributes ......

[0181] }

[0182] Note: In the translation of line 18, "职称" is translated as "Title" here for the sake of semantic consistency in the context. It may need to be adjusted according to the actual situation.It should be noted that in this embodiment, the key attribute information is recorded in the important attribute value change log table. In order to fully record the data of user information before and after synchronization, a fourth log table needs to be added, namely the user record change log table sync_record_changed_log, to record the complete content before and after synchronization. The main field information of this table includes: batch_id (i.e. batch information synchronization identifier), user code (i.e. identity information), all information recorded by the user before synchronization (i.e. initial information of the information to be synchronized), all information recorded by the user after synchronization (i.e. changed information of the information to be synchronized), etc. The specific code segment can be shown as follows:

[0183] public void addRecordChanged(UserBean newUserBean, UserBeanoldUserBean){

[0184] SyncRecordChangedBean bean = new SyncRecordChangedBean();

[0185] bean.setId(id);

[0186] bean.setUserCode(userBean.getUserCode());

[0187] bean.setOldContent(JSON.toJSONString(oldUserBean));

[0188] bean.setNewContent(JSON.toJSONString(newUserBean));

[0189] bean.setType(type);

[0190] bean.setBatchId(batchId);

[0191] syncRecordChangedMapper.insertSyncRecordChanged(bean);

[0192] }

[0193] It should be emphasized that during the data synchronization process, exception handling must be added to the processing of each piece of data without affecting the processing of other data in the batch of data. When processing problematic data, the exception information can also be assembled and returned to the caller after processing is completed, so that the caller can record the processing results and troubleshoot problems. Finally, the number of successful and failed data synchronization statistics needs to be used to update the relevant information in the batch data request parameter log table sync_data_batch_log. As can be seen from the above process, all log records are completed in user data synchronization.

[0194] Accordingly, see Figure 3 As shown, the embodiment of the present application also provides a log recording device for data synchronization between systems, including:

[0195] A first log recording module 11, used for obtaining a request parameter in a current synchronization request message based on a preset string data type, and logging the request parameter based on a preset relational database to complete a first log recording operation;

[0196] The second log recording module 12 is used to perform data conversion on the request parameters based on a preset statement, and use the preset relational database to log each piece of information to be synchronized in the obtained data conversion result to complete the second log recording operation;

[0197] The data judgment module 13 is used to judge whether each piece of information to be synchronized meets the preset data synchronization condition during the data synchronization process to obtain a corresponding judgment result;

[0198] The third log recording module 14 is used for analyzing the changes of the important attribute information and the information to be synchronized based on the preset relational database, the preset important attribute category and the obtained post-synchronization information and performing log recording if the judgment result shows that the conditions are met, after triggering and completing the data synchronization operation of the corresponding information to be synchronized, so as to complete the third log recording operation;

[0199] The fourth log recording module 15 is used to determine the synchronization failure information of the corresponding information to be synchronized based on the judgment result if the judgment result indicates that it is not satisfied, and use the preset relational database to log the synchronization failure information to complete the fourth log recording operation.

[0200] As can be seen from the above, the present application obtains the request parameters in the synchronization request message and uses a preset relational database for log recording. After the request parameter data is converted, the information to be synchronized in the data conversion result is logged. During the data synchronization process, it is determined whether the information to be synchronized meets the conditions. If it meets the conditions, the important attribute changes are analyzed and logged after the synchronization operation is completed. If it does not meet the conditions, the synchronization failure information is determined and logged. In this way, the present application flexibly and accurately records the complete changes in the data synchronization process between systems, which is convenient for troubleshooting abnormal information in the data synchronization process, causes of synchronization failure, and changes in important attribute information.

[0201] In a specific implementation, the first log recording module 11 may specifically include:

[0202] A parameter acquisition unit, used to acquire request parameters in the current synchronization request message based on a string data type;

[0203] A first log generation unit is used to generate log record information corresponding to the request parameters based on a first log table in a preset relational database to obtain a corresponding first system log; wherein the first system log includes a batch information synchronization identifier as a primary key, the request parameters, the type of synchronization this time, a first log generation time, the number of batch information synchronization items, the number of successful information synchronization items, and the number of failed information synchronization items.

[0204] In a specific implementation, the log recording device for data synchronization between the systems may further include:

[0205] A quantity counting unit, configured to count the number of successful data synchronization operations and the number of failed data synchronization operations after all the data synchronization operations are completed based on the judgment results respectively corresponding to the pieces of information to be synchronized, so as to obtain the corresponding number of successful information synchronizations and the number of failed information synchronizations;

[0206] A quantity determination unit, configured to determine the number of pieces of information to be synchronized in batches based on the number of pieces of information that have been successfully synchronized and the number of pieces of information that have failed to be synchronized;

[0207] A log updating unit is used to update the first system log in the first log table by using the number of successful information synchronization, the number of failed information synchronization and the number of batch information synchronization.

[0208] In a specific implementation, the second log recording module 12 may specifically include:

[0209] A list determination unit, configured to convert each piece of information to be synchronized in the request parameter into a preset object based on a preset statement, and determine a corresponding list to be synchronized according to each piece of converted information obtained;

[0210] A second log generating unit is used to log each of the converted information in the list to be synchronized one by one based on the second log table in the preset relational database, so as to obtain a second system log corresponding to each of the converted information; wherein the second system log includes corresponding identity information, second log generation time, corresponding character string information of the converted information and the batch information synchronization identifier.

[0211] In a specific implementation, the third log recording module 14 may specifically include:

[0212] An information analysis unit, configured to trigger a data synchronization operation of the corresponding information to be synchronized, and after the data synchronization operation is completed, analyze changes in the important attribute information based on the preset relational database, the preset important attribute categories, and the obtained post-synchronization information to obtain a first analysis result;

[0213] A third log generation unit is used to generate log record information corresponding to the first analysis result in a third log table in the preset relational database based on the first analysis result to obtain a corresponding third system log; wherein the third system log includes the identity information, the batch information synchronization identifier, the third log generation time, the target important attribute name, and the corresponding initial target important attribute value and the updated target important attribute value.

[0214] In a specific implementation, the third log recording module 14 may specifically include:

[0215] an information analysis unit, configured to analyze changes in the information to be synchronized based on the preset relational database and the obtained post-synchronization information after the corresponding data synchronization operation is completed, so as to obtain a second analysis result;

[0216] A fourth log generation unit is used to generate log record information corresponding to the second analysis result in a fourth log table in the preset relational database based on the second analysis result to obtain a corresponding fourth system log; wherein the fourth system log includes the corresponding batch information synchronization identifier, the identity identification information, and the initial information and changed information of the information to be synchronized.

[0217] In a specific implementation, the fourth log recording module 15 may specifically include:

[0218] The fifth log generating unit is used to determine the synchronization failure information of the corresponding information to be synchronized based on the judgment result, and use the fifth log table in the preset relational database to generate log record information corresponding to the synchronization failure information to obtain the corresponding fifth system log.

[0219] Furthermore, the present application also discloses an electronic device. Figure 4 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input and output interface 25 and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the log recording method when synchronizing data between systems disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment can specifically be an electronic computer.

[0220] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0221] In addition, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0222] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the log recording method for data synchronization between systems performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0223] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the log recording method for data synchronization between the aforementioned systems is implemented. The specific steps of the method can be referred to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0224] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0225] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0226] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0227] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0228] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.< / string> < / userbeanbean>

Claims

1. A log recording method for data synchronization between systems, characterized in that: include: Obtaining a request parameter in the current synchronization request message based on a preset string data type, and logging the request parameter based on a preset relational database to complete a first logging operation; Performing data conversion on the request parameters based on a preset statement, and using the preset relational database to log each piece of information to be synchronized in the obtained data conversion result, so as to complete a second log recording operation; During the data synchronization process, it is determined whether each piece of information to be synchronized meets the preset data synchronization condition to obtain a corresponding determination result; If the judgment result indicates that the condition is satisfied, after triggering and completing the data synchronization operation of the corresponding information to be synchronized, the important attribute information and the changes of the information to be synchronized are analyzed and logged based on the preset relational database, the preset important attribute categories and the obtained post-synchronization information, so as to complete the third log recording operation; If the judgment result indicates that it is not satisfied, the synchronization failure information of the corresponding information to be synchronized is determined based on the judgment result, and the synchronization failure information is logged using the preset relational database to complete the fourth log recording operation.

2. The log recording method for data synchronization between systems according to claim 1, characterized in that: The obtaining of the request parameters in the current synchronization request message based on the preset string data type, and logging the request parameters based on the preset relational database, includes: Get the request parameters in the current synchronization request message based on the string data type; Generate log record information corresponding to the request parameter based on a first log table in a preset relational database to obtain a corresponding first system log; The first system log includes a batch information synchronization identifier as a primary key, the request parameters, the synchronization type, the first log generation time, the number of batch information synchronization items, the number of successful information synchronization items, and the number of failed information synchronization items.

3. The log recording method for data synchronization between systems according to claim 2, characterized in that: Also includes: After all the data synchronization operations are completed based on the judgment results corresponding to the respective pieces of the information to be synchronized, the number of successful data synchronization operations and the number of failed data synchronization operations are counted to obtain the corresponding number of successful information synchronizations and the number of failed information synchronizations; Determine the number of batch synchronization items of the information based on the number of successful synchronization items and the number of failed synchronization items of the information; The first system log in the first log table is updated using the number of successful information synchronizations, the number of failed information synchronizations, and the number of batch information synchronizations.

4. The log recording method for data synchronization between systems according to claim 2, characterized in that: The performing data conversion on the request parameters based on the preset statement, and using the preset relational database to log each piece of information to be synchronized in the obtained data conversion result, includes: Based on the preset statement, each piece of information to be synchronized in the request parameter is converted into a preset object, and a corresponding list to be synchronized is determined according to each converted information obtained; Based on the second log table in the preset relational database, log each of the converted information in the to-be-synchronized list one by one to obtain a second system log corresponding to each of the converted information; The second system log includes corresponding identity information, second log generation time, corresponding character string information of the converted information, and batch information synchronization identifier.

5. The log recording method for data synchronization between systems according to claim 4, characterized in that: After triggering and completing the data synchronization operation of the corresponding information to be synchronized, analyzing the changes of the important attribute information and the information to be synchronized based on the preset relational database, the preset important attribute categories and the obtained post-synchronization information and recording them in logs, including: triggering a data synchronization operation of the corresponding information to be synchronized, and after the data synchronization operation is completed, analyzing changes in the important attribute information based on the preset relational database, the preset important attribute category and the obtained post-synchronization information to obtain a first analysis result; Based on the first analysis result, log record information corresponding to the first analysis result is generated in a third log table in the preset relational database to obtain a corresponding third system log; The third system log includes the identity information, the batch information synchronization identifier, the third log generation time, the target important attribute name, and the corresponding initial target important attribute value and updated target important attribute value.

6. The log recording method for data synchronization between systems according to claim 4 or 5, characterized in that: After triggering and completing the data synchronization operation of the corresponding information to be synchronized, analyzing the changes of the important attribute information and the information to be synchronized based on the preset relational database, the preset important attribute categories and the obtained post-synchronization information and recording them in logs, including: After the corresponding data synchronization operation is completed, analyzing the change of the information to be synchronized based on the preset relational database and the obtained post-synchronization information to obtain a second analysis result; Based on the second analysis result, generating log record information corresponding to the second analysis result in a fourth log table in the preset relational database to obtain a corresponding fourth system log; The fourth system log includes the corresponding batch information synchronization identifier, the identity information, and the initial information and changed information of the information to be synchronized.

7. The log recording method for data synchronization between systems according to claim 1, characterized in that: The determining the synchronization failure information of the corresponding information to be synchronized based on the judgment result, and using the preset relational database to log the synchronization failure information, includes: The synchronization failure information of the corresponding information to be synchronized is determined based on the judgment result, and the log record information corresponding to the synchronization failure information is generated by using the fifth log table in the preset relational database to obtain the corresponding fifth system log.

8. A log recording device for data synchronization between systems, characterized in that: include: A first log recording module, used for obtaining a request parameter in a current synchronization request message based on a preset string data type, and logging the request parameter based on a preset relational database to complete a first log recording operation; A second log recording module, used to perform data conversion on the request parameters based on a preset statement, and log each piece of information to be synchronized in the obtained data conversion result using the preset relational database, so as to complete a second log recording operation; The data judgment module is used to judge whether each piece of information to be synchronized meets the preset data synchronization condition during the data synchronization process to obtain a corresponding judgment result; A third log recording module is used for analyzing the changes of the important attribute information and the information to be synchronized based on the preset relational database, the preset important attribute category and the obtained post-synchronization information and performing log recording if the judgment result indicates that the conditions are met, after triggering and completing the data synchronization operation of the corresponding information to be synchronized, so as to complete the third log recording operation; The fourth log recording module is used to determine the synchronization failure information of the corresponding information to be synchronized based on the judgment result if the judgment result indicates that it is not satisfied, and use the preset relational database to log the synchronization failure information to complete the fourth log recording operation.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the log recording method for data synchronization between systems as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, a log recording method for data synchronization between systems as described in any one of claims 1 to 7 is implemented.

Images