Data integration method, device and computer-readable storage medium

Abstract

The present application discloses a data integration method, device and computer-readable storage medium, the method comprising: executing a first data integration request sent by a target system to obtain a data integration result; if the data integration result is a failure, counting the number of integration failures of the first business data and recording the cause of the failure; after determining that a first preset condition is satisfied, executing the first data integration request again. By means of the above method, when the data integration request processing fails, that is, when the data integration fails, the request can be automatically reprocessed at a scheduled time or after the cause of the failure is eliminated, or the request can be reprocessed based on user operations, without waiting for technical personnel to check and repair one by one, thereby improving the processing efficiency when data integration errors occur, improving the integration efficiency of the business data of the target system, and thereby improving the data consistency between the target system and the data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data integration method, device, and computer-readable storage medium. Background Art

[0002] As an integrated information management system, ERP (Enterprise Resource Planning) systems need to exchange data with other business systems for integrated management. For example, they can exchange e-commerce data with e-commerce systems. This data exchange, or integration, process often encounters problems due to factors such as excessive data size, incorrect integration order of dependent data, and discrepancies in data meaning between systems.

[0003] During the long-term research and development process, the applicant of this application found that the existing data integration process is prone to errors, resulting in consistency issues in the integrated data, and the error handling process is too complicated, requiring manual investigation with the help of system logs, and technical personnel to participate in the data repair process, which makes error resolution inefficient and costly. Summary of the Invention

[0004] The main technical problem solved by this application is to provide a data integration method, device and computer-readable storage medium, which can improve the data integration consistency during the data integration process.

[0005] To solve the above technical problems, a technical solution adopted in this application is: providing a data integration method, the method comprising: receiving a first data integration request sent by a target system, the first data integration request being used to request integration of first business data; executing the first data integration request to obtain a data integration result; if the data integration result is a failure, counting the number of integration failures of the first business data and recording the cause of the failure; after determining that a first preset condition is met, executing the first data integration request again, wherein the first preset condition includes at least one of an interval of a preset time, a response to a user operation, and elimination of the cause of failure.

[0006] In order to solve the above technical problems, another technical solution adopted in this application is: to provide a data integration device, which includes a processor and a memory, the memory is used to store program data, and the processor is used to execute the program data to implement the above method.

[0007] In order to solve the above technical problems, another technical solution adopted in this application is: providing a computer-readable storage medium, which is used to store program data, and the program data can be executed to implement the above method.

[0008] In the above scheme, the target data is integrated by processing the data integration request sent by the target system; the first data integration request sent by the target system is executed to obtain a data integration result; if the data integration result is a failure, the number of integration failures of the first business data is counted, and the reason for the failure is recorded; after determining that the first preset condition is met, the first data integration request is executed again. Through the above method, when the data integration request processing fails, that is, when the data integration fails, the request can be automatically reprocessed at a fixed time or after the reason for the failure is eliminated, or the request can be reprocessed based on user operations, without waiting for technical personnel to check and repair one by one, thereby improving the processing efficiency when data integration errors occur, improving the integration efficiency of the business data of the target system, and thereby improving the data consistency between the target system. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a flow chart of an embodiment of the data integration method of the present application;

[0010] Figure 2 This is a flowchart of another embodiment of the data integration method of the present application;

[0011] Figure 3 This is a flow chart of another embodiment of the data integration method of the present application;

[0012] Figure 4 This is a flowchart of another embodiment of step S330 of the present application;

[0013] Figure 5 This is a flowchart of another embodiment of step S340 of the present application;

[0014] Figure 6 This is a schematic diagram of the framework of an embodiment of the data integration device of the present application;

[0015] Figure 7 It is a schematic diagram of the framework of an embodiment of the computer-readable storage ring of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and effects of this application clearer and more specific, this application is further described in detail below with reference to the accompanying drawings and examples.

[0017] The term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship. Furthermore, "many" in this document means two or more than two.

[0018] It can be understood that the method of the present application may include the method provided by any one of the following method embodiments and any non-conflicting combination of the following method embodiments.

[0019] It can be understood that the data integration method in the present application can be executed by a data integration device, which can also be simply referred to as a device. The data execution device is any device with execution capabilities, such as a mobile phone, computer, tablet computer, etc.

[0020] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of a data integration method of the present application, which includes:

[0021] Step S110: receiving a first data integration request sent by a target system.

[0022] It should be noted that in the present application, the device can run a system for integrating data, namely a data integration system. As business progresses, business data is generated in the target system. The data integration system needs to obtain the business data in the target system, that is, the target data. Therefore, there is a need for integration of the target data between the data integration system and the target system. The data integration system provides an interface. After the business data is generated, the target system calls the interface to send a data integration request to the data integration system to achieve data integration. The first data integration request is used to request the integration of the first business data. For example, the system for data integration run by the device can be an ERP system, and the target system can be an e-commerce system. Since users using the ERP system conduct transactions in the e-commerce system, the e-commerce system stores the user's e-commerce data, such as the transaction parties, purchase order data, and shipping data. Since users use the ERP system for enterprise information integrated management, there is a need for integration of e-commerce data between the ERP system and the e-commerce system. The first business data is e-commerce data related to the user, such as purchase order information and shipping information. The ERP system provides a business data interface. The e-commerce system can call this business data interface and send a first data integration request to the target system, thereby achieving the integration of the first business data.

[0023] It should be noted that business data can be divided into multiple categories depending on the business stage. Each category of business data corresponds to a business data interface. The target system will call the corresponding business data interface based on the data type of the first business data. The target system will continuously generate business data according to the business progress. When a piece of business data is generated, the target system will call the interface and send a data integration request. Therefore, the target system can call the interface multiple times and send multiple data integration requests. The first data integration request in this application is a data integration request sent by the target system.

[0024] Step S120: Execute the first data integration request to obtain a data integration result.

[0025] The data integration result includes integration success or integration failure. It is understandable that in some cases, an error may occur during the execution of the first data integration request, resulting in data integration failure. If the data integration result is failure, step S130 is executed.

[0026] Step S130: Count the number of execution failures of the first data integration request and record the reasons for the failures.

[0027] Specifically, the device may store the failure reason of the first data integration request in the form of an error log, thereby achieving structured storage of problems encountered during the execution of the first data integration request, thereby enabling subsequent processing based on the structured stored problems, such as re-executing the first data integration request. Specifically, the device may store the failure reason of the data integration request in a structured manner, and then re-execute the data integration request after determining that the failure reason has been eliminated. In addition, the device may cache the failed first data integration request for subsequent re-execution of the first data integration request after the first preset condition is met.

[0028] Step S140: Determine whether a first preset condition is met.

[0029] If so, execute step S120 again, that is, when the data integration result is a failure, execute the first data integration request again after the first preset condition is met, wherein the first preset condition includes at least one of an interval preset time, a response to a user operation, and elimination of the cause of failure.

[0030] Among them, the preset time can be a time determined by the user according to needs, and the user can adjust the preset time. Specifically, the device can start timing from the most recent execution failure of the first data integration request, and execute step S120 again after the current time meets the first preset condition of the interval preset time.

[0031] Specifically, taking the first preset condition of responding to user operations as an example, the device can provide the user with a data integration status tracking page, and display the first data integration request that failed and all corresponding historical failure reasons on the page for the user to view, and in response to the user's operation on the page, execute the data integration request selected by the user on the page again.

[0032] In the above scheme, the target data is integrated by processing the data integration request sent by the target system; the first data integration request sent by the target system is executed to obtain a data integration result; if the data integration result is a failure, the number of integration failures of the first business data is counted, and the reason for the failure is recorded; after determining that the first preset condition is met, the first data integration request is executed again. Through the above method, when the data integration request processing fails, that is, when the data integration fails, the request can be automatically reprocessed at a fixed time or after the reason for the failure is eliminated, or the request can be reprocessed based on user operations, without waiting for technical personnel to check and repair one by one, thereby improving the processing efficiency when data integration errors occur, improving the integration efficiency of the business data of the target system, and thereby improving the data consistency between the target system.

[0033] See also Figure 2 , Figure 2 : is a flow chart of another embodiment of the data integration method of the present application, the method further comprising:

[0034] Step S210: Receive a second data integration request sent by the target system.

[0035] The second data integration request is used to request the integration of basic data. It should be noted that business data is generated based on basic data. Business data includes basic data, which is the upstream data that business data depends on. Continuing with the e-commerce system as the target system, basic data can include business entities, product information, or operators in e-commerce transactions. Business data can be order information including business entities, product information, and operators. Order information is generated based on business entities, product information, and operators. Furthermore, the same basic data can appear repeatedly in different business data. For example, the same product information can appear in different order information.

[0036] There is a need to integrate basic data between the data integration system and the target system. Similar to the integration of business data, the data integration system provides an interface for the target system to call and send a second data integration request.

[0037] Specifically, in addition to the several business data interfaces mentioned above, the data integration system also provides a basic data integration interface. After the basic data is generated, the target system calls the basic data integration interface to send a second data integration request to request the integration of the basic data. Similarly, different interfaces are provided for different types of basic data.

[0038] In some embodiments, before executing step S220, a basic verification of the basic data is further performed. If the basic verification fails, a notification of the failure of the basic data verification is sent to the target system, and steps S220 to S240 are no longer executed. Steps S220 to S240 are executed only if the basic verification passes.

[0039] Specifically, the basic check includes at least one of the data uniqueness check and the interface field check, wherein the data uniqueness check is used to determine the uniqueness of the basic data in the data sent by the target system to avoid the target system from repeatedly pushing data. The interface field check is used to check whether the fields contained in the basic data comply with the interface agreement. The interface field check can be specifically divided into data type check, required check and character length check. The data type check is used to determine whether the data type of the basic data is consistent with the data type corresponding to the interface called by the target system. The required check is used to determine whether the required fields contained in the basic data are required. The character length check is used to determine whether the character length of the fields contained in the basic data meets the interface agreement. If, during the basic check of the basic data, any one of the data uniqueness check and the interface field check fails, it can be determined that the basic check has failed.

[0040] If the target system repeatedly sends the same basic data, the duplicate data can be eliminated through the basic verification step. Even if the same basic data is processed and a new mapping relationship that is the same as the existing data mapping relationship is obtained, the device can eliminate the duplicate content in the data mapping table to prevent duplicate content.

[0041] Step S220: In response to the second data integration request, the fixed fields of the basic data are used to determine whether the first preset data includes the basic data.

[0042] It should be noted that for the same type of basic data, the target system and the data integration system may have different requirements, such as different naming conventions. Even if the substantive content of the two data is identical, the field contents stored in the target system and the data integration system may differ significantly. For example, the product information in the e-commerce system is named A 10PLUS, while the material name in the data integration system is a10plus. Both represent basic data representing the product and are essentially the same product, but the specific content of their names differs. Therefore, after receiving the second data integration request from the target system, the device can establish a data mapping table in the data integration system. The data mapping table includes the data mapping relationship between the basic system and the data integration system sent by the target system, indicating that the data content of both parties is substantially identical. Please refer to Table 1, which provides an example of a basic data mapping relationship in an application scenario. For the data integration system, the product information sent by the e-commerce system is the material basic data, and the e-commerce system calls the material basic data interface to integrate the material basic data. The third party in Table 1 refers to the target system, which in this example is the e-commerce system, and the system itself is the data integration system.

[0043]

[0044] Table 1 Basic data mapping relationship example

[0045] It should be noted that the data integration system may be pre-stored with first preset data, and the first preset data includes a number of data that may appear in the target system as basic data. For example, the first preset data includes 300 units, which may appear in the e-commerce system as trading units. The device can respond to the second data integration request, determine the basic data sent by the target system, and compare the basic data sent by the target system with the first preset data stored in the data integration system. If the first preset data includes the basic data sent by the target system, then execute step S230, use the basic data and the first preset data corresponding to the basic data to create a new first data mapping relationship, and add the first data mapping relationship to the data mapping table. If the first preset data does not include basic data, then execute step S240, generate second preset data based on the basic data, use the basic data and the second preset data to create a new second data mapping relationship, and add the second data mapping relationship to the data mapping table.

[0046] It is understood that in some embodiments, some basic data includes multiple fields, at least one of which is a fixed field. Fixed fields are fixed contents included in the basic data and generally do not change. The contents of fixed fields vary for different data. For example, basic data for a business entity includes fields such as the business entity name, unified social credit code, and business entity address. The unified social credit code is a fixed field for the business entity and does not change. Different business entities correspond to different unified social credit codes.

[0047] Basic data can be divided into multiple types. Different types of basic data include different fixed fields. For example, if the basic data is a trading unit, then the fixed field can be a unified social credit code; if the basic data is product information (material), then the fixed field can be a SKU code; if the basic data is an operator, then the fixed field can be a telephone number.

[0048] Specifically, the device can use the fixed fields of the basic data to determine whether the basic data is included in the first preset data. For example, the basic data is a trading unit, and its fixed field is a unified social credit code. Then the device can determine whether the unified social credit code of the trading unit sent by the target system is consistent with the unified social credit code of the first preset data. If there is a consistency, it means that the basic data is included in the first preset data. If there is no consistency, it means that the basic data is not included in the first preset data.

[0049] Step S230: creating a first data mapping relationship using the basic data and the corresponding first preset data, and adding the first data mapping relationship to the data mapping table.

[0050] Step S240: Generate second preset data based on the basic data, create a second data mapping relationship using the basic data and the second preset data, and add the second data mapping relationship to the data mapping table.

[0051] It should be noted that since the target system includes multiple subsystems, business data and basic data may come from different subsystems, and different types of business data or different types of basic data may also come from different subsystems. Different subsystems cannot know each other when sending data integration requests. Therefore, the steps of the target system sending the first data integration request and the second data integration request are independent of each other, and there is no fixed order of sequence. Therefore, the execution order of steps S210-step S240 and steps S110-step S140 can be swapped.

[0052] It can be understood that after completing the above steps, after receiving the business data containing the basic data, the preset data corresponding to the basic data can be determined based on the mapping relationship of the basic data in the data mapping table. The preset data can also be understood as the result of converting the basic data in the business data to meet the requirements of the data integration system, so that the data integration system can process the basic data contained in the business data.

[0053] It should be noted that if the data integration system first receives the second data integration request and executes steps S210-S240, and then receives the first data integration request, where the first business data includes basic data and executes steps S110-S140, then the integration will not fail during steps S110-S140 due to the lack of basic data integration. Conversely, the integration will fail during steps S110-S140 due to the lack of basic data integration.

[0054] See also Figure 3 , Figure 3 : is a flow chart of another embodiment of the data integration method of the present application, the method comprising:

[0055] Step S310: Receive a first data integration request sent by a target system.

[0056] For the description of step S310 , please refer to the aforementioned description of step S110 .

[0057] In some embodiments, after receiving the data to be integrated sent by the target system, the data to be integrated is stored in a data backup table. In the event of data inconsistency, the data can be traced back according to the data backup table.

[0058] It should be noted that the data integration system and the target system will agree on data interface messages and requirements for interface fields, such as the attributes and length of the data fields in the interface. Please refer to Table 2, which provides an example of some fields in the procurement and warehousing integration interface in a specific application scenario, including the requirements for these fields.

[0059]

[0060] Table 2 Examples of some fields in the purchase warehousing integration interface

[0061] In addition, the data integration system and the target system may agree on the correspondence between the interface fields. Please refer to Table 3, which provides an example of the field correspondence in a specific application scenario.

[0062]

[0063] Table 3 Field correspondence example

[0064] It should be noted that, based on this field correspondence, the data integration system can match the fields of the data sent by the target system with the fields of the data in this system. For example, it can be determined that the content corresponding to the field SKU code is the content corresponding to the material code.

[0065] In some embodiments, due to the large number of target system services, after receiving a data integration request from the target system, the device may also intercept the request through a current limiting device and limit the data integration request to ensure that only one request is processed at the same time.

[0066] Step S320: Perform a basic check on the first business data. If the basic check fails, a notification indicating that the basic check of the first business data fails is sent to the target system.

[0067] Furthermore, if the basic verification of the first business data fails, the subsequent steps will not be executed.

[0068] Among them, the basic check includes at least one of the data uniqueness check and the interface field check. The data uniqueness check is used to determine the uniqueness of the first business data in the data sent by the target system to avoid the target system from pushing data repeatedly. The interface field check is used to check whether the fields contained in the first business data comply with the interface agreement. The interface field check can be specifically divided into data type check, required check and character length check. The data type check is used to determine whether the data type of the first business data is consistent with the data type corresponding to the interface called by the target system. The required check is used to determine whether the required fields contained in the first business data are required. The character length check is used to determine whether the character length of the fields contained in the first business data meets the interface agreement. Among them, the requirements for interface field check can refer to the interface field requirements in Table 2.

[0069] If, during the basic verification of the first business data, any one of the data uniqueness verification and the interface field verification fails, it can be determined that the basic verification fails, and it is determined that the first business data does not meet the second preset condition, and a notification that the basic verification of the first business data fails is sent to the target system.

[0070] It should be noted that if the first business data fails the basic verification, the data integration system can determine that the first business data has been integrated and does not need to be integrated again, or the first business data does not comply with the interface agreement, then the first business data cannot be processed, and thus the first business data cannot be integrated.

[0071] Step S330: Determine whether the first service data meets the second preset condition and obtain a determination result.

[0072] Based on the judgment result, it is determined whether to execute step S340 to integrate the first business data. If the judgment result is that the first business data does not meet the second preset condition, the integration of the first business data is not performed. If the judgment result is that the first business data meets the second preset condition, the integration of the first business data is performed.

[0073] Please refer to Figure 3 and Figure 4 , Figure 4 3 is a flow chart of another embodiment of step S330 of the present application. Step S330 includes at least one of the following steps:

[0074] Step S431: Determine whether the data mapping table contains the mapping relationship of the basic data. If not, determine that the first business data does not meet the second preset condition, and use the failure reason of the non-integration of the basic data. If so, use the data mapping table to convert the first business data.

[0075] It should be noted that business data is generated based on basic data, and business data includes basic data. After determining the first business data, it is necessary to determine whether the basic data contained therein is mapped. By judging whether the data mapping table contains the mapping relationship of the basic data, it can be determined whether the basic data is mapped, and thus it can be determined that the basic data corresponds to the preset data stored in the data integration system. In the subsequent processing process, the preset data can be used to replace the basic data, so that the data integration system can process the data.

[0076] It should be noted that the data mapping table includes the mapping relationship of the basic data. Using the data mapping table to convert the first business data is to search the preset data corresponding to the basic data included in the first business data according to the data mapping table, and use the content of the preset data to replace the content of the basic data in the first business data, so that in the subsequent processing process, the preset data that meets the requirements of the data integration system can be directly used. The step of converting the first business data using the data mapping table is different from the aforementioned field correspondence step. The field correspondence step does not change the specific content of the field, but only corresponds the field of the data sent by the target system to the field in the data integration system. Even if the received data is field-correlated, it cannot be determined whether the data is the same as the preset data, nor can the data be converted into a form that can be processed by the data integration system.

[0077] Since the basic data in the first business data is not integrated, the data mapping table does not contain the mapping relationship of the basic data, so the basic data cannot be mapped, the first business data cannot be converted into data that meets the requirements of the data integration system, and the data integration system cannot process it. Then it can be determined that the first business data does not meet the second preset condition. Even if the first business data is integrated, it will fail because the basic data cannot be mapped. Therefore, it can be directly determined that the integration of the first business data has failed, the data integration result is failure, and the failure of the basic data to be integrated is used as the reason for the failure.

[0078] In some embodiments, the data mapping table also includes a solidified mapping relationship. Converting the first business data using the data mapping table also includes converting the content of the data using the solidified mapping relationship. Please refer to Table 4, which gives an example of a solidified mapping relationship in a specific application scenario.

[0079]

[0080] Table 4 Example of solidified mapping relationship

[0081] For example, converting the first business data using a fixed mapping relationship includes converting an enumeration with an ID of 1 and a name of Common in the purchase warehousing information sent by the target system into an enumeration with an ID of 10000035 and a name of 00000001 in the data integration system.

[0082] In some cases, due to network issues or other factors, the fixed mapping relationship may be missing, resulting in a failure in the step of converting the first business data using the fixed mapping relationship, which in turn may cause the first data integration request to fail. In this case, the missing fixed mapping relationship may be determined as the cause of the failure. The device may respond to a user's repair of the fixed mapping relationship by updating and publishing the fixed mapping relationship, determine that the cause of the failure has been eliminated, and then re-execute the first data integration request. Alternatively, the device may respond to a user operation or re-execute the first data integration request at a preset interval.

[0083] Step S432: Determine whether the second business data is integrated to perform business verification on the first business data. If not, determine that the first business data does not meet the second preset condition and use the failure reason as the failure reason. If yes, determine that the first business data passes the business verification.

[0084] Among them, the second business data is the upstream business data that the first business data depends on, and the first business data is generated based on the second business data. For example, the first business data is shipment data, and the second business data is order data. Since there is a fixed order in the business stages in the target system, there is also a dependency relationship between the business data generated in different business stages, and the shipment data is generated based on the order data.

[0085] Therefore, in the integration process, since the integration processes of different business data are independent of each other, there may be a situation where the target system first requests the integration of the first business data and then requests the integration of the second business data. At this time, if the second business data is not integrated, the data integration system cannot integrate the first business data. Therefore, if it is determined that the second business data is not integrated, it is determined that the first business data does not meet the second preset condition. Even if the integration of the first business data is performed, it will fail due to the failure of the second business data. Therefore, it can be directly determined that the integration of the first business data has failed, the data integration result is failure, and the failure of the second business data to be integrated is used as the reason for the failure.

[0086] It should be noted that if step S330 includes both step S431 and step S432, then the execution order of step S431 and step S432 can be swapped, and no limitation is imposed here.

[0087] If step S330 includes only one of step S431 and step S432, then whether the first business data satisfies the second preset condition is determined based on the judgment result of the step included in step S330. If the judgment result is yes, then the first business data is determined to satisfy the second preset condition. If step S330 includes both step S431 and step S432, if the judgment results in both step S431 and step S432 are yes, then it can be determined that the first business data satisfies the second preset condition. If either step is no, then it is determined that the first business data does not satisfy the second preset condition. After determining that the first business data satisfies the second preset condition, step S340 is executed to integrate the first business data. If the second preset condition is not satisfied, even if step S340 is executed, the execution of the first business data will fail, thereby directly determining that the execution of the first data integration request has failed and recording the reason for the failure.

[0088] Step S340: Integrate the first business data to obtain a data integration result of the first business data.

[0089] Step S120 can be implemented through steps S330 to S340. If the data integration result of the first business data obtained in step S340 is failure, step S350 is executed.

[0090] The integration method of the first business data includes filling a target document with the first business data. If the target document is filled, it indicates that the first business data integration is successful. If the target document cannot be filled, it indicates that the first business data integration fails.

[0091] Please refer to Figure 3 and Figure 5 , Figure 52 is a flow chart of another embodiment of step S340 of the present application, wherein step S340 includes:

[0092] Step S541: Save the first data integration request to a to-be-executed list, and send a successful receipt notification to the target system.

[0093] It should be noted that data will be generated in the target system. After each data is generated, a data integration request will be generated accordingly, and the interface will be called to send the data integration request. Since the amount of data generated by the target system is very large, but the data integration system running on the device has a relatively weak ability to execute data integration requests, in this case, the data integration system takes a long time to execute the data integration request, so that it cannot promptly feedback to the target system the execution status of the data integration request it sent (that is, the data integration system cannot promptly feedback to the target system whether its call to the interface is successful). The target system does not receive feedback within a certain period of time and cannot determine whether the data integration is completed, resulting in the data integration system completing the execution of the data integration request, but because it exceeds the feedback time, the target system cannot confirm the execution status of the data integration request. Inconsistent judgments on the integration status of the same data between the two systems will cause errors.

[0094] In order to avoid the occurrence of the above situation, the data integration method in this application adopts an asynchronous message processing mechanism. After receiving the first data integration request sent by the target system, the data integration system will not execute the first data integration request immediately, but will put it into the to-be-executed list, poll the requests in the to-be-processed list one by one, and then provide feedback to the target system after the first data integration request is executed.

[0095] After the first data integration request is saved in the to-be-executed list, a successful reception notification is sent to the target system to notify the target system that the first data integration request sent by calling the business data interface has been successfully received.

[0096] Step S542: Integrate the first business data according to the execution order in the to-be-executed list.

[0097] It can be understood that there are several requests stored in the to-be-executed list, and the above-mentioned several requests can be several data integration requests, which are arranged in a certain execution order, including the first data inheritance request. The thread pool polls the to-be-executed list and executes the requests therein in the execution order.

[0098] It should be noted that the integration method for the first business data involves using the first business data to populate target documents. Target documents can include inbound documents, outbound documents, invoices, and other documents. For example, if the first business data is sales order shipment information, integrating this sales order shipment information can include using this information to populate a shipment outbound document. In this case, the shipment outbound document becomes the target document. After the target document is populated, it can be used for subsequent enterprise information data management.

[0099] In some embodiments, the first business data only partially populates the target document. The data integration system can determine the remaining content of the target document based on the first business data and automatically complete the filling of the target document. Specifically, the data integration system can fill the target document using methods such as default copying or data retrieval based on the attributes of the managed object. In a specific application scenario, the first business data is sales order information, including the salesperson, the product sold, the sales amount, etc. The target document also includes the sales department, which is not included in the first business data. The data integration system can determine the sales department for the order based on the pre-stored correspondence between salespersons and departments in the system and fill the sales department into the corresponding position in the target document.

[0100] If the target document is successfully populated, the integration of the first business data is successful. If the target document cannot be populated, the integration of the first business data has failed. This failure includes situations such as incomplete populating the target document or errors during the populating process. Therefore, if the basic data is not integrated, the second business data is not integrated, or the fixed mapping relationship is missing, even if the first business data is integrated, the first business data cannot be processed and the target document cannot be populated. Therefore, if these situations occur, it can be directly determined that the first business data integration has failed.

[0101] Step S543: remove the executed data integration request from the to-be-executed list.

[0102] Regardless of whether the target data integration succeeds or fails, after the data integration request is executed, it will be removed from the pending execution list.

[0103] Step S350: Count the number of execution failures of the first data integration request and record the reasons for the failures.

[0104] It is understandable that the number of execution failures of the above-mentioned first data integration request includes the number of execution failures of step S330 and step S340, and the corresponding failure reasons are recorded.

[0105] In a specific application scenario, please refer to Table 5, which provides an example of storing failure reasons.

[0106]

[0107] Table 5 Example of failure reason storage

[0108] Step S360: Determine whether the first preset condition is met.

[0109] The first preset condition includes at least one of a preset time interval, a response to a user operation, and elimination of a failure cause. If the first preset condition is met, the process returns to re-execute steps S330 and S340.

[0110] It should be noted that the re-execution of step S330 and step S340 can be based on the failure reason and re-executed from the point where the execution failed, without the need to re-execute the steps that were successfully executed to avoid wasting resources.

[0111] In some embodiments, the first preset condition also includes that the number of execution failures of the first data integration request is less than a preset upper limit, thereby preventing the device from repeatedly executing the first data integration request too many times and avoiding wasting resources.

[0112] Step S370: Send a processing success notification to the target system.

[0113] If the data integration result is successful, the processing result of the first data integration request needs to be fed back to the target system. Specifically, after determining that the data integration result is successful, the device may send a processing success notification to the target system.

[0114] In some embodiments, the data integration request may be processed successfully but the sending of the processing success notification may fail. The device may regard this situation as one of the situations in which the data integration request fails to execute, and display the execution failure situation on the integration tracking query page. In response to the user triggering a resend operation on the page, the processing success notification may be resent, or the processing success notification may be resent after a preset time interval or when the failure reason disappears.

[0115] In the above scheme, the target data is integrated by processing the data integration request sent by the target system; if the data integration result is a failure, the number of integration failures of the first business data is counted, and the reason for the failure is recorded; after determining that the first preset condition is met, the first data integration request is executed again. Through the above method, when the data integration request processing fails, that is, when the data integration fails, the request can be automatically reprocessed at a scheduled time or after the reason for the failure is eliminated, or the request can be reprocessed based on user operations, without waiting for technical personnel to check and repair one by one, thereby improving the processing efficiency when data integration errors occur, improving the integration efficiency of the business data of the target system, and thus improving the data consistency between the target system. At the same time, data integration adopts an asynchronous message processing method, and does not process the data integration request immediately after receiving it, and feedback is provided after the processing is completed.

[0116] See also Figure 6 , Figure 6 It is a schematic diagram of the framework of an embodiment of the data integration device of the present application.

[0117] In this embodiment, data integration device 60 includes a memory 61 and a processor 62, wherein memory 61 is coupled to processor 62. Specifically, the various components of data integration device 60 may be coupled together via a bus, or the processor 62 of data integration device 60 may be connected one-to-one with the other components. Data integration device 60 may be any device with processing capabilities, such as a computer, tablet computer, or mobile phone.

[0118] The memory 61 is used to store program data executed by the processor 62 and data processed by the processor 62, such as basic data, first preset data, first business data, etc. The memory 61 includes a non-volatile storage part for storing the above program data.

[0119] Processor 62 controls the operation of data integration device 60. Processor 62 may also be referred to as a CPU (Central Processing Unit). Processor 62 may be an integrated circuit chip with signal processing capabilities. Processor 62 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. A general-purpose processor may be a microprocessor or any conventional processor. In addition, processor 62 may be implemented by multiple integrated circuit chips.

[0120] The processor 62 is configured to execute instructions to implement any of the above-mentioned data integration methods by calling program data stored in the memory 61 .

[0121] In the above scheme, the target data is integrated by processing the data integration request sent by the target system; the first data integration request sent by the target system is executed to obtain a data integration result; if the data integration result is a failure, the number of integration failures of the first business data is counted, and the reason for the failure is recorded; after determining that the first preset condition is met, the first data integration request is executed again. Through the above method, when the data integration request processing fails, that is, when the data integration fails, the request can be automatically reprocessed at a fixed time or after the reason for the failure is eliminated, or the request can be reprocessed based on user operations, without waiting for technical personnel to check and repair one by one, thereby improving the processing efficiency when data integration errors occur, improving the integration efficiency of the business data of the target system, and thereby improving the data consistency between the target system.

[0122] See also Figure 7 , Figure 7 It is a schematic diagram of the framework of an embodiment of the computer-readable storage ring of the present application.

[0123] In this embodiment, the computer-readable storage medium 70 stores program data 71 executable by a processor, and the program data can be executed to implement any of the above-mentioned data integration methods.

[0124] The computer-readable storage medium 70 can specifically be a medium that can store program data, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or it can also be a server that stores the program data. The server can send the stored program data to other devices for execution, or it can also execute the stored program data itself.

[0125] In some embodiments, the computer readable storage medium 70 may also be Figure 6 Memory shown.

[0126] In the above scheme, the target data is integrated by processing the data integration request sent by the target system; the first data integration request sent by the target system is executed to obtain a data integration result; if the data integration result is a failure, the number of integration failures of the first business data is counted, and the reason for the failure is recorded; after determining that the first preset condition is met, the first data integration request is executed again. Through the above method, when the data integration request processing fails, that is, when the data integration fails, the request can be automatically reprocessed at a fixed time or after the reason for the failure is eliminated, or the request can be reprocessed based on user operations, without waiting for technical personnel to check and repair one by one, thereby improving the processing efficiency when data integration errors occur, improving the integration efficiency of the business data of the target system, and thereby improving the data consistency between the target system.

[0127] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A data integration method, characterized in that: The method comprises: receiving a first data integration request sent by a target system, and receiving a second data integration request sent by the target system, wherein the first data integration request is used to request integration of first business data, and the second data integration request is used to request integration of basic data; In response to the second data integration request, determining, using a fixed field of the basic data, whether the first preset data includes the basic data; if so, creating a first data mapping relationship using the basic data and the corresponding first preset data, and adding the first data mapping relationship to a data mapping table; if not, generating second preset data based on the basic data, creating a second data mapping relationship using the basic data and the second preset data, and adding the second data mapping relationship to the data mapping table; In response to the first data integration request, determining whether the first business data satisfies a second preset condition and obtaining a determination result; the second preset condition includes that the data mapping table contains a mapping relationship of basic data, and that the second business data is successfully integrated, and the first business data is generated based on the second business data; Based on the judgment result, determining whether to integrate the first business data to obtain a data integration result of the first business data, wherein the data integration result includes whether the integration of the first business data is successful or failed; If the data integration result is failure, count the number of integration failures of the first business data and record the reasons for the failure; After determining that a first preset condition is satisfied, the first data integration request is executed again, wherein the first preset condition includes at least one of a preset time interval, a response to a user operation, and elimination of the failure cause.

2. The method according to claim 1, characterized in that The integrating of the first business data includes: Saving the first data integration request to a to-be-executed list and sending a receipt success notification to the target system; Integrate the first business data according to the execution order in the to-be-executed list; removing the first data integration request that has been executed from the to-be-executed list; And / or, the integration method of the first business data includes using the first business data to fill the target document. If the target document is filled, it means that the first business data integration is successful; if the target document cannot be filled, it means that the first business data integration has failed.

3. The method according to claim 1, characterized in that The determining whether the first service data satisfies a second preset condition comprises at least one of the following steps: determining whether the data mapping table contains a mapping relationship of the basic data; if not, determining that the first business data does not meet the second preset condition, and taking the non-integration of the basic data as the failure reason; and if so, converting the first business data using the data mapping table; Determining whether the second business data is integrated to perform business verification on the first business data; if not, determining that the first business data does not meet the second preset condition, and setting the non-integration of the second business data as the failure reason; Determining whether to integrate the first business data based on the judgment result includes: If the first business data does not meet the second preset condition, the integration of the first business data is not performed; otherwise, the integration of the first business data is performed.

4. The method according to claim 1, wherein The method further comprises: If the data integration result is successful, a processing success notification is sent to the target system.

5. The method according to claim 1, wherein The executing the first data integration request and obtaining the data integration result includes: A basic verification is performed on the first business data. If the basic verification fails, a notification indicating that the basic verification of the first business data fails is sent to the target system, and the first data integration request is not executed.

6. The method according to claim 1, characterized in that Before determining whether the first preset data includes the basic data by using the fixed fields of the basic data, the method further includes: A basic check is performed on the basic data. If the basic check passes, the fixed field of the basic data is used to determine whether the basic data is included in the first preset data. If the basic check fails, a notification indicating that the basic check of the basic data fails is sent to the target system.

7. A data integration device, characterized in that: The data integration device includes a processor and a memory, wherein the memory is used to store program data, and the processor is used to execute the program data to implement the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program data, and the program data can be executed to implement the method according to any one of claims 1 to 6.

Images