Heterogeneous system data integration method and device

By presetting data specifications in cloud middleware and using producer and consumer processes for data integration, the repetitive work and stability problems during ERP and third-party systems are solved, and efficient and low-cost data integration and unification are achieved. operation and maintenance monitoring.

CN114996034BActive Publication Date: 2024-11-29EFUTURE (BEIJING) ROYALSTONE INFORMATION TECH INC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210555491.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-19
Publication Date
2024-11-29
Estimated Expiration
2042-05-19

AI Technical Summary

Technical Problem

When integrating ERP with third-party systems, the existing technology has problems such as many integrated links, many repetitive tasks, long construction periods, high costs, and low customer satisfaction. It is impossible to operate and maintain monitoring in a unified manner, with poor stability and high maintenance costs.

Method used

By presetting data specifications that are adapted to the first system and the second system data in cloud middleware, the producer process and the consumer process respectively read configuration files, obtain data, generate messages and put them into the message queue, parse and Convert data to realize data integration in heterogeneous systems.

Benefits of technology

Reduces duplicate work, shortens development construction period, reduces costs, improves customer satisfaction, and reduces maintenance costs and improves system stability through unified message queue scheduling.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114996034B_ABST
    Figure CN114996034B_ABST
Patent Text Reader

Abstract

The present invention provides a heterogeneous system data integration method and device, which relate to the field of computer technology. The method is applied to cloud middleware, in which a first data specification adapted to data in a first system and a second data specification adapted to data in a second system are preset, and the first system and the second system are heterogeneous systems. The method is based on the cloud middleware. Compared with establishing integration links between each other, each internal and external system is integrated with the cloud middleware only once based on standard specifications, which reduces duplication of work, shortens development period, reduces costs, and improves customer satisfaction. Moreover, through the cloud middleware, data distribution links and distribution frequencies can be configured, distribution status can be monitored, and maintenance costs are reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a heterogeneous system data integration method and device. Background Art

[0002] At present, some leading supply chain front-end ERP (Enterprise Resource Planning) manufacturers have launched a variety of ERP product forms for different business formats, such as ROC system and MyShop system for supermarket industry, BF system and MSS system for department store industry, etc. After purchasing the corresponding ERP products, users basically expect to be able to integrate with the existing enterprise system or third-party system, such as third-party micro mall, third-party invoice platform, third-party takeaway platform and third-party payment platform.

[0003] According to the traditional project management and delivery model, this type of ERP connection with a third-party system belongs to the category of project customization. Each project will be designed and developed by the delivery personnel based on the customer's demands and their understanding of the business, which will bring the following problems:

[0004] From the perspective of a single project, the following issues exist:

[0005] 1. There are many integrated links, much repetitive work, long construction period, high cost and low customer satisfaction;

[0006] 2. Unable to unify operation and maintenance monitoring, poor stability and high maintenance costs. Summary of the invention

[0007] The present invention provides a heterogeneous system data integration method and device, which are used to solve the above-mentioned problems generated when integrating ERP and a third-party system in the prior art.

[0008] In a first aspect, the present invention provides a heterogeneous system data integration method for cloud middleware, wherein a first data specification adapted to data in a first system and a second data specification adapted to data in a second system are preset in the cloud middleware, and the first system and the second system are heterogeneous systems. The method comprises:

[0009] The producer process reads the producer configuration file and obtains the first business type of the data integration task;

[0010] Acquire configuration information that conforms to the first service type from a user configuration file, the configuration information comprising: information of the first system and information of the second system;

[0011] Acquire first data that conforms to a first business type from the cloud middleware, where the first data is acquired in advance by the cloud middleware from a first system according to the first data specification;

[0012] Generate a message according to the first data, the information of the first system and the information of the second system, and put the message into a message queue;

[0013] The consumer process reads the consumer configuration file to obtain the second business type of the data integration task and information of the first system;

[0014] Reading a message from the message queue, parsing the message according to the second service type and the information of the first system to obtain first data conforming to the second service type and information of the second system corresponding to the information of the first system in the message;

[0015] Converting the first data parsed from the message into second data according to the second data specification;

[0016] The second data is sent to a second system parsed from the message.

[0017] According to the heterogeneous system data integration method provided by the present invention, obtaining first data conforming to the first business type from the cloud middleware includes:

[0018] Acquire the last synchronization time of the first system and the second system from the user synchronization information configuration table of the user profile;

[0019] The data in the cloud middleware that conforms to the first business type and has changed after the last synchronization time is determined as the first data, and the first data is acquired.

[0020] According to the heterogeneous system data integration method provided by the present invention, after putting the message into the message queue, the method further includes:

[0021] Determine a new synchronization time according to the current time or the synchronization step configured in the producer configuration file;

[0022] The last synchronization time between the first system and the second system in the user synchronization information configuration table is updated with the new synchronization time.

[0023] According to the heterogeneous system data integration method provided by the present invention, converting the first data parsed from the message into the second data according to the second data specification includes:

[0024] If the second system is a first type system, converting the first data parsed from the message into second data according to the second data specification;

[0025] If the second system is a second type system, the first data specification and the second data specification are the same, and the first data parsed from the message is designated as the second data.

[0026] According to the heterogeneous system data integration method provided by the present invention, the step of converting the first data parsed from the message into the second data according to the second data specification includes, before the conversion:

[0027] According to the keyword of the first data parsed in the message, query the updated first data corresponding to the keyword in the cloud middleware;

[0028] During conversion, the updated first data is converted into the second data according to the second data specification.

[0029] According to the heterogeneous system data integration method provided by the present invention, before the producer process reads the producer configuration file and obtains the first business type of the data integration task, it also includes:

[0030] The first data uploaded by the first system by calling an interface function conforming to the first data specification is received and stored in the cloud middleware.

[0031] According to the heterogeneous system data integration method provided by the present invention, before the producer process reads the producer configuration file and obtains the first business type of the data integration task, it also includes:

[0032] Reading the preliminary data in the data table set according to the first data specification in the first system;

[0033] The first data uploaded by calling an interface function conforming to the first data specification is taken as a parameter of the prepared data and stored in the cloud middleware.

[0034] In a second aspect, the present invention further provides a heterogeneous system data integration device for cloud middleware, wherein a first data specification adapted to data in a first system and a second data specification adapted to data in a second system are preset in the cloud middleware, and the first system and the second system are heterogeneous systems, and the device comprises:

[0035] The producer configuration reading module is used for the producer process to read the producer configuration file and obtain the first business type of the data integration task;

[0036] A user configuration acquisition module, configured to acquire configuration information that conforms to the first service type from a user configuration file, the configuration information including: information of the first system and information of the second system;

[0037] A first data acquisition module, configured to acquire first data conforming to a first business type from the cloud middleware, wherein the first data is acquired in advance by the cloud middleware from the first system according to the first data specification;

[0038] A message generating module, used for generating a message according to the first data, the information of the first system and the information of the second system, and putting the message into a message queue;

[0039] A consumer configuration reading module, used for a consumer process to read a consumer configuration file and obtain the second business type of the data integration task and information of the first system;

[0040] a message parsing module, configured to read a message from the message queue, and parse the message according to the second service type and the information of the first system to obtain first data conforming to the second service type and information of the second system corresponding to the information of the first system in the message;

[0041] A data conversion module, used for converting the first data parsed from the message into second data according to the second data specification;

[0042] The second data sending module is used to send the second data parsed from the message to the second system parsed from the message.

[0043] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the heterogeneous system data integration methods described above are implemented.

[0044] In a fourth aspect, the invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the heterogeneous system data integration methods described above.

[0045] The heterogeneous system data integration method and device provided by the present invention respectively provide the first data specification and the second data specification that are compatible with the data in the heterogeneous first system and the second system through the cloud middleware. During data integration, the producer process reads the producer configuration file to obtain the first business type of the data integration task; obtains the configuration information that conforms to the first business type from the user configuration file, and the configuration information includes: the information of the first system and the information of the second system; obtains the first data that conforms to the first business type from the cloud middleware, and the first data is obtained by the cloud middleware from the first system in advance according to the first data specification; generates a message according to the first data, the information of the first system and the information of the second system, and puts the message into the message queue; the consumer process reads the consumer configuration file to obtain the second business type of the data integration task and the information of the first system; reads the message from the message queue, and parses the message according to the second business type and the information of the first system to obtain the first data that conforms to the second business type and the information of the second system corresponding to the information of the first system in the message; converts the first data parsed from the message into the second data according to the second data specification; and sends the second data to the second system parsed from the message. Through the above integration process, compared with establishing integration links between each other, the method of the present invention integrates each internal and external system with the cloud middleware only once based on standard specifications, which reduces duplication of work, shortens development period, reduces costs, and improves customer satisfaction. In addition, data distribution is uniformly scheduled through the message queue in the cloud middleware, and the data distribution link and distribution frequency are configurable, the distribution status can be monitored, and the maintenance cost is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 It is a flow chart of the heterogeneous system data integration method provided by the present invention;

[0048] Figure 2 It is a schematic diagram of the structure of the heterogeneous system data integration device provided by the present invention;

[0049] Figure 3 It is a schematic diagram of the composition structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] See also Figure 1 , Figure 1 It is a flow chart of a heterogeneous system data integration method provided by an embodiment of the present invention. The method is used for cloud middleware. A first data specification adapted to data in a first system and a second data specification adapted to data in a second system are pre-integrated in the cloud middleware. The first system and the second system are heterogeneous systems. The cloud middleware is an intermediary, which is deployed in a cloud server and provides data integration, transmission or forwarding services for related heterogeneous systems, such as an ERP system and a third-party system. The first system can be understood as an ERP system, and the second system can be understood as a third-party system. The method includes:

[0052] Step S110, the producer process reads the producer configuration file and obtains the first business type of the data integration task. The producer configuration file is pre-stored in the cloud middleware, and mainly includes the following four types of configuration tables: task role table taskrole, task role resource table taskroleres, business configuration table taskbiz, and task instruction table taskevent. Based on the producer configuration file, the role (including: producer and consumer) process set in the cloud middleware can be configured as a producer process. The configuration items of the producer process include: role type (producer), business type (producer is responsible for obtaining data of this business type), synchronization interval time for obtaining data, synchronization step length, and call class corresponding to the producer process, etc.

[0053] Step S120, obtaining configuration information that conforms to the first service type from the user configuration file, the configuration information including: information of the first system and information of the second system. The user configuration file is an information file that the user pre-configures in the cloud middleware through the configuration interface, wherein the user configuration file mainly includes: a user configuration table entconfig and a user synchronization information configuration table entsyncinfo, and the main information configured in these two tables includes: information of the first system, information of the second system, and service type, etc.

[0054] Step S130, obtaining first data that conforms to the first business type from the cloud middleware, the first data being obtained by the cloud middleware from the first system in advance according to the first data specification. Specifically, the producer process corresponding to the first business type obtains the first data from the cloud middleware, and when the first system requests the cloud middleware to synchronize data, the cloud middleware obtains the first data from the first system according to the request and the first data specification, and after obtaining, stores the first data in a data table corresponding to the first data of the cloud middleware.

[0055] Step S140, generating a message according to the first data, the information of the first system and the information of the second system, and placing the message into a message queue. Specifically, the producer configuration file is read in step S110, and the producer process can place the message into the message queue through the corresponding function in the calling class. The message queue is equivalent to a queuing mechanism, and the data obtained in the first system is forwarded to the second system in an orderly manner through queuing, avoiding the problem of server paralysis caused by large-scale data forwarding when the cloud middleware integrates multiple first systems and multiple second systems.

[0056] Step S150, the consumer process reads the consumer configuration file to obtain the second business type of the data integration task and the information of the first system. The consumer configuration file is pre-stored in the cloud middleware, and mainly includes the following three types of configuration tables: task role table taskrole, task role resource table taskroleres and task instruction table taskevent. Based on the consumer configuration file, the role (including: producer and consumer) process set in the cloud middleware can be configured as a consumer process. The configuration items of the consumer process mainly include: role type (consumer), business type (consumer is responsible for obtaining data of this business type from the message), information of the first system, information of the second system and the call class corresponding to the consumer process, etc.

[0057] Step S160, reading a message from the message queue, parsing the message according to the second business type and the information of the first system to obtain first data conforming to the second business type and information of the second system corresponding to the information of the first system in the message.

[0058] Step S170: convert the first data parsed from the message into the second data according to the second data specification. Specifically, after parsing the message in step S160, a call class corresponding to the information of the second type and the second system is obtained from the consumer configuration file, and the consumer process can convert the first data into the second data according to the second data specification through the corresponding function in the call class.

[0059] Step S180: sending the second data to the second system parsed from the message. The consumer process may send the second data to the second system through a corresponding function in the calling class.

[0060] It should be noted that there are multiple role processes in the cloud middleware, and the role process can be configured as a producer process or a consumer process for processing a certain business type of data through the producer configuration table and the consumer configuration table. Among them, the first system can be an ERP system, and the second system can be a third-party system connected to the ERP system.

[0061] Through the above data integration process, compared with establishing integration links between each other, the heterogeneous system data integration method of the present invention only integrates each internal and external system with the cloud middleware once based on various standard specifications, which reduces duplication of work, shortens the development period, reduces costs, and improves customer satisfaction. In addition, data distribution is uniformly scheduled through the message queue in the cloud middleware, and the data distribution link and distribution frequency are configurable, the distribution status can be monitored, and the maintenance cost is reduced.

[0062] In this embodiment, step S130 includes:

[0063] Step S131: Obtain the last synchronization time of the first system and the second system from the user synchronization information configuration table of the user profile. The synchronization configuration table is similar to the user configuration table and is also configured by the user. On the basis of the user configuration table, fields such as priority and last synchronization time are added.

[0064] Step S132: determine the data in the cloud middleware that meets the first business type and changes after the last synchronization time as the first data, and obtain the first data. That is, only the first data that has changed in the first system after the last synchronization is completed is synchronized with the second system, because after the data in the first system changes, it will request data synchronization to the cloud middleware, so the corresponding first data in the cloud middleware is also changing.

[0065] Furthermore, the producer process obtains the configuration information in the user synchronization information configuration table at preset synchronization intervals (the producer process obtains the configuration information in the user synchronization information configuration table according to the synchronization interval) and according to the user configuration priority. The first data of a certain business type to be synchronized between a first system and a second system in the configuration information obtained first will be obtained first and a message will be generated and placed in the message queue. When the subsequent consumer process processes it, the message will also be read out of the queue first and the data in the message will be sent to the second system.

[0066] In this embodiment, in step S140, after putting the message into the message queue, the following steps are further included:

[0067] The new synchronization time is determined according to the current time or the synchronization step configured in the producer configuration file. Specifically, the new synchronization time t2 can be the current last synchronization time t1 plus the synchronization step syncstep, that is, t2=t1+syncstep. If t2 is greater than the current time, then t2=current time.

[0068] The last synchronization time between the first system and the second system in the user synchronization information configuration table is updated with the new synchronization time t2.

[0069] In this embodiment, step S170 includes:

[0070] If the second system is a first-type system, the first data parsed from the message is converted into the second data according to the second data specification. The first-type system is a well-known third-party system on the market, such as: a third-party food delivery platform (Meituan, JD.com, Ele.me), a third-party payment platform (WeChat, Alipay, Cloud QuickPass), etc. The cloud middleware needs to adapt to the second data specification provided by the first-type system. Specifically, the second data specification is also provided to the cloud middleware in the form of an API interface function.

[0071] If the second system is a second-type system, the first data specification and the second data specification are the same, and the first data parsed from the message is designated as the second data. The second-type system is an unknown third-party system on the market. At this time, the cloud middleware actively formulates the second data specification, and the second data specification is also provided to the second-type system in the form of an API interface function. Of course, in order to reduce the amount of calculation for data conversion, the first data specification and the second data specification are set to be the same. At this time, the first data is the second data, and no additional conversion is required.

[0072] For some first data with low real-time performance, the second data corresponding to the first data can be parsed from the message and sent directly to the second system. However, for first data with high real-time performance, such as price, the price of the merchant online may change in minutes or even seconds. After the corresponding second data is parsed, its price may change, resulting in the second data being out of sync with the first data. In order to avoid this problem, step S170 includes the following steps before the conversion:

[0073] Step S181: query the updated first data corresponding to the keyword in the cloud middleware according to the keyword of the first data parsed in the message. In order to save storage space, for the first data with high real-time performance, the message can be generated only according to the keyword of the first data.

[0074] Step S182: during conversion, the updated first data is converted into the second data according to the second data specification.

[0075] In this embodiment, before step S110, the step further includes: the cloud middleware obtains data in the first system, which can be implemented in the following two ways:

[0076] Method 1: Receive the first data uploaded by the first system by calling an interface function that complies with the first data specification, and store it in the cloud middleware. That is, provide the API interface function to the first system, and the first system can prepare parameters according to the input parameter format of the API interface function. Method 1 is for the first system with secondary development capabilities, and through secondary development, the first system can call the API interface function according to the input parameter format of the API interface function.

[0077] Method 2: Read the prepared data in the data table set according to the first data specification in the first system; use the prepared data as a parameter to call the first data uploaded by the interface function (API interface function) that complies with the first data specification, and store it in the cloud middleware. Method 2 is for the first system that does not have secondary development capabilities. The first system needs to set up a data table according to the first data specification in advance, and the prepared data in the data table are the input parameters of the API interface function. Method 2 can be understood as setting up a front-end processor in the cloud middleware, and the front-end processor reads the prepared data in the data table set according to the first data specification in the first system; the front-end processor uses the prepared data as a parameter to call the first data uploaded by the interface function that complies with the first data specification.

[0078] The following is an example of integrating commodity data. Tables 1 to 6 are first configuration tables for configuring data in the first system, and Tables 7 to 9 are second configuration tables for configuring data in the second system.

[0079] Table 1 Task role taskrole

[0080]

[0081] Table 2 Task role resources taskroleres

[0082]

[0083] Table 3 Business configuration taskbiz

[0084]

[0085] Table 4 Task instruction taskevent

[0086]

[0087] Table 5 User configuration entconfig

[0088]

[0089] Table 6 User synchronization information configuration entsyncinfo

[0090]

[0091] Table 7 Task role taskrole

[0092]

[0093] Table 8 Task role resources taskroleres

[0094]

[0095] Table 9 Task instruction taskevent

[0096]

[0097] In the above tables, % and All data are represented.

[0098] The following takes the product data synchronization process as an example to explain:

[0099] The producer process steps are as follows:

[0100] Step 1: Start tasknode = 1001 and read the role to be started on the node from the taskrole table:

[0101] select roleid, roletype, aligroupid from taskrole where nodeid=1001order by roleid.

[0102] The node can be configured as a producer or consumer based on the roleid, i.e. the role code. If the roleid is the role code in the task role table taskrole in the producer configuration file, for example: 110001, then the node is configured as a producer. The producer configuration file includes: the task role table taskrole, the task role resource table taskroleres, the business configuration table taskbiz and the task instruction table taskevent, i.e. the above Tables 1 to 4.

[0103] Step 2: Determine the role type = producer, and read the business type and business group corresponding to the role from the task role resource table taskroleres according to the role:

[0104] select rescod1, rescod2 from taskroleres where roleid=110001 order byseqno.

[0105] The business type and business group are obtained from the task role resource table taskroleres in sequence, where rescod1 and rescod2 represent the first and second columns in taskroleres, namely the business type and business group.

[0106] Step 3: Read the corresponding TOPIC keyword and related business configuration parameters from the business configuration table taskbiz according to the business type + business group:

[0107] select topic, tableschema, syncinterval, syncstep, mesgbizcnt fromtaskbiz where biztype = 'GOODS' and bizgroup = '1'.

[0108] The keyword TOPIC is the keyword of the business type, and the business configuration parameters include: table SCHEMAtableschema, synchronization interval syncinterval, synchronization step syncstep, and message number mesgbizcnt. biztype indicates the business type, and bizgroup indicates the business group.

[0109] Step 4. According to instruction type = producer and business type = commodity, read the running call class from the task instruction table taskevent:

[0110] select classinfo from taskevent where eventtype='PRODUCER'and biztype='GOODS'.

[0111] Different business types correspond to different calling classes, and the calling class is used to assemble the data of the corresponding business type into a message in step 8 and send it to the message queue.

[0112] Step 5: According to the business type and business group, extract the data with the highest priority that meets the conditions from the merchant synchronization information configuration table entsyncinfo:

[0113] select from entsyncinfo where biztype='GOODS'and bizgroup ='1'andstatus = 1 and; lastsynctime<= variable (current time - synchronization interval syncinterval) order bypriority decs, lastsynctime asc limit 1.

[0114] According to the rules specified by the business configuration parameters in step 3 (for example, according to a certain synchronization interval), the merchant information (first system information) and docking platform information (second system information) that conform to the business type and business group are obtained from the user synchronization information configuration table in the user profile. That is, the data that conforms to the business type and business group in the merchant (first system, such as the ERP system used by the merchant) is synchronized to the docking platform (second system, such as a third-party system). The user profile includes: user configuration table entconfig and user synchronization information configuration table entsyncinfo. Among them, the business group is divided according to different merchants (first system), and different merchants corresponding to the same business type are divided into different business groups.

[0115] Furthermore, synchronization configuration information with an enabled status and high priority can be selected according to the priority and status configured in the user synchronization information configuration table. The data of a certain business type to be synchronized between a merchant and a docking platform in the synchronization configuration information will be obtained first and a message will be generated and placed in the message queue. When the subsequent consumer process is processed, the message will also be read out of the queue first and sent to a docking platform.

[0116] Step 6: According to the merchant + operating company + store obtained in step 5, retrieve the minimum time of change from the commodity table obtained in advance from the merchant in the cloud middleware, t1 = the last synchronization time lastsynctime obtained in step 5:

[0117] select min(last update time) from product table where merchant=merchant1and operating company=1and store=A01storeand last update time>=t1.

[0118] The minimum time of change is obtained from the commodity table in the cloud middleware, that is, the starting time t1 of obtaining data from the commodity table is determined.

[0119] Step 7: According to the merchant + operating company + store obtained in step 5 and the minimum update time obtained in step 6, retrieve the changed product set from the product table, t1 = min (last update time), t2 = t1 + synchronization step syncstep (if t2 > current time, then t2 = current time):

[0120] select from product table where merchant=merchant1and operating company=1and store=A01storeand last updated time>=t1and last updated time <t2。

[0121] From the commodity table in the cloud middleware, obtain the data of the merchant = merchant 1 and operating company = 1 and store = A01 store that changes from time t1 to t2 and meets the above step 5. For commodities, the real-time requirement of data is high, so it is sufficient to obtain the keywords of commodity data.

[0122] Step 8. Assemble product keywords, such as merchant + operating company + store + product + docking platform, into a json object, and assemble multiple json objects into a message. Send the message to RocketMQ by calling the corresponding function in the class, TOPIC = TP-GOODS-1, TAG = merchant one, message KEY = uid of the product data row, where the identifier of the product data row is used as the key value of the message, TOPIC = TP-GOODS-1 and TAG = merchant one are used for subsequent consumers to read the message.

[0123] Step 9: After all messages are sent, update the merchant synchronization information configuration table entsyncinfo with the last synchronization time = t2.

[0124] The consumer process steps are as follows:

[0125] Step 1: Start tasknode = 1002 and read the role to be started on the node from the taskrole table:

[0126] select roleid, roletype, aligroupid from taskrole where nodeid=1002order by roleid.

[0127] If roleid is the role code in the task role table taskrole in the consumer configuration file, for example: 210001, then the node is configured as a consumer. The producer configuration file includes: task role table taskrole, task role resource table taskroleres and task instruction table taskevent, that is, the above Tables 7 to 9.

[0128] Step 2: Determine if the role type is consumer, and read the corresponding TOPIC (business type) and TAG (merchant information) from taskroleres according to the Alibaba Cloud consumer group (aligroupid) corresponding to the role:

[0129] select roleid as aligroupid, rescode1 as TOPIC, rescode2 as TAG fromtaskroleres where roleid in ('CG-GOODS-1', 'CG-PRICE-1', 'CG-STOCK-1') order byseqno.

[0130] Step 3: Read messages based on consumer group aligroupid and TOPIC+TAG. Consumers belonging to consumer group aligroupid read messages corresponding to TOPIC+TAG from the message queue, that is, the messages include business type and merchant information.

[0131] Step 4. Parse the information in the message (all the information in the message can be obtained according to TOPIC+TAG, including: business type and docking platform), and read the running call class from taskevent according to instruction type = consumer and business type = product and docking platform = platform:

[0132] select classinfo from taskevent where eventtype ='CONSUMER'andbiztype ='GOODS'and platcode = platform one.

[0133] Step 5: Pass the message json object to the corresponding call class and send the product to the docking platform. Different business types and docking platforms correspond to different call classes. The call class sends the message to the corresponding docking platform. Since the message itself contains the merchant's product data (the first data of the first system), it is equivalent to sending the merchant's product data (the first data of the first system) to the docking platform (the second system). Since the first system and the second system are heterogeneous systems, before sending the message, the call class also converts the product data in the message into the data format corresponding to the docking platform according to the second data specification, and then sends the data after conversion.

[0134] Among them, for data such as products and prices that have high real-time requirements, it is necessary to re-read the information in the corresponding database in the cloud middleware based on keywords. Data such as inventory that do not require high real-time requirements can be assembled into messages and directly based on the messages in the message queue.

[0135] The heterogeneous system data integration device provided by the present invention is described below. The heterogeneous system data integration device described below and the heterogeneous system data integration method described above can be referenced to each other.

[0136] like Figure 2 As shown, the heterogeneous system data integration device of the present invention is used for cloud middleware, in which a first data specification adapted to data in a first system and a second data specification adapted to data in a second system are preset, and the first system and the second system are heterogeneous systems. The device includes:

[0137] The producer configuration reading module 210 is used for the producer process to read the producer configuration file and obtain the first business type of the data integration task;

[0138] A user configuration acquisition module 220, configured to acquire configuration information that conforms to the first service type from a user configuration file, the configuration information including: information of the first system and information of the second system;

[0139] A first data acquisition module 230 is used to acquire first data that complies with a first business type from the cloud middleware, where the first data is acquired in advance by the cloud middleware from the first system according to the first data specification;

[0140] A message generating module 240, configured to generate a message according to the first data, the information of the first system and the information of the second system, and put the message into a message queue;

[0141] A consumer configuration reading module 250 is used for the consumer process to read the consumer configuration file and obtain the second business type of the data integration task and the information of the first system;

[0142] a message parsing module 260, configured to read a message from the message queue, and parse the message according to the second service type and the information of the first system to obtain first data conforming to the second service type and information of the second system corresponding to the information of the first system in the message;

[0143] A data conversion module 270, configured to convert the first data parsed from the message into second data according to the second data specification;

[0144] The second data sending module 280 is used to send the second data parsed from the message to the second system parsed from the message.

[0145] The heterogeneous system data integration device of this embodiment, through the above-mentioned integrated functional modules, is compared with establishing integration links between each other. Each internal and external system is integrated with the cloud middleware only once based on standard specifications, which reduces duplication of work, shortens development period, reduces costs, and improves customer satisfaction. In addition, data distribution is uniformly scheduled through the message queue in the cloud middleware, and the data distribution link and distribution frequency are configurable, and the distribution status can be monitored, which reduces maintenance costs.

[0146] Optionally, the first data acquisition module 230 is specifically used for:

[0147] The last synchronization time of the first system and the second system is obtained from the user synchronization information configuration table of the user profile.

[0148] The data in the cloud middleware that conforms to the first business type and has changed after the last synchronization time is determined as the first data, and the first data is acquired.

[0149] Optionally, the heterogeneous system data integration device further includes:

[0150] A new synchronization time determination module, used to determine a new synchronization time according to the current time or the synchronization step configured in the producer configuration file;

[0151] The last synchronization time updating module is used to update the last synchronization time between the first system and the second system in the user synchronization information configuration table with the new synchronization time.

[0152] Optionally, the data conversion module 270 is specifically used for:

[0153] If the second system is a first type system, the first data parsed from the message is converted into second data according to the second data specification.

[0154] If the second system is a second type system, the first data specification and the second data specification are the same, and the first data parsed from the message is designated as the second data.

[0155] Optionally, the data conversion module 270 is further specifically used for:

[0156] Before the conversion, the updated first data corresponding to the keyword in the cloud middleware is queried according to the keyword of the first data parsed from the message.

[0157] During conversion, the updated first data is converted into the second data according to the second data specification.

[0158] Optionally, the heterogeneous system data integration device further includes:

[0159] The first data receiving module is used to receive the first data uploaded by the first system by calling an interface function that complies with the first data specification, and store the first data in the cloud middleware.

[0160] Optionally, the heterogeneous system data integration device further includes:

[0161] A preliminary data reading module, used for reading preliminary data in a data table set according to the first data specification in the first system;

[0162] The interface function calling module is used to call the first data uploaded by the interface function conforming to the first data specification with the prepared data as a parameter, and store the first data in the cloud middleware.

[0163] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a heterogeneous system data integration method, which is applied to a cloud middleware, in which a first data specification adapted to data in a first system and a second data specification adapted to data in a second system are preset, and the first system and the second system are heterogeneous systems, and the method includes:

[0164] The producer process reads the producer configuration file and obtains the first business type of the data integration task.

[0165] Configuration information that complies with the first service type is acquired from the user configuration file, where the configuration information includes: information of the first system and information of the second system.

[0166] First data conforming to a first business type is obtained from the cloud middleware, where the first data is obtained in advance by the cloud middleware from a first system according to the first data specification.

[0167] A message is generated according to the first data, the information of the first system and the information of the second system, and the message is put into a message queue.

[0168] The consumer process reads the consumer configuration file to obtain the second business type of the data integration task and information about the first system.

[0169] A message is read from the message queue, and the message is parsed according to the second business type and the information of the first system to obtain first data conforming to the second business type and information of the second system corresponding to the information of the first system in the message.

[0170] The first data parsed from the message is converted into second data according to the second data specification.

[0171] The second data is sent to a second system parsed from the message.

[0172] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0173] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the heterogeneous system data integration method provided by the above methods, the method is applied to a cloud middleware, a first data specification adapted to the data in the first system and a second data specification adapted to the data in the second system are preset in the cloud middleware, the first system and the second system are heterogeneous systems, the method includes:

[0174] The producer process reads the producer configuration file and obtains the first business type of the data integration task.

[0175] Configuration information that complies with the first service type is acquired from the user configuration file, where the configuration information includes: information of the first system and information of the second system.

[0176] First data conforming to a first business type is obtained from the cloud middleware, where the first data is obtained in advance by the cloud middleware from a first system according to the first data specification.

[0177] A message is generated according to the first data, the information of the first system and the information of the second system, and the message is put into a message queue.

[0178] The consumer process reads the consumer configuration file to obtain the second business type of the data integration task and information about the first system.

[0179] A message is read from the message queue, and the message is parsed according to the second business type and the information of the first system to obtain first data conforming to the second business type and information of the second system corresponding to the information of the first system in the message.

[0180] The first data parsed from the message is converted into second data according to the second data specification.

[0181] The second data is sent to a second system parsed from the message.

[0182] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the heterogeneous system data integration method provided by the above methods, the method is applied to a cloud middleware, a first data specification adapted to data in a first system and a second data specification adapted to data in a second system are preset in the cloud middleware, the first system and the second system are heterogeneous systems, and the method includes:

[0183] The producer process reads the producer configuration file and obtains the first business type of the data integration task.

[0184] Configuration information that complies with the first service type is acquired from the user configuration file, where the configuration information includes: information of the first system and information of the second system.

[0185] First data conforming to a first business type is obtained from the cloud middleware, where the first data is obtained in advance by the cloud middleware from a first system according to the first data specification.

[0186] A message is generated according to the first data, the information of the first system and the information of the second system, and the message is put into a message queue.

[0187] The consumer process reads the consumer configuration file to obtain the second business type of the data integration task and information about the first system.

[0188] A message is read from the message queue, and the message is parsed according to the second business type and the information of the first system to obtain first data conforming to the second business type and information of the second system corresponding to the information of the first system in the message.

[0189] The first data parsed from the message is converted into second data according to the second data specification.

[0190] The second data is sent to a second system parsed from the message.

[0191] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0192] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A heterogeneous system data integration method, characterized in that: For cloud middleware, a first data specification adapted to data in a first system and a second data specification adapted to data in a second system are preset in the cloud middleware, the first system and the second system are heterogeneous systems, and the method includes: The producer process reads the producer configuration file and obtains the first business type of the data integration task; Acquire configuration information that conforms to the first service type from a user configuration file, the configuration information comprising: information of the first system and information of the second system; Acquire first data that conforms to a first business type from the cloud middleware, where the first data is acquired in advance by the cloud middleware from a first system according to the first data specification; Generate a message according to the first data, the information of the first system and the information of the second system, and put the message into a message queue; The consumer process reads the consumer configuration file to obtain the second business type of the data integration task and information of the first system; Reading a message from the message queue, parsing the message according to the second service type and the information of the first system to obtain first data conforming to the second service type and information of the second system corresponding to the information of the first system in the message; Converting the first data parsed from the message into second data according to the second data specification; The second data is sent to a second system parsed from the message.

2. The heterogeneous system data integration method according to claim 1, characterized in that: Acquiring first data that complies with a first business type from the cloud middleware includes: Acquire the last synchronization time of the first system and the second system from the user synchronization information configuration table of the user profile; The data in the cloud middleware that conforms to the first business type and has changed after the last synchronization time is determined as the first data, and the first data is acquired.

3. The heterogeneous system data integration method according to claim 2, characterized in that: After putting the message into the message queue, it also includes: Determine a new synchronization time according to the current time or the synchronization step configured in the producer configuration file; The last synchronization time between the first system and the second system in the user synchronization information configuration table is updated with the new synchronization time.

4. The heterogeneous system data integration method according to claim 1, characterized in that: Converting the first data parsed from the message into the second data according to the second data specification includes: If the second system is a first type system, converting the first data parsed from the message into second data according to the second data specification; If the second system is a second type system, the first data specification and the second data specification are the same, and the first data parsed from the message is designated as the second data.

5. The heterogeneous system data integration method according to claim 1, characterized in that: The step of converting the first data parsed from the message into the second data according to the second data specification includes, before the conversion: According to the keyword of the first data parsed in the message, query the updated first data corresponding to the keyword in the cloud middleware; During conversion, the updated first data is converted into the second data according to the second data specification.

6. The heterogeneous system data integration method according to any one of claims 1 to 5, characterized in that: Before the producer process reads the producer configuration file and obtains the first business type of the data integration task, the process further includes: The first data uploaded by the first system by calling an interface function conforming to the first data specification is received and stored in the cloud middleware.

7. The heterogeneous system data integration method according to any one of claims 1 to 5, characterized in that: Before the producer process reads the producer configuration file and obtains the first business type of the data integration task, the process further includes: Reading the preliminary data in the data table set according to the first data specification in the first system; The first data uploaded by calling an interface function conforming to the first data specification is taken as a parameter of the prepared data and stored in the cloud middleware.

8. A heterogeneous system data integration device, characterized in that: Used in cloud middleware, in which a first data specification adapted to data in a first system and a second data specification adapted to data in a second system are preset, the first system and the second system are heterogeneous systems, and the device includes: The producer configuration reading module is used for the producer process to read the producer configuration file and obtain the first business type of the data integration task; A user configuration acquisition module, configured to acquire configuration information that conforms to the first service type from a user configuration file, the configuration information including: information of the first system and information of the second system; A first data acquisition module, configured to acquire first data conforming to a first business type from the cloud middleware, wherein the first data is acquired in advance by the cloud middleware from the first system according to the first data specification; A message generating module, used for generating a message according to the first data, the information of the first system and the information of the second system, and putting the message into a message queue; A consumer configuration reading module, used for a consumer process to read a consumer configuration file and obtain the second business type of the data integration task and information of the first system; a message parsing module, configured to read a message from the message queue, and parse the message according to the second service type and the information of the first system to obtain first data conforming to the second service type and information of the second system corresponding to the information of the first system in the message; A data conversion module, used for converting the first data parsed from the message into second data according to the second data specification; The second data sending module is used to send the second data parsed from the message to the second system parsed from the message.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the heterogeneous system data integration method as described in any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the heterogeneous system data integration method as described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Integration tool for telecommunication area inheritance system based on configuration policy

    CN101188520A

  • Device and method for data integration processing for heterogenous system

    CN102867067A