Data processing method and device, equipment and computer readable storage medium

Abstract

The application provides a data processing method, device and equipment and a computer readable storage medium; the method comprises: obtaining a data processing request of a second service system which is heterogeneous with a first service system, the data processing request comprising version information of a data set to be processed used in the second service system and data processing logic for the data set; wherein the data records in the data set are obtained from the first service system; determining, from the data set used in the first service system, first data records to be processed which match the version information of the data set; and processing the first data records based on the data processing logic of the data set. Through the application, the problem of inaccurate data processing caused by unsynchronized data between the first service system and the second service system can be avoided, so that the processing logic for the data records in the data set can be reliably completed in the heterogeneous system, and the consistency of the final result of the data processing logic is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to, but is not limited to, the technical field of computer information, and in particular to a data processing method and device, equipment and a computer readable storage medium. BACKGROUND

[0002] In an Internet environment, there are often cases that different service systems or devices need to query, modify and process data records in the same data set. For multiple heterogeneous service systems or devices, a copy of the data set needs to be stored in different service systems or devices, and the data records in the data set need to be synchronized regularly. However, since the data between the multiple heterogeneous service systems or devices cannot be guaranteed to be completely synchronized in real time, during the process of processing the data, the data processing logic and the processing result may be inconsistent due to the inconsistent data used by different service systems or devices. SUMMARY

[0003] The embodiments of the present application provide a data processing method, device, equipment and computer readable storage medium, which can avoid the problem of inaccurate data processing caused by unsynchronized data between a first service system and a second service system, so as to reliably complete the processing logic of the data records in the data set in the heterogeneous system and ensure the consistency of the final result of the data processing logic.

[0004] The technical scheme of the embodiments of the present application is implemented as follows:

[0005] The embodiments of the present application provide a data processing method, which comprises: obtaining a data processing request of a second service system which is heterogeneous with a first service system, wherein the data processing request comprises version information of a data set to be processed used in the second service system and a data processing logic for the data set; wherein the data records in the data set are obtained from the first service system by synchronization; determining first data records to be processed which match the version information of the data set from a data set used in the first service system; and processing the first data records based on the data processing logic of the data set.

[0006] The embodiments of the present application provide a data processing method, which comprises:

[0007] Obtaining each data record in a data set to be processed used in a second service system and version information of the data set; wherein the data records in the data set are obtained from a first service system which is heterogeneous with the second service system by synchronization;

[0008] Determining a data processing logic for the data set based on each data record in the data set;

[0009] determine a data processing request based on the version information and the data processing logic of the data set, and send the data processing request to the first service system, so that the first service system determines, from the data set used by the first service system, a first data record to be processed that matches the version information, and processes the first data record based on the data processing logic of the data set.

[0010] In some embodiments, the method further includes: timing synchronization of data records in a data set used by the first service system to a data set used by the second service system, and obtaining current version information of the data set used by the first service system.

[0011] In some embodiments, each data record in the data set has an attribute value, and in a process in which the first service system adds data records to the data set, the attribute value of each newly added data record is monotonically changed relative to the attribute value of a previously added data record, and the obtaining of each data record in the data set to be processed used by the second service system and the version information of the data set includes: obtaining each data record in the data set to be processed used by the second service system; and determining the version information of the data set based on the attribute value of each data record.

[0012] In some embodiments, in the process in which the first service system adds data records to the data set, the attribute value of each newly added data record is monotonically increased relative to the attribute value of a previously added data record, and the determining of the version information of the data set based on the attribute value of each data record includes: determining the maximum value of the attribute values of the data records as the version information of the data set.

[0013] In some embodiments, the first service system includes a data processing service, and the second service system includes a business logic service, the determining of the data processing logic for the data set based on each data record in the data set includes: the business logic service determines the data processing logic for the data set based on each data record in the data set; the determining of the data processing request based on the version information and the data processing logic of the data set, and the sending of the data processing request to the first service system includes: the business logic service determines the data processing request based on the version information and the data processing logic of the data set; and the business logic service sends the data processing request to the data processing service, so that the data processing service determines, from the data set used by the first service system, a first data record to be processed that matches the version information, and processes the first data record based on the data processing logic of the data set.

[0014] In some embodiments, the first service system further comprises a data access service, the second service system further comprises a data synchronization service, and the method further comprises: the data synchronization service receiving each data record in a data set used in the first service system and current version information of the data set sent by the data access service, the current version information of the data set being determined by the data access service based on the attribute value of each data record; and the data synchronization service updating a data set used in the second service system by using each data record.

[0015] Embodiments of the present application provide a data processing apparatus, comprising:

[0016] A first obtaining module is configured to obtain a data processing request of a second service system which is heterogeneous with a first service system, the data processing request comprising version information of a data set to be processed used in the second service system and data processing logic for the data set; wherein the data records in the data set are obtained from the first service system by synchronization.

[0017] A first determining module is configured to determine, from a data set used in the first service system, first data records to be processed which match the version information of the data set.

[0018] A first processing module is configured to process the first data records based on the data processing logic of the data set.

[0019] Embodiments of the present application provide a data processing apparatus, comprising:

[0020] A fourth obtaining module is configured to obtain each data record in a data set to be processed used in a second service system and version information of the data set; wherein the data records in the data set are obtained from a first service system which is heterogeneous with the second service system by synchronization.

[0021] A fifth determining module is configured to determine, based on each data record in the data set, data processing logic for the data set.

[0022] A second sending module is configured to determine a data processing request based on the version information and the data processing logic of the data set, and send the data processing request to the first service system, so that the first service system determines, from a data set used in the first service system, first data records to be processed which match the version information, and processes the first data records based on the data processing logic of the data set.

[0023] The embodiment of the present application provides a data processing device, comprising: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the method provided by the embodiment of the present application.

[0024] The embodiment of the present application provides a computer readable storage medium storing executable instructions for causing a processor to execute the method provided by the embodiment of the present application.

[0025] The embodiment of the present application has the following beneficial effects:

[0026] By obtaining a data processing request of a second service system heterogeneous with a first service system, the data processing request comprising version information of a data set to be processed used in the second service system and data processing logic for the data set, wherein the data records in the data set are obtained from the first service system in synchronization; from the data set used in the first service system, a first data record to be processed matching the version information of the data set is determined; and the first data record is processed based on the data processing logic of the data set. In this way, since the version information of the data set to be processed is carried in the process of data processing logic interaction between the heterogeneous first service system and the second service system, the first data record to be processed in the data set used in the first service system is determined based on the version information, which can avoid the problem of inaccurate data processing caused by unsynchronized data between the first service system and the second service system, so that the processing logic of the data records in the data set can be reliably completed in the heterogeneous system, and the consistency of the final result of the data processing logic is ensured. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is an optional architecture schematic diagram of the data processing system provided by the embodiment of the present application;

[0028] Figure 2A is an optional structure schematic diagram of the data processing device provided by the embodiment of the present application;

[0029] Figure 2B is an optional structure schematic diagram of the data processing device provided by the embodiment of the present application;

[0030] Figure 3 is an optional flow schematic diagram of the data processing method provided by the embodiment of the present application;

[0031] Figure 4 is an optional flow schematic diagram of the data processing method provided by the embodiment of the present application;

[0032] Figure 5 is an optional flow schematic diagram of the data processing method provided by the embodiment of the present application;

[0033] Figure 6 is an optional flowchart of a data processing method provided by an embodiment of the present application;

[0034] Figure 7 is an optional flowchart of a data processing method provided by an embodiment of the present application;

[0035] Figure 8A is an implementation architecture diagram of a data processing method provided by an embodiment of the present application;

[0036] Figure 8B is an implementation diagram of determining version information of a data set based on generation time of each data record in the data set provided by an embodiment of the present application;

[0037] Figure 8C is an implementation diagram of determining version information of a data set based on self-increment data ID of each data record in the data set provided by an embodiment of the present application;

[0038] Figure 8D is an implementation flowchart of a data processing method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be described in further detail below with reference to the drawings, and the described embodiments should not be regarded as limiting the present application, all other embodiments obtained by those skilled in the art without making creative labor are within the scope of protection of the present application.

[0040] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict.

[0041] If similar descriptions of "first / second" appear in the application file, the following description is added, in the following description, the terms "first\second\third" referred to only distinguish similar objects, and do not represent a specific order of the objects, and it can be understood that "first\second\third" can be interchanged with a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0043] The related data collection and processing in the embodiments of the present application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and within the scope of authorization of laws and regulations and the personal information subject, carry out subsequent data use and processing when applied.

[0044] In order to better understand the data processing method provided by the embodiments of the present application, the data processing scheme in the related art will be described first.

[0045] In the related art, the isomorphic multiple service systems usually synchronize data sets through interfaces, and store the data sets to be processed in each service system. Different service systems can determine data processing logic based on the data sets stored by themselves according to business needs, and send the corresponding data processing logic to other service systems, so that other service systems can perform corresponding data processing on the stored data sets based on the data processing logic. For example, some data management systems providing basic services will periodically or in real time synchronize data records in data sets with external business systems, and at the same time, the external business systems will also configure or adjust the data sets used in the data management systems according to their own business needs. In this scenario, since the data records in the data sets used in the data management systems can be dynamically increased, decreased or modified at any time, the data records in the data sets synchronized by the external business systems may not be the latest, which may cause inconsistent data processing logic or inconsistent final processing results.

[0046] Taking the process of implementing rule inactivation configuration by the rule system based on the rule configuration logic determined by the external business system as an example, in the related art, the external business system usually pulls the full set of rule data through the data synchronization interface of the rule system regularly (or before the rule inactivation configuration is performed), and then the external business system determines the rule configuration logic for configuring rule inactivation based on the rule data set pulled recently according to business needs, and sends a rule configuration request to the rule system through the rule configuration interface of the rule system, so that the rule system configures the rules in the rule data set used in the rule system based on the rule configuration logic.

[0047] In the rule invalidation configuration scheme of the related art, there are the following problems: 1) there is a time window between the completion of rule data synchronization and the sending of the rule configuration request by the external business system, and in the time window, the rule system can increase or decrease rules at any time, so that the consistency of the rule data between the two systems and the consistency of the rule invalidation configuration logic result cannot be guaranteed; 2) due to the inconsistency of the rule data, the external business system may have added some rules in the rule system when determining the rule invalidation configuration logic, and the new rules will not participate in the rule invalidation configuration because they are not perceived by the external business system, resulting in the misoperation of the new rules; 3) the rule system lacks a version management mechanism for rule data, so that when receiving the rule configuration request, the difference between the rule data set and the external business system cannot be compared, and the newly added rules in the current rule data set cannot be distinguished, so that the same configuration logic is performed on the newly added rules and other rules, which may result in unexpected rule configuration results.

[0048] The embodiment of the present application provides a data processing method, device and medium, which can avoid the problem of inaccurate data processing caused by unsynchronized data between the first service system and the second service system, thereby reliably completing the processing logic of the data records in the heterogeneous system and ensuring the consistency of the final result of the data processing logic. The following describes an exemplary application of the data processing device provided by the embodiment of the present application. The data processing device provided by the embodiment of the present application can be implemented as a notebook computer, a tablet computer, a desktop computer, a set-top box, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated message device, a portable game device) and various types of user terminals, or as a server. The following describes an exemplary application of the data processing device provided by the embodiment of the present application when implemented as a first server or a second server.

[0049] Referring to Figure 1 , Figure 1 is an optional architecture schematic diagram of the data processing system 100 provided by the embodiment of the present application, which can reliably complete the processing logic of the data records in the heterogeneous system. The terminal (exemplarily shown as the terminal 400-1 and the terminal 400-2) is connected to the second server 202 through the network 301, the second server 202 is connected to the first server 201 through the network 302, and the network 301 and the network 302 can be a wide area network or a local area network, or a combination of the two. The network 301 and the network 302 can be the same network or different networks.

[0050] The terminal is configured to display an interactive interface for the user to configure the data processing logic in a graphical interface (exemplarily shown as graphical interface 410-1 and graphical interface 410-2), receive the data processing logic configuration operation of the user, and send the configured data processing logic to the second service system deployed in the second server 202.

[0051] The first server 201 is configured to deploy a first service system, which is configured to: obtain a data processing request of a second service system heterogeneous to the first service system, the data processing request including version information of a data set to be processed used in the second service system and data processing logic for the data set; wherein the data records in the data set are obtained from the first service system; determine, from the data set used in the first service system, first data records to be processed matching the version information of the data set; and process the first data records based on the data processing logic of the data set.

[0052] The second server 202 is configured to deploy a second service system, which is configured to: obtain each data record in a data set to be processed used in the second service system and version information of the data set; wherein the data records in the data set are obtained from a first service system heterogeneous to the second service system; determine, based on each data record in the data set, data processing logic for the data set; determine, based on the version information and the data processing logic of the data set, a data processing request, and send the data processing request to the first service system, so that the first service system determines, from the data set used in the first service system, first data records to be processed matching the version information, and processes the first data records based on the data processing logic of the data set.

[0053] In some embodiments, the first server 201 and the second server 202 can each be an independent physical server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and the present embodiments do not limit the connection manner.

[0054] Further, the data processing system involved in the embodiments of the present application can also be a distributed system applied to a blockchain system, and the first server 201 and the second server 202 can be implemented as nodes on the blockchain. The distributed system can be a distributed node formed by a plurality of nodes and clients, and the nodes can be any form of computing devices in the access network, such as servers, user terminals, etc., and the nodes form a peer-to-peer (P2P, Peer To Peer) network.

[0055] Referring to Figure 2A , Figure 2A is a structural schematic diagram of the data processing device 201 provided by the embodiments of the present application, Figure 2A The data processing device 201 shown in the figure includes at least one processor 210, a memory 250, at least one network interface 220, and a user interface 230. The various components in the data processing device 201 are coupled together through a bus system 240. It can be understood that the bus system 240 is used to realize the connection communication between the components. In addition to the data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 240 in the figure. Figure 2A

[0056] The processor 210 can be an integrated circuit chip with signal processing capability, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor.

[0057] The user interface 230 includes one or more output devices 231 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 230 also includes one or more input devices 232, including user interface components that facilitate user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons and controls.

[0058] The memory 250 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 250 optionally includes one or more storage devices physically located in proximity to the processor 210.

[0059] ​The memory 250 includes volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. The nonvolatile memory can be read only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 250 described in the embodiments of the present application is intended to include any suitable type of memory.

[0060] In some embodiments, the memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or superset thereof, which are exemplarily illustrated below.

[0061] The operating system 251 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks;

[0062] The network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including Bluetooth, wireless compatibility certification (WiFi), and universal serial bus (USB), and the like;

[0063] The presentation module 253 is used to enable the presentation of information via one or more output devices 231 (e.g., display screens, speakers, and the like) associated with the user interface 230 (e.g., user interfaces for operating peripheral devices and displaying content and information);

[0064] The input processing module 254 is used to detect and interpret one or more user inputs or interactions from one or more input devices 232.

[0065] In some embodiments, the data processing apparatus provided by the embodiments of the present application can be realized in software, Figure 2A The data processing apparatus 255 stored in the memory 250 is shown, which can be software in the form of programs and plug-ins, including the following software modules: a first acquisition module 2551, a first determination module 2552, and a first processing module 2553. These modules are logical, and thus can be combined or further split according to the functions implemented.

[0066] The functions of the various modules will be described below.

[0067] In some embodiments, the data processing apparatus provided by the embodiments of the present application can be implemented in a hardware manner. For example, the data processing apparatus provided by the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to perform the storage method provided by the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can use one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic elements.

[0068] Referring to Figure 2B , Figure 2B is a structural schematic diagram of the data processing device 202 provided by the embodiments of the present application, Figure 2B The data processing device 202 shown in the figure includes at least one processor 310, a memory 350, at least one network interface 320, and a user interface 330. The various components in the data processing device 202 are coupled together through a bus system 340. It can be understood that the bus system 340 is used to realize the connection and communication between the components. In addition to including a data bus, the bus system 340 also includes a power bus, a control bus, and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 340 in Figure 2B .

[0069] The processor 310 can be an integrated circuit chip with signal processing capability, such as a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor.

[0070] The user interface 330 includes one or more output devices 331 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 330 also includes one or more input devices 332 that facilitate user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons and controls.

[0071] The memory 350 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 350 optionally includes one or more storage devices physically located in proximity to the processor 310.

[0072] The memory 350 includes volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. The nonvolatile memory can be ROM, and the volatile memory can be RAM. The memory 350 described in the embodiments of the present application is intended to include any suitable type of memory.

[0073] In some embodiments, the memory 350 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or superset thereof, which are exemplarily illustrated below.

[0074] The operating system 351 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks;

[0075] The network communication module 352 is used to reach other computing devices via one or more (wired or wireless) network interfaces 320, exemplary network interfaces 320 including Bluetooth, WiFi, and USB, and the like;

[0076] The presentation module 353 is used to enable the presentation of information via one or more output devices 331 associated with the user interface 330 (e.g., a display screen, a speaker, and the like) (e.g., a user interface for operating peripheral devices and displaying content and information);

[0077] The input processing module 354 is used to detect and interpret one or more user inputs or interactions from one or more input devices 332.

[0078] In some embodiments, the data processing apparatus provided by the embodiments of the present application can be realized in software, Figure 2B The data processing apparatus 355 stored in the memory 350 is shown, which can be software in the form of programs and plug-ins, including the following software modules: a fourth acquisition module 3551, a fifth determination module 3552, and a second sending module 3553. These modules are logical, and thus can be combined or further split according to the implemented functions.

[0079] The functions of the various modules will be described below.

[0080] In some embodiments, the data processing apparatus provided by the embodiments of the present application can be implemented in a hardware manner. For example, the data processing apparatus provided by the embodiments of the present application can be a hardware decoding processor programmed to execute the storage method provided by the embodiments of the present application. For example, the hardware decoding processor can be one or more ASICs, DSPs, PLDs, CPLDs, FPGAs or other electronic elements.

[0081] The data processing method provided by the embodiments of the present application will be described below in combination with exemplary applications and implementations of the terminal or server provided by the embodiments of the present application.

[0082] Referring to Figure 3 , Figure 3 is an optional flowchart of the data processing method provided by the embodiments of the present application. The following steps will be described in combination with the steps shown in Figure 3 The execution subject of the following steps can be the terminal or the first server described above.

[0083] In step S101, a data processing request of a second service system heterogeneous to a first service system is obtained, the data processing request comprising version information of a data set to be processed used in the second service system and data processing logic for the data set; wherein the data records in the data set are obtained by synchronization from the first service system.

[0084] Here, the first service system and the second service system are heterogeneous systems. Heterogeneous systems refer to systems with different technical architectures, such as systems developed in different languages, systems running on different system platforms, systems using different databases for data storage, etc. Heterogeneous systems are composed of different architectures, so the organization methods of message data and the like are different, and it is not easy to synchronize and share information, and data needs to be accessed by each other through interfaces. The first service system and the second service system can be deployed on the same server terminal, or can be deployed separately. The deployment mode can be independent deployment on a separate server, or can be deployed in a distributed manner, which is not limited here. In implementation, the first service system and the second service system can each include at least one service, and the number of services included in the first service system and the second service system can be determined by those skilled in the art according to actual conditions, which is not limited here.

[0085] For a data set to be processed, a copy of the data set can be stored in the first service system and the second service system respectively, the data set can include at least one data record, and the first service system and the second service system can use the respective stored data set to complete different business requirements. The data set used in the first service system and the second service system can be stored in a suitable manner, for example, it can be stored in memory or a file system, or it can be stored in a database. The data set to be processed can be maintained in the first service system, including but not limited to adding, deleting, modifying, etc. of the data set in the first service system. In implementation, the data set used in the first service system can be entered by the user in the first service system, can be generated by the first service system according to actual requirements, or can be synchronized from other service systems, which is not limited here.

[0086] The data record in the data set to be processed used in the second service system is synchronized from the first service system, and the synchronization method can include at least one of subscription, timed pulling, etc. The data set obtained by the second service system each time after synchronizing the data record from the first service system can have a version information of the data set, which can include a version number or a version identifier. Based on the current version information of the data set, the data record currently included in the data set can be determined. For example, after the second service system synchronizes the data record from the first service system for the first time, the data set obtained will correspond to a version information, and after the second service system synchronizes the data record from the first service system for the second time, the data set obtained will correspond to a new version information. In implementation, the version information of the data set to be processed used in the second service system can be sent by the first service system to the second service system each time when the second service system synchronizes the data record from the first service system, or can be determined by the second service system based on the data record in the data set to be processed used in the second service system, which is not limited here.

[0087] The data processing request can be a request sent by the second service system to the first service system. The data processing request can include the version information of the data set used in the second service system and the data processing logic for the data set. The data processing logic for the data set can be a logic for processing the data record in the data set according to actual business requirements, such as adding, modifying, deleting, etc. of the data record in the data set. In implementation, the data processing logic for the data set can be configured by the user in the second service system according to actual business requirements, or can be generated automatically by the second service system based on the data record in the current data set and actual business requirements.

[0088] In step S102, from the data set used by the first service system, determine a first data record to be processed matching the version information of the data set.

[0089] Here, since there is a time window between the time when the second service system completes the synchronization of the data records of the data set and the time when the second service system receives the data processing request of the second service system from the first service system, the data records in the data set used by the first service system can change in the time window. The first data record to be processed matching the version information of the data set refers to the data record included in the data set corresponding to the version information, and the data record added after the version information does not match the version information. In implementation, the version information of the data set to be processed in the second service system can be used to filter the first data record to be processed matching the version information from the data set used by the first service system. For example, the first service system can store the mapping relationship between the version information of the data set and the data record, and by querying the mapping relationship, the first data record to be processed matching the version information can be determined. For another example, each data record in the data set can include an attribute value representing the version to which the data record belongs, and by comparing the attribute value of each data record in the data set with the version information, it can be determined whether each data record matches the version information.

[0090] In step S103, based on the data processing logic of the data set, process the first data record.

[0091] Here, the data processing logic of the data set can be a logic for processing all data records in the first data record, such as modifying a specific field of each data record in the first data record, adding a specific attribute to each data record in the first data record, etc. The first service system can execute the data processing logic on each data record in the first data record based on the data processing logic.

[0092] The data processing logic of the data set can also be a logic for processing data records in the first data record that meet a data filtering condition, and the data processing logic can include a data filtering condition and a processing operation for data records meeting the filtering condition. For example, modifying a specific field of a data record in the first data record whose modification level is greater than a level threshold, modifying a specific field of a data record in the first data record having a specified identifier, adding a specific attribute to each data record in the first data record whose state is valid, etc. The first service system can filter target data records from the first data record based on the data filtering condition in the data processing logic, and execute the processing operation in the data processing logic on the target data records.

[0093] In the embodiment of the present application, a data processing request of a second service system heterogeneous to a first service system is acquired, the data processing request comprising version information of a data set to be processed used in the second service system and data processing logic for the data set; wherein the data records in the data set are obtained from the first service system in synchronization; from the data set used in the first service system, a first data record to be processed matching the version information of the data set is determined; and the first data record is processed based on the data processing logic of the data set. In this way, since the version information of the data set to be processed is carried in the process of data processing logic interaction between the heterogeneous first service system and the second service system, the first data record to be processed in the data set used in the first service system is determined based on the version information, which can avoid the problem of inaccurate data processing caused by unsynchronized data between the first service system and the second service system, so that the processing logic of the data records in the data set can be reliably completed in the heterogeneous system, and the consistency of the final result of the data processing logic is ensured.

[0094] In some embodiments, referring to Figure 4 , Figure 4 is an optional flowchart of a data processing method provided by the embodiment of the present application, based on Figure 3 , each data record in the data set has an attribute value, and in the process of adding a new data record to the data set, the attribute value of each new data record is monotonically changed relative to the attribute value of the previous new data record. The method can further perform the following steps S401 to S403. The following steps will be described in combination with each step, and the execution subject of the following steps can be the terminal or the first server described above.

[0095] In step S401, each current data record in the data set used in the first service system is acquired.

[0096] Here, each data record in the data set has an attribute value, and in the process of adding a new data record to the data set, the attribute value of each new data record is monotonically changed relative to the attribute value of the previous new data record. In implementation, the attribute value of each new data record can be monotonically increased relative to the attribute value of the previous new data record, such as the generation time of the data record, the self-incremented ID, etc., and the attribute value of each new data record can also be monotonically decreased relative to the attribute value of the previous new data record, such as the self-decremented ID of the data record, etc. Those skilled in the art can select a suitable attribute value according to actual conditions, which is not limited here.

[0097] In step S402, the current version information of the data set is determined based on the attribute value of each data record.

[0098] Here, since the attribute value of each newly added data record is monotonically changed relative to the attribute value of the previous newly added data record in the process of adding data records to the data set, and the version information can be used to distinguish each data record currently included in the data set from the subsequently added data record, the current version information of the data set can be determined based on the attribute value of each data record currently included in the data set. In implementation, the version information can be a critical value that distinguishes the attribute value of the data record already in the current data set from the attribute value of the subsequently added data record, or a numerical interval that includes the attribute value of the data record already in the current data set but does not include the attribute value of the subsequently added data record.

[0099] For example, in the case where the attribute value of each newly added data record is monotonically increased relative to the attribute value of the previous newly added data record, the maximum value among the attribute values of the data records can be determined as the current version information of the data set, or the numerical interval between the minimum value and the maximum value among the attribute values of the data records can be determined as the current version information of the data set.

[0100] For another example, in the case where the attribute value of each newly added data record is monotonically decreased relative to the attribute value of the previous newly added data record, the minimum value among the attribute values of the data records can be determined as the current version information of the data set, or the numerical interval between the minimum value and the maximum value among the attribute values of the data records can be determined as the current version information of the data set.

[0101] In step S403, each of the data records and the version information is synchronized to the second service system.

[0102] Here, the first service system can synchronize each data record and the current version information of the data set to the second service system in the manner of message pushing. The second service system can also send a data synchronization request to the first service system at a regular time, and the second service system can return each data record and the current version information of the data set to the first service system after receiving the data synchronization request.

[0103] It should be noted that the above steps S401 to S403 are not limited to the execution order as shown in Figure 4 for example, the above steps S401 to S403 can also be executed after step S103.

[0104] In some embodiments, in the process of adding data records to the data set, the attribute value of each newly added data record is monotonically increased relative to the attribute value of the previous newly added data record. The above step S402 can be implemented by the following step S421, and the above step S102 can be implemented by the following step S422.

[0105] In step S421, the maximum value among the attribute values of the data records is determined as the current version information of the data set.

[0106] Here, the attribute value of each newly added data record is monotonically increasing relative to the attribute value of the previous newly added data record, and the maximum value among the attribute values of the data records is determined as the current version information of the data set. In this way, the attribute value of each data record in the current data set does not exceed the current version information of the data set, and the attribute value of the subsequently added data record exceeds the version information, so that the version information can distinguish each data record currently included in the data set from the subsequently added data record.

[0107] In step S422, the data record in the data set used by the first service system and having an attribute value not exceeding the version information is determined as the first data record to be processed matching the version information.

[0108] Here, the data record in the data set used by the first service system and having an attribute value not exceeding the version information is the data record already existing in the data set used by the first service system at the last time of data record synchronization of the second service system, that is, the first data record to be processed matching the version information.

[0109] In some embodiments, the attribute value of the data record includes the generation time of the data record or the self-incremented identifier of the data record in the data set. For example, the attribute value can be a timestamp representing the generation time of the data record or a self-incremented index ID of the data record in the database, etc.

[0110] In the embodiments of the present application, each data record in the data set has an attribute value, and in the process of adding data records to the data set, the attribute value of each newly added data record is monotonically changed relative to the attribute value of the previous newly added data record. The current version information of the data set is determined based on the attribute value of each data record by obtaining each data record currently in the data set used by the first service system. In this way, the current version information of the data set can be determined simply and quickly, so that each data record and the version information can be synchronized to the second service system.

[0111] In some embodiments, referring to Figure 5 , Figure 5 is an optional flowchart of the data processing method provided by the embodiments of the present application, based on Figure 3 , the first service system includes a data processing service, and the second service system includes a business logic service. The above step S101 can be implemented by the following step S501. The following steps will be described in combination with each step, and the execution subject of the following steps can be the terminal or the first server described above.

[0112] In step S501, the data processing service receives a data processing request sent by the business logic service; the data processing logic in the data processing request is determined by the business logic service based on each data record in a data set to be processed used in the second service system.

[0113] Here, the first service system can include a data processing service, which can receive a data processing request and perform corresponding processing on data records in a data set used in the first service system based on the data processing request. In implementation, the data processing service can be an independently deployable application program or microservice, etc., or a thread of an application program in the first service system. Those skilled in the art can implement the data processing service in any suitable manner according to actual conditions, which is not limited here.

[0114] The second service system can include a business logic service, which can determine data processing logic for a data set to be processed based on each data record in the data set used in the second service system, generate a data processing request carrying version information of the data set to be processed and the data processing logic, and send the data processing request to the first service system. In implementation, the business logic service can be an independently deployable application program or microservice, etc., or a thread of an application program in the second service system. Those skilled in the art can implement the business logic service in any suitable manner according to actual conditions, which is not limited here.

[0115] The data processing logic in the data processing request can be logic for processing data records in the data set according to actual business requirements. In implementation, the business logic service can generate the data processing logic in a default manner based on each data record in the data set to be processed used in the second service system, or determine the data processing logic based on user-configured logic information and each data record in the data set to be processed used in the second service system.

[0116] In some embodiments, the first service system further includes a data access service, and the second service system further includes a data synchronization service. The method can further perform steps S511 to S513.

[0117] In step S511, the data access service acquires each current data record in the data set used in the first service system in a case where there is a newly added data record in the data set.

[0118] Here, the first service system can include a data access service, which can provide the second service system with data record access capability of the data set based on the version information. The data access service can acquire each data record in the data set currently in the case that there is a new data record in the data set used in the first service system. In implementation, the data access service can be an independently deployable application or microservice, or a thread of an application in the first service system, and any suitable implementation of the data access service can be adopted by those skilled in the art according to actual conditions, which is not limited herein.

[0119] In step S512, the data access service determines the version information of the data set currently based on the attribute value of each data record.

[0120] Here, step S512 corresponds to the aforementioned step S402, and the specific embodiments of step S402 can be referred to in implementation.

[0121] In step S513, the data access service sends each data record and the version information to the data synchronization service, so that the data synchronization service updates the data set used in the second service system by using each data record.

[0122] Here, the second service system can include a data synchronization service, which can synchronize the data record in the data set used in the first service system and the version information of the data set currently from the first service system through the data access service, and update the data set used in the second service system by using each data record. Each data record in the updated data set used in the second service system corresponds to the version information of the data set currently. In implementation, the data synchronization service can be an independently deployable application or microservice, or a thread of an application in the second service system, and any suitable implementation of the data synchronization service can be adopted by those skilled in the art according to actual conditions, which is not limited herein.

[0123] It should be noted that the above steps S511 to S513 can be executed before step S501, or after step S103, which is not limited herein.

[0124] In the embodiments of the present application, the first service system comprises a data processing service, the second service system comprises a business logic service, the data processing service receives a data processing request sent by the business logic service, and performs corresponding processing on the data records in the data set used by the first service system based on the data processing request. In this way, the first service system and the second service system can be functionally decoupled, so that the overall service performance of the first service system and the second service system can be improved to a certain extent, and the maintenance and update of the system can be facilitated. Further, the first service system further comprises a data access service for providing the second service system with data record access capability of the data set based on version information, and the second service system further comprises a data synchronization service for synchronizing the data records in the data set used by the first service system and the current version information of the data set from the first service system through the data access service, and updating the data set used by the second service system with each data record. In this way, the coupling between the service functions in the first service system and the second service system can be further reduced, so that the overall service performance of the first service system and the second service system can be further improved, and the maintenance and update of the system can be further facilitated.

[0125] In some embodiments, referring to Figure 6 , Figure 6 is an optional flowchart of a data processing method provided by the embodiments of the present application, based on Figure 3 , the method can further perform steps S601 to S602. The execution subject of the following steps can be the terminal or the first server described above.

[0126] In step S601, from the data set used by the first service system, a second data record to be processed that does not match the version information is determined.

[0127] Here, the second data record to be processed that does not match the version information refers to the data record newly added in the first service system after the second service system last synchronized the data records from the first service system. The version information of the data set to be processed used by the second service system is the version information of the data records in the data set last synchronized by the second service system from the first service system. Based on the version information, the data records already existing in the first service system at the time of synchronization and the data records newly added in the first service system after synchronization can be distinguished, that is, the second data record to be processed that does not match the version information can be determined.

[0128] In implementation, the version information of the to-be-processed data set used in the second service system can be used to filter the to-be-processed second data records that do not match the version information from the data set used in the first service system. For example, the first service system can store a mapping relationship between the version information of the data set and the data records, and by querying the mapping relationship, the to-be-processed first data records that match the version information can be determined, and then the to-be-processed second data records that do not match the version information from the data set used in the first service system can be determined. For another example, each data record of the data set can include an attribute value representing the version to which the data record belongs, and by comparing the attribute value of each data record of the data set with the version information, it can be determined whether each data record matches the version information.

[0129] In step S602, the second data records are processed based on the preset processing logic.

[0130] Here, the preset processing logic can be a logic for processing the second data records in the data set according to actual business requirements set by a user in advance, or can be a default processing logic of the first system. In implementation, the preset processing logic can be the same as or different from the processing logic of the first data records (i.e., the data processing logic carried in the data processing request).

[0131] It should be noted that the above steps S601 to S602 are not limited to the execution order shown in Figure 6 , for example, the above steps S601 to S602 can also be executed after step S102.

[0132] In the embodiments of the present application, the to-be-processed second data records that do not match the version information are determined from the data set used in the first service system, and the second data records are processed based on the preset processing logic. In this way, since the second data records are the newly added data records in the first service system after the second service system last time synchronizes the data records from the first service system, by presetting the processing logic for the newly added data records, the consistency of the processing logic for the newly added data records can be ensured, thereby further improving the reliability of processing the data records in the data set in the heterogeneous system, and further ensuring the consistency of the final result of the data processing logic.

[0133] Referring to Figure 7 , Figure 7 is an optional flowchart of the data processing method provided by the embodiments of the present application, which will be described below in combination with the steps shown in Figure 7 . The execution subject of the following steps can be the terminal or the second server described above.

[0134] In step S701, each data record in a to-be-processed data set used in the second service system and version information of the data set are acquired; wherein the data records in the data set are obtained from the first service system which is heterogeneous with the second service system.

[0135] Here, the version information of the to-be-processed data set used in the second service system can be obtained from the first service system at the same time when the data records in the data set are synchronized from the first service system, or can be determined by the second service system based on the data records currently included in the data set, which is not limited herein. In implementation, a data processing task can be configured in the second service system according to actual business requirements, the data processing task can be triggered at a regular time, and each data record in the to-be-processed data set used in the second service system and the version information of the data set can be acquired, and the data processing logic for the data set can be determined based on each data record in the data set. The second service system can also acquire each data record in the to-be-processed data set used in the second service system and the version information of the data set in response to a data processing operation of a user.

[0136] In some embodiments, the second service system can synchronize the data records in the data set used in the first service system to the data set used in the second service system at a regular time, and acquire the current version information of the data set used in the first service system.

[0137] In step S702, the data processing logic for the data set is determined based on each data record in the data set.

[0138] Here, the data processing logic for the data set can be a logic for processing the data records in the data set according to actual business requirements. In implementation, the second service system can generate a default data processing logic based on each data record in the to-be-processed data set used in the second service system, or can determine the data processing logic based on the logic information configured by a user and each data record in the to-be-processed data set used in the second service system in response to a data processing logic configuration operation of the user.

[0139] In step S703, the data processing request is determined based on the version information and the data processing logic of the data set, and the data processing request is sent to the first service system, so that the first service system determines the to-be-processed first data record matching the version information from the data set used in the first service system, and processes the first data record based on the data processing logic of the data set.

[0140] Here, the data processing request can carry version information and data processing logic of the data set. By sending the data processing request to the first service system, the first service system can determine, based on the version information and the data processing logic carried in the data processing request, a first data record to be processed that matches the version information from the data set used by the first service system, and process the first data record based on the data processing logic of the data set.

[0141] In some embodiments, each data record in the data set has an attribute value, and in the process of adding data records to the data set by the first service system, the attribute value of each newly added data record is monotonically changed relative to the attribute value of the previous newly added data record. The above step S701 can be implemented by the following steps S711 to S712: in step S711, obtaining each data record in the data set to be processed used in the second service system; in step S712, determining the version information of the data set based on the attribute value of each data record.

[0142] In some embodiments, in the process of adding data records to the data set by the first service system, the attribute value of each newly added data record is monotonically increasing relative to the attribute value of the previous newly added data record. The above step S712 can be implemented by the following step S7121: step S7121, determining the maximum value of the attribute value of each data record as the version information of the data set.

[0143] In some embodiments, the first service system includes a data processing service, and the second service system includes a business logic service. The above step S702 can be implemented by the following step S721: in step S721, the business logic service determines, based on each data record in the data set, data processing logic for the data set. The above step S703 can be implemented by the following steps S731 to S732: in step S731, the business logic service determines a data processing request based on the version information and the data processing logic of the data set; in step S732, the business logic service sends the data processing request to the data processing service, so that the data processing service determines, from the data set used by the first service system, a first data record to be processed that matches the version information, and processes the first data record based on the data processing logic of the data set.

[0144] In some embodiments, the first service system further comprises a data access service, and the second service system further comprises a data synchronization service. The method can further perform the following steps S741-S742: in step S741, the data synchronization service receives each data record in a data set used in the first service system and current version information of the data set sent by the data access service, the current version information of the data set being determined by the data access service based on the attribute value of each data record; in step S742, the data synchronization service updates each data record in the data set used in the second service system. Here, each data record can be updated into the data set used in the second service system in the manner of incremental update or full update.

[0145] In the embodiments of the present application, each data record in a data set to be processed used in the second service system and version information of the data set are obtained; wherein the data record in the data set is obtained from the first service system which is heterogeneous with the second service system; based on each data record in the data set, data processing logic for the data set is determined; based on the version information and the data processing logic of the data set, a data processing request is determined and sent to the first service system, so that the first service system determines the first data record to be processed matching the version information from the data set used in the first service system, and processes the first data record based on the data processing logic of the data set. In this way, since the version information of the data set to be processed is carried in the process of data processing logic interaction between the heterogeneous first service system and the second service system, the first data record to be processed in the data set used in the first service system is determined based on the version information, which can avoid the problem of inaccurate data processing caused by unsynchronized data between the first service system and the second service system, so that the processing logic of the data record in the data set can be reliably completed in the heterogeneous system, and the consistency of the final result of data processing logic is ensured.

[0146] In the following, an exemplary application of the embodiments of the present application in an actual application scenario will be described. Taking data interaction in a distributed heterogeneous system as an example, the embodiments of the present application provide a data processing method based on version management, which can be applied between distributed heterogeneous systems, so that the data processing logic between heterogeneous systems can be reliably completed, and the consistency of the final result is ensured.

[0147] Taking data interaction between the heterogeneous system A and the heterogeneous system B as an example, the method can realize data interaction between the heterogeneous system A and the heterogeneous system B, wherein the heterogeneous system A can provide service capability for a set business field, and has data dependency on the heterogeneous system B and needs to interact with the heterogeneous system B. The heterogeneous system B can provide basic service support for the heterogeneous system A, and a version management mechanism is implemented for data stored in the heterogeneous system B. Referring to Figure 8A , Figure 8A An implementation architecture diagram of the data processing method provided by the embodiment of the present application is shown in FIG. 8. The heterogeneous system A can include two service modules: a data synchronization service 811 and a business logic service 812. The data synchronization service 811 can be used to synchronize a data set with version information in the heterogeneous system B, and cache the synchronized data set with version information. The synchronization mode can be subscription data or pull data. The business logic service 812 can process logic related to a business level in the heterogeneous system A, such as rule invalid configuration, white list configuration, and the like. When implementing the logic related to the business level, the business logic service 812 can generate a data processing request for the heterogeneous system B, for example, a request for adding, deleting, or modifying a data record in the data set in the heterogeneous system B (such as adding, deleting, modifying, or invalid configuration of a rule data record), which carries version information of a current synchronized data set in the heterogeneous system A, so that the heterogeneous system B can correctly process data based on the version information. The heterogeneous system B can include two service modules: a data access service 821 and a data processing service 822. The data access service 821 can be used to provide data access capability based on version information to the heterogeneous system A by querying a database, and the provided access mode can be data push or passive pull data. The data processing service 822 can receive a data processing request with version information and perform related data processing. In addition, the data processing service 822 can also pre-agree with the heterogeneous system A a default processing logic for other data records in addition to the data record corresponding to the version information carried by the data processing request.

[0148] For the version management mechanism of the data stored in the heterogeneous system B, any attribute information with a monotone increasing property in the data attribute of each data record in the data set can be used to implement the version management mechanism of the data set. For example, the version information of the data set can be determined based on the generation time of each data record in the data set or based on the self-incrementing data ID of the data record.

[0149] In some embodiments, the version information of the data set can be determined based on the generation time of each data record in the data set. Since time has a monotone increasing property, when a batch of data records are inserted into the data set table, the maximum value of the generation time of each data record can be selected as the current version number of the data set.Figure 8B The data set of version 1 includes two data records data1 and data2, and the corresponding version number is the maximum of the generation times of data1 and data2, that is, 1613705790. The data set of version 2 adds two data records data3 and data4, and the corresponding version number is the maximum of the generation times of data1, data2, data3 and data4, that is, 16113705813. The version number can be used as version information in data interaction between two heterogeneous systems to ensure correct execution of data processing logic and consistency of data processing results.

[0150] In some embodiments, the version information of the data set can be determined based on the self-incremental data ID of each data record in the data set. Since the self-incremental data ID has the characteristic of monotonous increment without wraparound, when a batch of data records are inserted into the data set table, the maximum of the self-incremental data IDs of the data records can be selected as the current version number of the data set. See Figure 8C The data set of version 1 includes two data records data1 and data2, and the corresponding version number is the maximum of the self-incremental data IDs of data1 and data2, that is, 2. The data set of version 2 adds two data records data3 and data4, and the corresponding version number is the maximum of the self-incremental data IDs of data1, data2, data3 and data4, that is, 4.

[0151] See Figure 8D , Figure 8D An implementation flowchart of a data processing method provided by an embodiment of the present application is shown. As shown in Figure 8D The method takes determining the version information of the data set based on the generation time of each data record in the data set as an example, and includes the following steps S801 to S805:

[0152] Step S801, the heterogeneous system B synchronizes the latest data set to the heterogeneous system A;

[0153] Here, as shown in Figure 8D , it is assumed that when the data synchronization occurs, the data set of the heterogeneous system B includes two data records data1 and data2, and the data set is in version 1, and the version number is the maximum of the generation times of data1 and data2, that is, 1613705790. The heterogeneous system B can send the data set in the data set table to the data synchronization service 811 in the heterogeneous system A by carrying the version number through the data access service 821.

[0154] Step S802, the heterogeneous system A receives the data set of version 1;

[0155] Here, the heterogeneous system A can receive the data set of version 1 through the data synchronization service to realize data synchronization. After completing the data synchronization, the heterogeneous system A obtains the data set corresponding to version 1, that is, the data set of version number 1613705790, including data records data1 and data2.

[0156] In step S803, new data is added in the heterogeneous system B, and the version number of the data set is incremented.

[0157] Here, after the data synchronization is completed, the data set in the heterogeneous system B is changed, and two data data3 and data4 are added. At this time, the data set is in version 2, and the version number is the maximum value 16113705813 in the generation time of data1, data2, data3 and data4.

[0158] In step S804, the heterogeneous system A determines to perform data interaction based on the data set of version 1, and sends a data processing request to the heterogeneous system B.

[0159] Here, the heterogeneous system A can determine the data processing logic based on the data set of version 1 through the business logic service 812, and send a data processing request to the data processing service 822 of the heterogeneous system B. The data processing request can be initiated according to the business requirement, and the data processing logic for the data set can be included in the data processing request, such as setting the specified rule data to an invalid state. At this time, the version information in the data set used in the heterogeneous system is also carried in the parameter of the data processing request, which can include the version number 1613705790 corresponding to version 1.

[0160] In step S805, the heterogeneous system B processes the data records in the data set matched with the version information in the data processing request based on the data processing logic and the version information in the data processing request, and processes the new data by using the preset processing logic.

[0161] Here, the heterogeneous system B can receive the data processing request sent by the business logic service 812 through the data processing service 822, and process the data records in the data set that match the version information based on the data processing logic in the data processing request. The data processing service 822 can compare the version information of the data set carried in the data processing request with the generation time of each data record to determine the data records in the data set that match the version information. For example, the version information of the data set carried in the data processing request is the version number 1613705790 of version 1, and the data differences between version 2 and version 1 can be compared to filter out the new data records data3 and data4. The data data1 and data2 in the data set of version 1 can be processed based on the data processing logic in the data processing request. For the new data data3 and data4, a pre-set default processing action can be taken, such as setting the new rule data to be invalid by default, and supplementing the new rule data to the current invalid rule list. Assuming that the generation time of the data record data is represented as create_time, and the version number corresponding to version 1 is represented as v1_version_id, the new data in the data set can be filtered out using the following formula 1-1:

[0162] (1-1).

[0163] It should be noted that the heterogeneous system A described above can be implemented as the second service system described above, and the heterogeneous system B can be implemented as the first service system described above.

[0164] The data processing method provided by the embodiments of the present application can avoid the problem of inaccurate data processing caused by unsynchronized data between the first service system and the second service system, so that the processing logic of the data records in the data set can be reliably completed in the heterogeneous system, and the consistency of the final result of the data processing logic is ensured.

[0165] The following continues to describe an exemplary structure of the data processing apparatus 255 provided by the embodiments of the present application, which is implemented as a software module. In some embodiments, as shown in Figure 2A The software module stored in the data processing apparatus 255 of the memory 250 can include:

[0166] The first obtaining module 2551 is configured to obtain a data processing request of a second service system that is heterogeneous with a first service system, the data processing request including version information of a data set to be processed used in the second service system and data processing logic for the data set; wherein the data records in the data set are obtained from the first service system;

[0167] The first determining module 2552 is configured to determine, from a data set used by the first service system, a first data record to be processed that matches version information of the data set.

[0168] The first processing module 2553 is configured to process the first data record based on data processing logic of the data set.

[0169] In some embodiments, each data record in the data set has an attribute value, and in a process of adding data records to the data set, the attribute value of each newly added data record is monotonically changed relative to the attribute value of a previous newly added data record. The apparatus further includes a second obtaining module configured to obtain each data record in the data set currently used by the first service system; a second determining module configured to determine, based on the attribute value of each data record, current version information of the data set; and a first synchronizing module configured to synchronize each data record and the version information to the second service system.

[0170] In some embodiments, in the process of adding data records to the data set, the attribute value of each newly added data record is monotonically increased relative to the attribute value of a previous newly added data record. The second determining module is further configured to determine, as the current version information of the data set, a maximum value in the attribute values of the data records. The first determining module is further configured to determine, as the first data record to be processed that matches the version information, a data record in the data set used by the first service system and having an attribute value not exceeding the version information.

[0171] In some embodiments, the first service system includes a data processing service, and the second service system includes a business logic service. The first obtaining module is further configured to receive, by the data processing service, a data processing request sent by the business logic service. Data processing logic in the data processing request is determined by the business logic service based on each data record in a data set to be processed used by the second service system.

[0172] In some embodiments, the first service system further comprises a data access service, the second service system further comprises a data synchronization service, and the apparatus further comprises: a third obtaining module, configured to, in a case where the data access service determines that there is a newly added data record in a data set used in the first service system, obtain each data record in the data set at present; a third determining module, configured to determine, by the data access service, version information of the data set at present based on the attribute value of each data record; and a first sending module, configured to send each data record and the version information to the data synchronization service, so that the data synchronization service updates a data set used in the second service system by using each data record.

[0173] In some embodiments, the apparatus further comprises: a fourth determining module, configured to determine, from a data set used in the first service system, a second data record to be processed which does not match the version information; and a second processing module, configured to process the second data record based on preset processing logic.

[0174] The following continues to illustrate an exemplary structure of the data processing apparatus 355 provided by the embodiments of the present application, which is implemented as a software module. In some embodiments, as shown in Figure 2B The software module stored in the data processing apparatus 355 of the memory 350 can include:

[0175] A fourth obtaining module 3551, configured to obtain each data record in a data set to be processed used in a second service system and version information of the data set; wherein the data record in the data set is obtained from a first service system which is heterogeneous to the second service system;

[0176] A fifth determining module 3552, configured to determine, based on each data record in the data set, data processing logic for the data set;

[0177] A second sending module 3553, configured to determine, based on the version information and the data processing logic of the data set, a data processing request, and send the data processing request to the first service system, so that the first service system determines, from a data set used in the first service system, a first data record to be processed which matches the version information, and processes the first data record based on the data processing logic of the data set.

[0178] In some embodiments, the apparatus further comprises a second synchronization module, configured to synchronize, at a timing, a data record in a data set used in the first service system to a data set used in the second service system, and obtain version information of the data set used in the first service system at present.

[0179] In some embodiments, each data record in the data set has an attribute value, in a process in which the first service system adds data records to the data set, the attribute value of each newly added data record is monotonically changed relative to the attribute value of a previous newly added data record, and the fourth obtaining module is further configured to: obtain each data record in a data set to be processed used by the second service system; and determine version information of the data set based on the attribute value of each data record.

[0180] In some embodiments, in the process in which the first service system adds data records to the data set, the attribute value of each newly added data record is monotonically increased relative to the attribute value of a previous newly added data record, and the fourth obtaining module is further configured to: determine, as the version information of the data set, a maximum value in the attribute values of the data records.

[0181] In some embodiments, the first service system includes a data processing service, and the second service system includes a business logic service, and the fifth determining module is further configured to: determine, by the business logic service, data processing logic for the data set based on the data records in the data set; and the second sending module is further configured to: determine, by the business logic service, a data processing request based on the version information and the data processing logic of the data set; and send, by the business logic service, the data processing request to the data processing service, so that the data processing service determines, from the data set used by the first service system, first data records to be processed that match the version information, and processes the first data records based on the data processing logic of the data set.

[0182] In some embodiments, the first service system further includes a data access service, and the second service system further includes a data synchronization service, and the apparatus further includes: a receiving module configured to receive, by the data synchronization service, each data record in a data set used by the first service system and current version information of the data set sent by the data access service, the current version information of the data set being determined by the data access service based on the attribute value of each data record; and an updating module configured to update, by the data synchronization service, the data set used by the second service system using each data record.

[0183] The embodiment of the present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method provided in the embodiment of the present application.

[0184] The embodiment of the present application provides a computer readable storage medium storing executable instructions, wherein the executable instructions are stored, and when the executable instructions are executed by a processor, the processor executes the data processing method provided by the embodiment of the present application, for example, as shown in the following method. Figure 3 The method shown.

[0185] In some embodiments, the computer readable storage medium can be FRAM, ROM, PROM, EP-ROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM memory, etc.; and can also be various devices including one or any combination of the above storage.

[0186] In some embodiments, the executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or being deployed as modules, components, subroutines or other units suitable for use in a computing environment.

[0187] As an example, the executable instructions can but not necessarily correspond to files in a file system, can be stored in part of a file storing other programs or data, for example, stored in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program in question, or stored in multiple cooperative files (for example, files storing one or more modules, subroutines or code portions).

[0188] As an example, the executable instructions can be deployed to execute on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed at multiple sites and interconnected through a communication network.

[0189] In summary, through the embodiment of the present application, the problem of inaccurate data processing caused by unsynchronized data between the first service system and the second service system can be avoided, so that the processing logic of the data centralized data record can be reliably completed in the heterogeneous system, and the consistency of the final result of the data processing logic is ensured.

[0190] The above is only an embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement and improvement made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that, include: A data processing request is obtained from a second service system that is heterogeneous with the first service system. The data processing request includes version information of the dataset to be processed used in the second service system and data processing logic for the dataset. The data records in the dataset are synchronized from the first service system. Each data record in the dataset has an attribute value. During the process of adding a new data record to the dataset, the attribute value of each new data record is monotonically increasing relative to the attribute value of the previous new data record. The version information is determined based on the maximum value among the attribute values ​​of each data record in the dataset. Data records whose attribute values ​​in the dataset used by the first service system do not exceed the version information are identified as the first data records to be processed that match the version information; Based on the data processing logic of the dataset, the first data record is processed.

2. The method according to claim 1, characterized in that, The method further includes: Retrieve each current data record in the dataset used by the first service system; Based on the attribute value of each data record, determine the current version information of the dataset; Each of the data records and the version information is synchronized to the second service system.

3. The method according to claim 1, characterized in that, The method further includes: From the dataset used by the first service system, identify a second data record to be processed that does not match the version information; The second data record is processed based on the preset processing logic.

4. The method according to any one of claims 1 to 3, characterized in that, The first service system includes a data processing service, and the second service system includes a business logic service. The step of obtaining a data processing request from the second service system, which is heterogeneous with the first service system, includes: The data processing service receives a data processing request sent by the business logic service; the data processing logic in the data processing request is determined by the business logic service based on each data record in the dataset to be processed used in the second service system.

5. The method according to claim 4, characterized in that, The first service system further includes a data access service, the second service system further includes a data synchronization service, and the method further includes: If the data access service determines that there are newly added data records in the dataset used by the first service system, it retrieves each current data record in the dataset. The data access service determines the current version information of the dataset based on the attribute value of each data record; The data access service sends each data record and the version information to the data synchronization service, so that the data synchronization service uses each data record to update the dataset used in the second service system.

6. A data processing method, characterized in that, The method includes: The system obtains each data record in the dataset to be processed used in the second service system and the version information of the dataset; wherein the data records in the dataset are obtained synchronously from a first service system that is heterogeneous with the second service system, each data record in the dataset has an attribute value, and during the process of adding a new data record to the dataset, the attribute value of each new data record is monotonically increasing relative to the attribute value of the previous new data record, and the version information is determined based on the maximum value among the attribute values ​​of each data record in the dataset; Based on each data record in the dataset, determine the data processing logic for the dataset; Based on the version information and the data processing logic of the dataset, a data processing request is determined and sent to the first service system. This allows the first service system to identify data records in the dataset whose attribute values ​​do not exceed the version information as first data records to be processed that match the version information. Based on the data processing logic of the dataset, the first data record is then processed.

7. A data processing apparatus, characterized in that, include: The first acquisition module is used to acquire a data processing request from a second service system that is heterogeneous with the first service system. The data processing request includes version information of the dataset to be processed used in the second service system and data processing logic for the dataset. The data records in the dataset are obtained synchronously from the first service system. Each data record in the dataset has an attribute value. During the process of adding a new data record to the dataset, the attribute value of each new data record is monotonically increasing relative to the attribute value of the previous new data record. The version information is determined based on the maximum value among the attribute values ​​of each data record in the dataset. The first determining module is used to determine data records whose attribute values ​​in the dataset used by the first service system do not exceed the version information as first data records to be processed that match the version information of the dataset. The first processing module is used to process the first data record based on the data processing logic of the dataset.

8. The apparatus as claimed in claim 7, characterized in that, It also includes a second acquisition module, used for: Retrieve each current data record in the dataset used by the first service system; The second determining module is used to determine the current version information of the dataset based on the attribute value of each data record; The first synchronization module is used to synchronize each data record and the version information to the second service system.

9. The apparatus as claimed in claim 7, characterized in that, It also includes a fourth determining module, used for: From the dataset used by the first service system, identify a second data record to be processed that does not match the version information; The second processing module is used to process the second data record based on preset processing logic.

10. The apparatus according to any one of claims 7 to 9, characterized in that, The first service system includes a data processing service, the second service system includes a business logic service, and the first acquisition module is further configured to: The data processing service receives a data processing request sent by the business logic service; the data processing logic in the data processing request is determined by the business logic service based on each data record in the dataset to be processed used in the second service system.

11. The apparatus as claimed in claim 10, characterized in that, The first service system also includes a data access service, the second service system also includes a data synchronization service, and a third acquisition module, used for: If the data access service determines that there are newly added data records in the dataset used by the first service system, it retrieves each current data record in the dataset. The third determining module is used by the data access service to determine the current version information of the dataset based on the attribute value of each data record; The first sending module is used by the data access service to send each data record and the version information to the data synchronization service, so that the data synchronization service uses each data record to update the dataset used in the second service system.

12. A data processing device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method according to any one of claims 1 to 6.

13. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the method of any one of claims 1 to 6 when executed by a processor.

14. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images