Data processing method and device

In data transmission between multiple systems, remote sessions receive synchronization requests and generate messages, combined with message queues and thread pooling technology, data transmission timeliness and concurrency problems are solved, and efficient and reliable data synchronization processing is achieved.

CN119003649APending Publication Date: 2024-11-22BEIJING WODONG TIANJUN INFORMATION TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310582931.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-22
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

When data transmission between multiple systems is transmitted, the existing technology cannot meet the high timeliness and concurrency requirements of data requests at the same time. The serial processing logic causes the service interface processing to be delayed, and the asynchronous transmission technology cannot obtain processing results, which limits the business scenarios. application.

Method used

By receiving synchronization requests based on remote sessions, generating messages and placing them on message queues, using thread pool loops to perform write-to-database operations, ensuring that messages are consumed, and reallocating processing threads in case of failure, combining synchronization and asynchronous transmission techniques to improve Efficiency and reliability.

Benefits of technology

It realizes the accuracy and timeliness of data synchronization requests, automatically analyzes data sources, returns received results in a timely manner, improves processing efficiency, and uses the concurrency capabilities of asynchronous transmission technology to ensure the reliability and efficiency of synchronous processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003649A_ABST
    Figure CN119003649A_ABST
Patent Text Reader

Abstract

The invention discloses a data processing method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: receiving a synchronization request initiated by a specified end based on a remote session, obtaining corresponding first synchronization data, and generating a corresponding first message; in response to determining that the first message is generated, generating received prompt information and sending the received prompt information to the specified end; the method comprises the following steps: placing a first message in a message queue, putting the message queue into a thread pool, and circularly executing the following steps on each message in the thread pool until the message is consumed: distributing a processing thread for each message so as to write synchronous data corresponding to the message into a database, and responding to successful writing of the synchronous data corresponding to the message into the database, if so, consuming the message; and in response to the failure of writing the synchronous data corresponding to the message into the database, distributing the processing thread for the message again. Therefore, the embodiment of the invention can solve the technical problem of low efficiency of data transmission among multiple systems in the prior art.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data processing method and device. Background Art

[0002] At present, data synchronization between multiple systems is very common, providing users with convenient data transmission services between multiple systems.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] When users transmit data between different application systems, there is a common problem that the high timeliness requirements of data requests and the large number of concurrent requirements cannot be met at the same time. This is because for existing synchronous transmission technology, the serial processing logic will inevitably prolong the processing time of the business interface, thereby reducing the throughput of the business interface; on the other hand, for existing asynchronous transmission technology, most business systems call message queues for data transmission. Although it effectively enhances the throughput performance of the business interface, the caller of the message queue cannot know the processing results of the called party, and therefore cannot promote subsequent business processing based on the processing results, which greatly limits the applicable business scenarios. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a data processing method and device, which can solve the technical problem of low efficiency of data transmission between multiple systems.

[0006] To achieve the above-mentioned purpose, according to one aspect of an embodiment of the present invention, a data processing method is provided, comprising: receiving a synchronization request initiated by a designated end based on a remote session, obtaining corresponding first synchronization data, and generating a corresponding first message; in response to determining that a first message is generated, generating a received prompt message and sending it to the designated end; placing the first message in a message queue, and putting the message queue into a thread pool, and executing the following steps for each message loop in the thread pool until the message is consumed: assigning a processing thread to each message to write the synchronization data corresponding to the message into a database, and consuming the message in response to successfully writing the synchronization data corresponding to the message into the database; and reassigning a processing thread to the message in response to a failure to write the synchronization data corresponding to the message into the database.

[0007] Optionally, after generating the corresponding first message, the method further includes:

[0008] The first synchronization data is cached in the first storage area to obtain first cache data, and the first cache data is associated with the first message.

[0009] Optionally, writing the synchronization data corresponding to the message into a database includes:

[0010] Reading the synchronization data corresponding to the message, and retrieving the cache data associated with the message from the first storage area;

[0011] The synchronization data is reviewed according to the cache data, and in response to passing the review, the synchronization data is written into the database.

[0012] Optionally, assign a processing thread to each message, including:

[0013] Extracting corresponding feature identifiers based on the synchronization data corresponding to the message;

[0014] In response to the presence of a lock mark in the feature identifier, the message is removed from the thread pool, and feature identifier duplication prompt information is generated;

[0015] In response to the feature identifier not having a locked mark, a processing thread is allocated for the message, and the feature identifier is marked as locked.

[0016] Optionally, after consuming the message, the process includes:

[0017] Based on the synchronization data corresponding to the message, a corresponding feature identifier is obtained, and a locking mark of the feature identifier is removed.

[0018] Optionally, before writing the synchronization data corresponding to the message into the database, the method includes:

[0019] In response to determining that no data record matching the synchronization data exists in the second storage area.

[0020] Optionally, after writing the synchronization data corresponding to the message into the database, the following steps are performed:

[0021] A feature identifier corresponding to the synchronization data is extracted and recorded in the second storage area.

[0022] Optionally, before putting the message queue into the thread pool, include:

[0023] Monitor all command requests initiated by the specified end in the remote session,

[0024] In response to monitoring that the designated end initiates a synchronization request, corresponding second synchronization data is captured and a corresponding second message is generated to be placed in a message queue.

[0025] In addition, the present invention also provides a data processing device, including an acquisition module, which is used to receive a synchronization request initiated by a designated end based on a remote session, obtain the corresponding first synchronization data, and generate a corresponding first message; a feedback module, which is used to generate a received prompt information and send it to the designated end in response to determining that the first message is generated; a pooling module, which is used to place the first message in a message queue, and put the message queue into a thread pool, and execute the following steps for each message loop in the thread pool until the message is consumed; a processing module, which is used to assign a processing thread to each message to write the synchronization data corresponding to the message into a database, and in response to successfully writing the synchronization data corresponding to the message into the database, consume the message; in response to failing to write the synchronization data corresponding to the message into the database, reassign a processing thread to the message.

[0026] An embodiment of the above invention has the following advantages or beneficial effects: the present invention completes the processing of transmitting data synchronization requests based on synchronous transmission technology by receiving a synchronization request initiated by a designated end based on a remote session, obtaining the corresponding first synchronization data, and generating a corresponding first message, thereby effectively ensuring the accuracy and timeliness of the synchronization request, and automatically parsing the synchronization request to obtain the first synchronization data, thereby obtaining the data source for subsequent system updates; and, the present invention completes the process of returning the receiving result to the request sending end in a timely manner by generating a received prompt message and sending it to the designated end in response to determining that the first message is generated, so that the request sending end can continue to advance the corresponding business processing, thereby improving the processing efficiency as a whole; at the same time, the present invention places the first message in the message Queue, and put the message queue into the thread pool, execute the following steps for each message loop in the thread pool until the message is consumed, and utilize the asynchronous transmission technology to promote the business processing corresponding to the above-mentioned first synchronization data, so as to give full play to the advantages of the strong concurrent processing capability of the asynchronous transmission technology and further improve the overall processing efficiency of the present invention; in addition, the present invention allocates a processing thread to each message to write the synchronization data corresponding to the message into the database, consumes the message in response to the success of writing the synchronization data corresponding to the message into the database, and re-assigns a processing thread to the message in response to the failure of writing the synchronization data corresponding to the message into the database, effectively utilizing the retry mechanism of the message queue to provide a backup for the processing of each synchronization request to ensure the reliability of the synchronization processing of the present technical solution.

[0027] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0029] Figure 1is a schematic diagram of the main flow of the data processing method according to the first embodiment of the present invention;

[0030] Figure 2 is a schematic diagram of a synchronization request processing process according to an embodiment of the present invention;

[0031] Figure 3 is a schematic diagram of the main flow of a data processing method according to a second embodiment of the present invention;

[0032] Figure 4 is a schematic diagram of the main flow of a data processing method according to a third embodiment of the present invention;

[0033] Figure 5 is a schematic diagram of main modules of a data processing device according to a first embodiment of the present invention;

[0034] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0035] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0037] Figure 1 is a schematic diagram of the main flow of the data processing method according to the first embodiment of the present invention. Figure 1 As shown, the data processing method includes:

[0038] Step S101: obtaining a synchronization request initiated by a designated end based on a remote session, obtaining corresponding first synchronization data, and generating a corresponding first message.

[0039] In some embodiments, the service middle station may issue the synchronization request and the first synchronization data, and the processing end that obtains the synchronization request and performs data synchronization may be the corresponding service subsystem. Figure 2As shown in the synchronous request processing process diagram, the business middle station is responsible for performing corresponding business processing on the initial business data, such as processing order information (writing order), processing payment information (writing payment), etc., so as to obtain the first synchronous data according to the business processing result, that is, the super body order in the figure, and issue a corresponding synchronous request to update the data of the business subsystem based on the first synchronous data, so as to facilitate the customer to update the data through the business subsystem of this processing end ( Figure 2 The superbody order part below) is queried to obtain the updated business status. According to the above embodiment, this step asynchronously configures the data system for processing business and the data system for storing business data and connecting to customer needs, thereby effectively alleviating the computing and processing burden of the business middle-office system, so as to improve the efficiency of business processing by the business middle-office system to a certain extent, and ensure the timeliness of the business data queried by the customer, thereby enhancing the customer experience as a whole. In a further embodiment, the customer can also place a business order through the business subsystem of this processing end, and after the business subsystem receives the business order, it will forward it to the business middle station for processing, and after the business middle station completes the processing, it will synchronize the updated business data to the business subsystem for customer query, thereby further clarifying the processing division of the business middle station and the business subsystem.

[0040] In an embodiment, the synchronization request received by the processing end may be a remote procedure call request, and synchronization technology is used to transmit data between the designated processing end and the processing end, thereby ensuring the accuracy and timeliness of the instruction request issued by the designated processing end.

[0041] Step S102: in response to determining that the first message is generated, generating received prompt information and sending it to the designated end.

[0042] In the embodiment, after receiving the synchronization request sent by the designated processing end based on the synchronous transmission technology, the processing end generates a corresponding first message, that is, the message queue based on the asynchronous transmission technology stores and records the synchronization request and further transmits it; and after determining to generate the first message (that is, after determining to successfully receive the synchronization request and successfully parse the corresponding first synchronization data), the processing end generates a received prompt message and returns it to the designated processing end to clearly respond to the request initiated by the designated processing end. After processing in this step, the multi-system data transmission mechanism of the present invention combines the advantages of high accuracy and strong timeliness of the synchronous transmission technology and the advantages of large processing interface throughput and message queue retry mechanism of the asynchronous transmission technology. At the same time, it also overcomes the shortcomings of the existing asynchronous transmission technology (most of which use message queues) that there is no return value after calling, resulting in the caller being unable to know the transmission result and processing result, thereby improving the data transmission performance of the present technical solution and expanding the application scenario of the present technical solution, that is, it can be used for application scenarios where the caller needs or relies on the processing result of the called party to make the next business judgment.

[0043] In some embodiments, in order to timely back up the first synchronization data received from the designated processing end and generate a disaster recovery database to further improve the security performance of the system, the first synchronization data can be cached in the first storage area after the corresponding first message is generated, and the first cache data is obtained and associated with the first message. The first storage area is the above-mentioned disaster recovery database. In addition to being used for emergency data retrieval, the cache data in the first storage area can also be used for the subsequent review stage before writing the first synchronization data into the database, thereby enhancing the security and reliability of the data processing of the present invention.

[0044] Step S103, placing the first message in a message queue, and putting the message queue into a thread pool, and executing the following steps for each message loop in the thread pool until the message is consumed.

[0045] In an embodiment, the generated first message is put into a message queue, and the message queue is pooled so that the first message is correspondingly written into the database of the processing end through the assigned processing thread. After processing in this step, the transmission write library request and the write library processing are respectively configured into the message queue and the processing thread, so that the present invention can not only cope with high-concurrency write library processing, but also monitor and retry the processing result of each write library request throughout the process, that is, the message is marked as consumed only when and only when the consumer (corresponding processing thread) of each message successfully completes the corresponding write library processing, and the message that has not successfully completed the corresponding write library processing is retried to cover the bottom line, thereby achieving the technical purpose of ensuring that the write library processing can be completed for each message in the message queue.

[0046] In some embodiments, in order to overcome the impact of low-probability network abnormal events (such as all servers going offline or machine restarts causing service interruptions) on remote session transmission, it is also possible to monitor all instruction requests initiated by the designated end in the remote session before putting the message queue into the thread pool, and in response to monitoring the synchronization request initiated by the designated end, capture the corresponding second synchronization data, and generate the corresponding second message to be placed in the message queue. Through the processing of this step, it can be ensured that the corresponding synchronization request and the first synchronization data will not be lost even in the case of network abnormalities, and that the processing end can definitely receive the first synchronization data sent by the designated processing end, and encapsulate it into the message queue to pass it to the processing thread for library writing processing, thereby minimizing the risk of data transmission between multiple systems.

[0047] Step S104, assigning a processing thread to each message to write the synchronization data corresponding to the message into the database. In response to successful writing of the synchronization data corresponding to the message into the database, the message is consumed; in response to failure to write the synchronization data corresponding to the message into the database, a processing thread is reassigned to the message.

[0048] In some embodiments, in order to prevent service resources from being consumed for concurrent repeated synchronization requests (and correspondingly generated repeated messages), when assigning a processing thread to each message, the corresponding feature identifier can be extracted based on the synchronization data corresponding to the message; in response to the presence of a lock mark in the feature identifier, the message is removed from the thread pool, and a feature identifier duplication prompt message is generated; in response to the absence of a lock mark in the feature identifier, a processing thread is assigned to the message, and the feature identifier is marked as locked. For example, if the synchronization data is order data, the corresponding feature identifier can be an order number. Therefore, if there are multiple messages corresponding to synchronization requests for the same order number in the thread pool, by locking the corresponding order number after assigning the processing thread, it is possible to effectively prevent the repeated assignment of processing threads to each data to be synchronized at the same time, thereby effectively saving service resources and improving the data processing efficiency of the present invention. On the other hand, for the detected duplicate messages, after processing in this step, the duplicate messages can be cleaned out of the thread pool in a timely manner, and the corresponding error prompt message is generated, thereby achieving the technical purpose of maintaining the stability of thread pool processing.

[0049] In a further embodiment, in order to promptly unlock the feature identifier that has been processed to avoid affecting the data synchronization processing of the feature identifier in the next stage, after consuming the message, the corresponding feature identifier can be obtained based on the synchronization data corresponding to the message, and the lock mark of the feature identifier can be eliminated.

[0050] In some embodiments, in order to prevent the synchronization data from being repeatedly written into the database, a second storage area may be set to cache the feature identifiers corresponding to the order data that has been written into the database; therefore, before writing the synchronization data corresponding to the message into the database, it may be first determined that there is no data record matching the synchronization data in the second storage area; and after writing the synchronization data corresponding to the message into the database, the feature identifier corresponding to the synchronization data is extracted and recorded into the second storage area. After this step is processed, the idempotent check process before writing the synchronization data into the database is realized, thereby ensuring the non-repetitiveness of the data in the database.

[0051] In some embodiments, in order to confirm the accuracy of the data in the database, when the synchronous data corresponding to the message is written into the database, the synchronous data corresponding to the message can be read, and the cache data associated with the message can be retrieved from the first storage area; the synchronous data can be reviewed based on the cache data, and in response to passing the review, the synchronous data can be written into the database.

[0052] In the embodiment, after processing in this step, all synchronization data sent by the designated processing end can be written into the database accurately and without duplication, thereby completing efficient data synchronization processing between multiple systems. After the synchronization data is completed, it is synchronized to the corresponding ES cluster cache storage through binary log (binlog) technology for customer order query services.

[0053] Figure 3 is a schematic diagram of the main flow of a data processing method according to a second embodiment of the present invention, the data processing method comprising:

[0054] Step S301, based on the remote session, a synchronization request initiated by a designated end is obtained, corresponding first synchronization data is obtained, and a corresponding first message is generated and placed in a message queue.

[0055] Step S302: in response to determining that the first message is generated, generating received prompt information and sending it to the designated end.

[0056] Step S303: cache the first synchronization data to the first storage area, obtain the first cache data, and associate it with the first message.

[0057] Step S304: monitor all instruction requests initiated by the designated end in the remote session.

[0058] Step S305, in response to monitoring that the designated end initiates a synchronization request, the corresponding second synchronization data is captured and a corresponding second message is generated to be placed in a message queue.

[0059] Step S306, putting the message queue into the thread pool, and executing the following steps for each message loop in the thread pool until the message is consumed.

[0060] Step S307: extracting corresponding feature identifiers based on the synchronization data corresponding to the message.

[0061] Step S308, determining whether the feature identifier has a locking mark, if so, executing step S309, if not, executing step S310.

[0062] Step S309: remove the message from the thread pool and generate feature identification duplication prompt information.

[0063] Step S310: assign a processing thread to the message and mark the feature identifier as locked.

[0064] Step S311, in response to determining that there is no data record matching the synchronization data in the second storage area.

[0065] Step S312, reading the synchronization data corresponding to the message, and retrieving the cache data associated with the message from the first storage area to review the synchronization data.

[0066] Step S313, in response to determining that the synchronization data has passed the review.

[0067] Step S314: writing the synchronization data into a database.

[0068] Step S315, determine whether the data is written into the database successfully, if so, execute step S316, if not, execute step S317.

[0069] Step S316: consume the message, extract the feature identifier corresponding to the synchronization data, and record it in the second storage area.

[0070] Step S317, clearing the locking mark of the feature identifier.

[0071] Figure 4 is a schematic diagram of the main flow of a data processing method according to a third embodiment of the present invention, the data processing method comprising:

[0072] Step S401, based on the remote session, a synchronization request initiated by a designated end is obtained, corresponding first synchronization data is obtained, and a corresponding first message is generated and placed in a message queue.

[0073] Step S402: in response to determining that the first message is generated, generating received prompt information and sending it to the designated end.

[0074] Step S403, putting the message queue into the thread pool, and executing the following steps for each message loop in the thread pool until the message is consumed.

[0075] Step S404: extracting corresponding feature identifiers based on the synchronization data corresponding to the message.

[0076] Step S405, determining whether the feature identifier has a locking mark, if yes, executing step S406, if no, executing step S407.

[0077] Step S406: remove the message from the thread pool and generate feature identification duplication prompt information.

[0078] Step S407: assign a processing thread to the message and mark the feature identifier as locked.

[0079] Step S408: In response to determining that there is no data record matching the synchronization data in the second storage area.

[0080] Step S409: writing the synchronization data into a database.

[0081] Step S410, determine whether the data is written into the database successfully, if so, execute step S411, if not, execute step S412.

[0082] Step S411: consume the message, extract the feature identifier corresponding to the synchronization data, and record it in the second storage area.

[0083] Step S412, clearing the locking mark of the feature identifier.

[0084] Figure 5 is a schematic diagram of main modules of a data processing device according to an embodiment of the present invention, such as Figure 5 As shown, the data processing device 500 includes an acquisition module 501, a feedback module 502, a pooling module 503 and a processing module 504. Among them, the acquisition module 501 acquires the synchronization request initiated by the designated end based on the remote session, acquires the corresponding first synchronization data, and generates the corresponding first message; the feedback module 502 generates a received prompt message in response to determining that the first message is generated and sends it to the designated end; the pooling module 503 puts the first message in the message queue, and puts the message queue into the thread pool, and performs the following steps for each message loop in the thread pool until the message is consumed; and the processing module 504 allocates a processing thread to each message to write the synchronization data corresponding to the message into the database, and consumes the message in response to the success of writing the synchronization data corresponding to the message into the database; in response to the failure of writing the synchronization data corresponding to the message into the database, the processing thread is re-allocated to the message.

[0085] In some embodiments, the acquisition module 501 is further used to: after generating the corresponding first message, cache the first synchronization data to the first storage area to obtain the first cache data and associate it with the first message.

[0086] In some embodiments, the processing module 504 is also used to: when writing the synchronization data corresponding to the message into the database, read the synchronization data corresponding to the message and retrieve the cache data associated with the message from the first storage area; review the synchronization data according to the cache data, and write the synchronization data into the database in response to passing the review.

[0087] In some embodiments, the processing module 504 is also used to: when assigning a processing thread to each message, extract the corresponding feature identifier based on the synchronization data corresponding to the message; in response to the presence of a lock mark on the feature identifier, remove the message from the thread pool and generate a feature identifier duplicate prompt message; in response to the absence of a lock mark on the feature identifier, assign a processing thread to the message and mark the feature identifier as locked.

[0088] In some embodiments, the processing module 504 is further used to: after consuming the message, based on the synchronization data corresponding to the message, obtain a corresponding feature identifier and remove a locking mark of the feature identifier.

[0089] In some embodiments, the processing module 504 is further used for: before writing the synchronization data corresponding to the message into the database, in response to determining that there is no data record matching the synchronization data in the second storage area.

[0090] In some embodiments, the processing module 504 is further used to: after writing the synchronization data corresponding to the message into the database, extract the feature identifier corresponding to the synchronization data, and record it in the second storage area.

[0091] In some embodiments, the acquisition module 501 is also used to: before putting the message queue into the thread pool, monitor all instruction requests initiated by the designated end in the remote session, and in response to monitoring the synchronization request initiated by the designated end, capture the corresponding second synchronization data and generate a corresponding second message to place it in the message queue.

[0092] It should be noted that the data processing method and the data processing device described in the present invention have a corresponding relationship in terms of specific implementation contents, so the repeated contents will not be described again.

[0093] Figure 6 An exemplary system architecture 600 is shown to which the data processing method or data processing device according to the embodiment of the present invention can be applied.

[0094] like Figure 6As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0095] The user can use the terminal devices 601, 602, 603 to interact with the server 605 through the network 604 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 601, 602, 603.

[0096] The terminal devices 601, 602, and 603 may be various electronic devices having a page display processing screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0097] The server 605 may be a server that provides various services, such as a background management server (only an example) that provides support for users to use the terminal devices 601, 602, and 603. The background management server may analyze and process the received product information query request and other data, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.

[0098] It should be noted that the data processing method provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, the computing device is generally arranged in the server 605 .

[0099] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0100] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of a terminal device suitable for implementing an embodiment of the present invention. Figure 7 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0101] like Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the computer system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0102] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal page display processor (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0103] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0104] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0105] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0106] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes an acquisition module, a feedback module, a pooling module and a processing module. The names of these modules do not, in some cases, constitute limitations on the modules themselves.

[0107] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device includes receiving a synchronization request initiated by a designated end based on a remote session, obtaining the corresponding first synchronization data, and generating a corresponding first message; in response to determining that the first message is generated, generating a received prompt message and sending it to the designated end; placing the first message in a message queue, and putting the message queue into a thread pool, and performing the following steps for each message loop in the thread pool until the message is consumed: assigning a processing thread to each message to write the synchronization data corresponding to the message into a database, consuming the message in response to the success of writing the synchronization data corresponding to the message into the database; in response to the failure of writing the synchronization data corresponding to the message into the database, reassigning a processing thread to the message. According to the technical solution of the embodiment of the present invention, the technical problem of low efficiency of data transmission between existing multiple systems can be solved.

[0108] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data processing method, characterized in that: include: Receiving a synchronization request initiated by a designated end based on a remote session, obtaining corresponding first synchronization data, and generating a corresponding first message; In response to determining that the first message is generated, generating received prompt information and sending it to the designated end; The first message is placed in a message queue, and the message queue is put into a thread pool. The following steps are performed for each message loop in the thread pool until the message is consumed: A processing thread is assigned to each message to write the synchronization data corresponding to the message into a database; in response to successful writing of the synchronization data corresponding to the message into the database, the message is consumed; in response to failure to write the synchronization data corresponding to the message into the database, a processing thread is reassigned to the message.

2. The method according to claim 1, characterized in that After the corresponding first message is generated, it includes: The first synchronization data is cached in the first storage area to obtain first cache data, and the first cache data is associated with the first message.

3. The method according to claim 2, characterized in that Writing the synchronization data corresponding to the message into the database includes: Reading the synchronization data corresponding to the message, and retrieving the cache data associated with the message from the first storage area; The synchronization data is reviewed according to the cache data, and in response to passing the review, the synchronization data is written into the database.

4. The method according to claim 1, characterized in that: Assign a processing thread to each message, including: Extracting corresponding feature identifiers based on the synchronization data corresponding to the message; In response to the presence of a lock mark in the feature identifier, the message is removed from the thread pool and feature identifier duplication prompt information is generated. In response to the feature identifier not having a locked mark, a processing thread is allocated for the message, and the feature identifier is marked as locked.

5. The method according to claim 4, characterized in that After consuming the message, it includes: Based on the synchronization data corresponding to the message, a corresponding feature identifier is obtained, and a locking mark of the feature identifier is removed.

6. The method according to claim 1, characterized in that Before writing the synchronization data corresponding to the message into the database, it includes: In response to determining that no data record matching the synchronization data exists in the second storage area.

7. The method according to claim 6, characterized in that After the synchronization data corresponding to the message is written into the database, it includes: A feature identifier corresponding to the synchronization data is extracted and recorded in the second storage area.

8. The method according to any one of claims 1 to 7, characterized in that: Before putting the message queue into the thread pool, include: Monitor all command requests initiated by the specified end in the remote session, In response to monitoring that the designated end initiates a synchronization request, corresponding second synchronization data is captured and a corresponding second message is generated to be placed in a message queue.

9. A data processing device, characterized in that: include: An acquisition module, configured to receive a synchronization request initiated by a designated end based on a remote session, acquire corresponding first synchronization data, and generate a corresponding first message; A feedback module, configured to generate received prompt information and send it to a designated end in response to determining that the first message is generated; A pooling module, used for placing the first message in a message queue, and putting the message queue into a thread pool, and executing the following steps for each message loop in the thread pool until the message is consumed; A processing module is used to assign a processing thread to each message to write the synchronization data corresponding to the message into a database. In response to the synchronization data corresponding to the message being successfully written into the database, the message is consumed; in response to the synchronization data corresponding to the message being failed to be written into the database, a processing thread is reassigned to the message.

10. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.