Data processing method and device and electronic equipment
By directly reading and writing inventory data on the target memory node of the memory database, and using data sharding and fallback mechanisms, the problems of low efficiency and consistency in the existing technology are solved, and efficient and reliable inventory data processing is achieved.
Patent Information
- Application Number
- CN202411714181.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-27
- Publication Date
- 2025-02-28
AI Technical Summary
The prior art lacks efficient and reliable methods in the storage management of inventory data, resulting in low reading and writing efficiency of inventory data and difficult to ensure timely update and consistency of data.
By directly reading and writing inventory data on the target memory node of the memory database, and routing data operation requests to the corresponding memory nodes in the data sharding method, dispersing the pressure of the memory database. At the same time, when the data operation result returned by the target memory node is not received, a data operation fallback message is sent to ensure the accuracy and consistency of data in a distributed environment.
It improves the reading and writing efficiency of inventory data, enhances the throughput and real-time of the memory management system, and ensures that the data can be rolled back to the state before the operation when the operation fails due to network exceptions or other reasons, and ensures the accuracy and consistency of the data.
Smart Images
Figure CN119536659A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, device and electronic device. Background Art
[0002] Inventory data refers to a collection of data related to the current available status and quantity of goods, accounts or other resources. It is usually stored in a database and used to track and manage various resources such as inventory, cash balances, and material reserves.
[0003] Traditional inventory data management methods include using a disk database to store and read and write inventory data; or using an in-memory database as a cache to read, write and update inventory data.
[0004] When using a disk database to store inventory data, the persistence of inventory data storage can be guaranteed, and inventory data is not easily lost due to power outages, but the disk has a low reading and writing speed for inventory data; although using a memory database to read and write inventory data can increase the reading and writing speed of inventory data, due to the volatility of memory, when using a memory database to read inventory data, it is still necessary to write inventory data to disk when inventory data changes, in order to prepare for the inventory data to be restored after the memory database is powered off. In this way, the high-speed reading and writing capabilities of the memory cannot be fully utilized in the process of reading and writing inventory data.
[0005] Currently, there is no efficient and reliable method for the storage and management of inventory data. Summary of the invention
[0006] The present invention provides a data processing method, device and electronic device to improve the efficiency of processing inventory data and ensure the timeliness of inventory data update. The specific technical solution is as follows: In a first aspect, the present application provides a data processing method, comprising: Receiving a data operation request sent by a target object, wherein the data operation request includes inventory data to be operated; After sending a data operation start message to the message queue, based on the association relationship between the data identifier of the inventory data and the memory node, determining the target memory node corresponding to the inventory data, and forwarding the data operation request to the target memory node; Determining whether a data operation result returned by the target memory node is received; If yes, sending a data operation success message to the message queue and the target object; If not, a data operation rollback message is sent to the message queue, and a data operation failure message is returned to the target object.
[0007] The present application can improve the reading and writing efficiency of inventory data by directly reading and writing inventory data in the target memory node of the memory database; by routing data operation requests to the target memory node corresponding to the inventory data through data sharding, the pressure of the memory database can be dispersed, and the throughput and real-time performance of the memory management system can be improved; by sending a data operation rollback message when the data operation result returned by the target memory node is not received, it can be ensured that in a distributed environment, the inventory data after an operation fails due to abnormal network interruption or other reasons can be rolled back to the state before the operation, thereby ensuring the accuracy and consistency of the data.
[0008] In a possible implementation, after forwarding the data operation request to the target memory node, the method further includes: Query whether there is a rollback log through the operation identifier corresponding to the data operation request; If yes, return the data operation rollback message; If not, query whether there is an operation log through the operation identifier. When it is determined that there is no operation log, adjust the relevant information of the inventory data, record the corresponding adjustment log, and return the data operation success message.
[0009] Based on the above method, the pressure on the memory database can be dispersed and the throughput and real-time performance of the memory management system can be improved.
[0010] In a possible implementation, when the data operation start message is consumed from the message queue, the method further includes: Parsing the data operation start message to obtain the operation identifier and operation content corresponding to the data operation request; The operation identifier and the operation content are stored in a disk database according to a preset format, and the operation identifier is added to a list of pending operation requests in the disk database.
[0011] Based on the above method, the persistence and consistency of inventory data storage can be guaranteed.
[0012] In a possible implementation, when the data operation success message is consumed from the message queue, the method further includes: Determine whether the current version number of the inventory data in the data operation success message is greater than the original version number of the inventory data in the disk database; If so, synchronizing the data operation result in the data operation success message to the disk database, recording a data operation success log in the disk database, and removing the operation identifier corresponding to the data operation request from the list of pending operation requests in the disk database; If not, a data operation success log is recorded in the disk database, and the operation identifier corresponding to the data operation request is removed from the list of pending operation requests in the disk database.
[0013] Based on the above method, after the inventory data in the memory database is updated, the corresponding inventory data in the disk can be synchronously updated.
[0014] In a possible implementation, when the data operation rollback message is consumed from the message queue, the method further includes: Rolling back the relevant information of the inventory data based on the rollback identifier corresponding to the data operation rollback message; Determine whether the current version number of the inventory data in the rollback success message returned after the rollback is successful is greater than the original version number of the inventory data in the disk database; If yes, the data operation result in the rollback success message is synchronized to the disk database, a data operation rollback log is recorded in the disk database, and the operation identifier corresponding to the data operation request is removed from the list of pending operation requests in the disk database; If not, the data operation rollback log is recorded in the disk database, and the operation identifier corresponding to the data operation request is removed from the list of pending operation requests in the disk database.
[0015] Based on the above method, it can be ensured that in a distributed environment, the inventory data can be rolled back to the state before the operation after an operation fails due to abnormal network interruption or other reasons, thereby ensuring the accuracy and ultimate consistency of the data.
[0016] In a possible implementation, after adding the operation identifier to the list of pending operation requests of the disk database, the method further includes: Scan the list of pending operation requests according to a set time period to determine the operation duration of the inventory data in the list of pending operation requests; When it is determined that the operation duration is greater than the set duration, the relevant information of the inventory data is rolled back based on the operation identifier; Determine whether the current version number of the inventory data in the rollback success message returned after the rollback is successful is greater than the original version number of the inventory data in the disk database; If yes, the data operation result in the rollback success message is synchronized to the disk database, a data operation rollback log is recorded in the disk database, and the operation identifier corresponding to the data operation request is removed from the list of pending operation requests in the disk database; If not, the data operation rollback log is recorded in the disk database, and the operation identifier corresponding to the data operation request is removed from the list of pending operation requests in the disk database.
[0017] Based on the above method, it can be ensured that the inventory data is operated accordingly within the set time range to ensure the efficiency of system operation.
[0018] In a possible implementation, the step of rolling back the relevant information of the inventory data based on the rollback identifier corresponding to the data operation rollback message includes: By using the rollback identifier, query whether there is a rollback log in the memory database; If yes, return the rollback success message; If not, the rollback identifier is used to query whether there is an operation log in the memory database. When it is determined that the operation log exists, the relevant information of the inventory data is rolled back, the rollback log is recorded, and the rollback success message is returned.
[0019] Based on the above method, it can be ensured that in a distributed environment, the inventory data can be rolled back to the state before the operation after an operation fails due to abnormal network interruption or other reasons, thereby ensuring the accuracy and ultimate consistency of the data.
[0020] In a second aspect, the present application provides a data processing device, comprising: A data receiving module, used to receive a data operation request sent by a target object, wherein the data operation request includes inventory data to be operated; A data forwarding module, configured to determine a target memory node corresponding to the inventory data based on an association relationship between a data identifier of the inventory data and a memory node after sending a data operation start message to a message queue, and forward the data operation request to the target memory node; A determination module, used to determine whether a data operation result returned by the target memory node is received; If yes, sending a data operation success message to the message queue and the target object; If not, a data operation rollback message is sent to the message queue, and a data operation failure message is returned to the target object.
[0021] In a third aspect, the present application provides an electronic device, including: Memory, used to store computer programs; The processor is used to implement the steps of the above-mentioned data processing method when executing the computer program stored in the memory.
[0022] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned data processing method are implemented.
[0023] For each aspect from the second to the fourth aspect and the technical effects that may be achieved by each aspect, please refer to the above description of the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 A flowchart of a data processing method provided for this application; Figure 2 A schematic diagram of the data processing system architecture provided for this application; Figure 3 A flowchart of the inventory operation client module provided by this application processing data operation requests; Figure 4 A flowchart of a target memory node processing a data operation request provided by the present application; Figure 5 A flowchart of the inventory operation synchronization module provided by the present application for processing a data operation start message; Figure 6 A flowchart of the inventory operation synchronization module provided in this application processing a data operation success message; Figure 7 A flowchart of the inventory operation synchronization module provided in this application for processing data operation rollback messages; Figure 8 Flowchart of processing data rollback request for the target memory node provided by this application; Fig. 9 The flowchart of the scheduled task processing of the inventory operation synchronization module provided for this application; Fig.10 A schematic diagram of the structure of a data processing device provided in this application; Fig.11 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION
[0025] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The specific operating methods in the method embodiments can also be applied to device embodiments or system embodiments. It should be noted that in the description of the present application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A is connected to B, which can represent: A is directly connected to B and A is connected to B through C. In addition, in the description of the present application, words such as "first" and "second" are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.
[0026] The embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0027] Inventory data refers to a collection of data related to the current available status and quantity of goods, accounts or other resources. It is usually stored in a database and used to track and manage various resources such as inventory, cash balances, and material reserves.
[0028] Traditional inventory data management methods include using a disk database to store and read and write inventory data; or using an in-memory database as a cache to read, write and update inventory data.
[0029] When using a disk database to store inventory data, the persistence of inventory data storage can be guaranteed, and inventory data is not easily lost due to power outages, but the disk has a low reading and writing speed for inventory data; although using a memory database to read and write inventory data can increase the reading and writing speed of inventory data, due to the volatility of memory, when using a memory database to read inventory data, it is still necessary to write inventory data to disk when inventory data changes, in order to prepare for the inventory data to be restored after the memory database is powered off. In this way, the high-speed reading and writing capabilities of the memory cannot be fully utilized in the process of reading and writing inventory data.
[0030] Currently, there is no efficient and reliable method for the storage and management of inventory data.
[0031] In view of this, in order to improve the efficiency of processing inventory data and ensure the timeliness of inventory data updating, the present application provides a data processing method, which specifically includes: first receiving a data operation request sent by a target object; after sending a data operation start message to a message queue, based on the association between the data identifier of the inventory data and the memory node, determining the target memory node corresponding to the inventory data, and forwarding the data operation request to the target memory node; then determining whether the data operation result returned by the target memory node is received; if so, sending a data operation success message to the message queue and the target object; if not, sending a data operation rollback message to the message queue, and returning a data operation failure message to the target object.
[0032] The present application can improve the reading and writing efficiency of inventory data by directly reading and writing inventory data in the target memory node of the memory database; by routing data operation requests to the target memory node corresponding to the inventory data through data sharding, the pressure of the memory database can be dispersed, and the throughput and real-time performance of the memory management system can be improved; by sending a data operation rollback message when the data operation result returned by the target memory node is not received, it can be ensured that in a distributed environment, the inventory data after an operation fails due to an abnormal network interruption or other reasons can be rolled back to the state before the operation, thereby ensuring the accuracy and consistency of the data.
[0033] Reference Figure 1 As shown, it is a flow chart of a data processing method provided in an embodiment of the present application, the method comprising: S1, receiving a data operation request sent by a target object, wherein the data operation request includes inventory data to be operated.
[0034] First of all, the method provided in this application can be applied to Figure 2 In the system architecture shown, the system architecture includes: a target object and an inventory data management device, and the method provided in the present application can be executed in the inventory data management device.
[0035] The target object can be any type of terminal device with communication function, including mobile and handheld terminals. For example, tablet computers, smart phones, etc. The target object can send various data operation requests related to inventory data to the inventory data management device through wireless or wired networks. For example, inventory data query, inventory data update request, etc.
[0036] The inventory data management device includes an inventory operation client module, a memory data management module, and an inventory operation synchronization module. The functions of each of the above modules are introduced below: The inventory operation client module is used to provide an external interface for reading, adding, and deducting inventory data.
[0037] The memory data management module runs on each memory node of the memory database, and is used to receive the data operation request sent by the inventory operation client module and complete the operation content in the data operation request in one atomic operation.
[0038] In the embodiment of the present application, a remote dictionary service (Redis) can be used as a memory database, and its atomic operations can be implemented through Redis's Lua script. Optionally, the memory database can also be a memory database such as a distributed cache system (Memcached), or a memory data model implemented by the memory data management module itself, using distributed locks or other methods to implement the operation content of the data operation request in one atomic operation in the application code.
[0039] The inventory operation synchronization module is used to receive the data synchronization message sent by the inventory operation client module and process the message accordingly, including processing the data operation start message, data operation success message and data operation rollback message. At the same time, a timer task can be started in the inventory operation synchronization module to monitor and roll back the data operation that has timed out and is not completed.
[0040] In the embodiment of the present application, the inventory data management device also includes a message queue, which can be a distributed message queue RocketMQ. Optionally, the message queue can also be other message queue middleware such as Kafka, RabbitMQ, etc., which can be used to realize the production and consumption of messages.
[0041] In an embodiment of the present application, the inventory operation client module and the inventory operation synchronization module included in the inventory data management device are implemented through different code modules in the same process. Optionally, these two modules can also communicate through a standard interface in different processes.
[0042] In the embodiment of the present application, the target object may be an application running on the client, and the inventory operation client module may receive a data operation request or a data query request sent by the target object. It should be noted that the inventory operation client may simultaneously receive data operation requests sent by multiple target objects, and simultaneously perform batch processing on multiple data operation requests. The present application only takes one target object as an example for illustration, but does not limit the solution of the present application.
[0043] Exemplarily, when the inventory operation client module receives a data query request sent by the target object, it can determine the target memory node in the memory database corresponding to the inventory data based on the data identifier of the inventory data in the data query request (such as the inventory key); then send the data query request to the target memory node, and the target memory node returns the inventory data query result to the inventory operation client or the application running on the inventory operation client.
[0044] In an embodiment of the present application, the inventory data may be a certain commodity, and the inventory key may be the commodity code; in addition, the node number of the target memory node may be calculated by calculating the hash value of the commodity code and then taking the modulus of the number of memory nodes in the memory database, thereby determining the target memory node corresponding to the inventory key.
[0045] Through the above method, when the inventory operation client receives the data query request sent by the target object, it can forward the data query request to the target memory node, and the target memory node returns the corresponding inventory data query result to the target object.
[0046] The following is a detailed description of the data processing process of the inventory operation client module receiving the data operation request sent by the target object: S2, after sending a data operation start message to the message queue, determines the target memory node corresponding to the inventory data based on the association between the data identifier of the inventory data and the memory node, and forwards the data operation request to the target memory node.
[0047] In the embodiments of this application, see the attached Figure 3 As shown, when the inventory operation client module receives a data operation request, it can send a data operation start message to the message queue. The data operation start message includes the operation identifier of the inventory data and the inventory data list of the requested operation; if the inventory operation client module fails to send the data operation start message to the message queue, the data operation request fails and the operation ends; if the inventory operation client module successfully sends the data operation start message to the message queue, the inventory operation client module can determine the target memory node corresponding to the inventory data based on the association between the data identifier of the inventory data and the memory node, and forward the data operation request to the target memory node.
[0048] In a possible implementation, if there are multiple data operation requests, the inventory operation client module can group and merge the multiple data operation requests according to the data identifier of each inventory data in the multiple data operation requests; finally, the multiple data operation requests are sent to the target memory nodes corresponding to the data operation requests in the memory database for processing. This application does not impose specific restrictions on the number of target memory nodes.
[0049] By routing data operation requests to the target memory nodes corresponding to the inventory data through data sharding, the pressure on the memory database can be dispersed and the throughput and real-time performance of the memory management system can be improved.
[0050] In the embodiments of this application, see the attached Figure 4As shown, when the target memory node receives a data operation request, it can first use the operation identifier corresponding to the data operation request to query whether there is a rollback log in the memory; the operation identifier can be an identity identifier used to characterize the data operation request, for example, an operation number, and this application does not impose specific restrictions on this.
[0051] If a rollback log already exists, the target memory node can return a data operation rollback message to the inventory operation client module or the message queue. The data operation rollback message is used to indicate that the current data operation request has been revoked and the inventory data related to the data operation request has been rolled back to the previous state.
[0052] If there is no rollback log, the target memory node can further query whether there is an operation log in the memory through the operation identifier; after determining that there is an operation log in the memory, the target memory node can return a data operation success message to the inventory operation client module or the message queue. The data operation success message is used to indicate that the current data operation request has been completed.
[0053] When the target memory node determines that there is no operation log in the memory, the target memory node can adjust the relevant information of the inventory data according to the data operation request, record the corresponding adjustment log, and return a data operation success message. Exemplarily, the target memory node can modify the quantity of inventory data in the data operation request; after modifying the inventory data, the node version number of the current target memory node and the data version number of the inventory data are incremented (the version number is increased by a set value), and then the above modifications are recorded in the adjustment log of the memory database. The adjustment log can also record information such as inventory keys, operation numbers, and operation results. This application does not impose specific restrictions on this.
[0054] Through the above method, the inventory data can be changed accordingly in the target memory node of the memory database, thereby increasing the rate of operating the inventory data.
[0055] In the embodiment of the present application, in order to ensure the persistence and consistency of inventory data storage, see the attached Figure 5 As shown, when the inventory operation synchronization module consumes the data operation start message from the message queue, it can first parse the operation identifier and operation content corresponding to the data operation request from the data operation start message, and then store the operation identifier and operation content in the disk database according to a preset data format, and add the operation identifier to the list of pending operation requests in the disk database.
[0056] Through message synchronization, data operations of inventory data are synchronized to the disk database, ensuring the persistence and consistency of inventory data storage.
[0057] In the embodiments of this application, see the attached Figure 6As shown, when the inventory operation synchronization module consumes the data operation success message from the message queue, the inventory operation synchronization module can determine whether the current version number of the inventory data in the data operation success message is greater than the original version number of the inventory data in the disk database; if the current version number is greater than the original version number, it means that the inventory data has been updated accordingly. Therefore, the inventory operation synchronization module can synchronize the data operation result in the data operation success message to the disk database, record the data operation success log in the inventory operation log table of the disk database, and remove the operation identifier corresponding to the data operation request from the pending operation request list of the disk database. Among them, the data operation result includes the version number of the inventory data and the version number of the target memory node used to store the inventory data.
[0058] If the current version number is less than or equal to the original version number, the inventory operation synchronization module does not need to synchronize the data operation results to the disk database. It can directly record the data operation success log in the disk database and remove the operation identifier corresponding to the data operation request from the list of pending operation requests in the disk database.
[0059] Through the above method, after the inventory data in the memory database is updated, the corresponding inventory data in the disk can be synchronously updated.
[0060] S3, determine whether the data operation result returned by the target memory node is received, if so, execute step S31; if not, execute step S32.
[0061] S31, sending a data operation success message to the message queue and the target object.
[0062] In an embodiment of the present application, after receiving the data operation success message returned by the target memory node, the inventory operation client module can send the data operation success message to the message queue and the target object.
[0063] In one possible implementation, if there are multiple data operation requests, the inventory operation client module can collect the data operation success messages returned by multiple target memory nodes respectively, and after determining that each target memory node successfully returns the data operation success message, send the data operation success message to the message queue and the target object.
[0064] S32, sending a data operation rollback message to the message queue, and returning a data operation failure message to the target object.
[0065] In an embodiment of the present application, when the inventory operation client module determines that the data operation result returned by the target memory node has not been received, it can send a data operation rollback message to the message queue and return a data operation failure message to the target object.
[0066] See attached Figure 7As shown, when the inventory operation synchronization module consumes the data operation rollback message from the message queue, it can determine the target memory node according to the rollback identifier in the data operation rollback message, and send a data rollback request to the target memory node, so that the target memory node rolls back the relevant information of the inventory data, wherein the rollback identifier and the operation identifier can be the same identifier or different identifiers, and this application does not make specific restrictions on this. The specific method of inventory data rollback is as follows: See attached Figure 8 As shown, when the target memory node receives a data rollback request (data operation rollback message), it can query whether there is a rollback log in the memory database according to the rollback identifier corresponding to the data operation rollback message. If there is a rollback log, the target memory node can return a rollback success message to the data queue or inventory operation client; if there is no rollback log, the target memory node can further query whether there is an operation log in the memory database according to the rollback identifier. When it is determined that there is an operation log, the target memory node can roll back the relevant information of the inventory data, record the rollback log, and return a rollback success message. Among them, rolling back the relevant information of the inventory data includes performing reverse increase and decrease operations on the quantity of the inventory data according to the operation record in the operation log, and increasing the node version number of the target memory node and the data version number of the inventory data, recording the node version number and the data version number in the rollback log, and storing them in the memory database with the operation number as the key. As with the above-mentioned adjustment log, the rollback log also includes information such as the inventory key, the number of operations, and the operation result, which will not be repeated here.
[0067] like Figure 7 As shown, after the target memory node rolls back the relevant information of the inventory data, the inventory operation synchronization module can determine whether the rollback is successful. If the rollback is successful, the inventory operation synchronization module can further determine whether the current version number of the inventory data in the rollback success message (data operation rollback message) returned after the rollback is successful is greater than the original version number of the inventory data in the disk database; when it is determined that the current version number of the inventory data in the rollback success message is greater than the original version number of the inventory data in the disk database, the inventory operation synchronization module can synchronize the data operation result in the rollback success message to the disk database, record the data operation rollback log in the disk database, and remove the operation identifier corresponding to the data operation request from the list of pending operation requests in the disk database. This process is similar to the above-mentioned step of synchronizing the data operation result in the data operation success message to the disk database, and will not be repeated here.
[0068] When the inventory operation synchronization module determines that the current version number of the inventory data in the rollback success message is less than or equal to the original version number of the inventory data in the disk database, the inventory operation synchronization module can directly record the data operation rollback log in the disk database and remove the operation identifier corresponding to the data operation request from the list of pending operation requests in the disk database.
[0069] In an embodiment of the present application, if the rollback fails, the inventory operation synchronization module can increase the number of retries of the corresponding operation identifier in the list of pending operation requests, and determine whether the number of retries exceeds the specified number. If so, it notifies and requests manual intervention.
[0070] In the embodiment of the present application, in order to ensure that in a distributed environment, the inventory data after abnormal interruption or other reasons for operation failure can be rolled back to the state before the operation, the accuracy and consistency of the data are guaranteed. Fig. 9 As shown, the inventory operation synchronization module can start a scheduled task, regularly scan the list of pending operation requests, and determine the operation time of the inventory data in the list of pending operation requests; for the pending inventory data whose operation time exceeds the set time, based on the operation identifier of the inventory data, query the target memory node corresponding to the operation identifier of the pending inventory data, and request the target memory node to roll back the relevant information of the inventory data. The steps for the target memory node to roll back the relevant information of the inventory data are the same as above and will not be repeated here.
[0071] To summarize, the present application can improve the reading and writing efficiency of inventory data by directly reading and writing inventory data in the target memory node of the memory database; routing data operation requests to the target memory node corresponding to the inventory data by means of data sharding can disperse the pressure of the memory database and improve the throughput and real-time performance of the memory management system; by sending a data operation rollback message when the data operation result returned by the target memory node is not received, it can ensure that in a distributed environment, the inventory data after an operation fails due to an abnormal network interruption or other reasons can be rolled back to the state before the operation, thereby ensuring the accuracy and ultimate consistency of the data.
[0072] Based on the method provided in the above embodiment, the embodiment of the present application also provides a data processing device, such as Fig.10 The figure is a schematic diagram of the structure of a data processing device in an embodiment of the present application, the device comprising: The data receiving module 1001 is used to receive a data operation request sent by a target object, wherein the data operation request includes inventory data to be operated; The data forwarding module 1002 is used to determine the target memory node corresponding to the inventory data based on the association relationship between the data identifier of the inventory data and the memory node after sending the data operation start message to the message queue, and forward the data operation request to the target memory node; A determination module 1003 is used to determine whether a data operation result returned by the target memory node is received; If yes, sending a data operation success message to the message queue and the target object; If not, a data operation rollback message is sent to the message queue, and a data operation failure message is returned to the target object.
[0073] Based on the same inventive concept, an electronic device is also provided in the embodiment of the present application, and the electronic device can realize the functions of the aforementioned data processing device, referring to Fig.11 , the electronic device comprises: At least one processor 1101, and a memory 1102 connected to the at least one processor 1101. The specific connection medium between the processor 1101 and the memory 1102 is not limited in the embodiment of the present application. Fig.11 In the example, the processor 1101 and the memory 1102 are connected via the bus 1100. The bus 1100 is Fig.11 The bus 1100 is represented by a bold line, and the connection between other components is only for schematic illustration and is not intended to be limiting. The bus 1100 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 1101 can also be called a controller, and there is no limitation on the name.
[0074] In the embodiment of the present application, the memory 1102 stores instructions that can be executed by at least one processor 1101. The at least one processor 1101 can execute the data processing method discussed above by executing the instructions stored in the memory 1102. The processor 1101 can implement Fig.10 The functions of each module in the device shown.
[0075] Among them, the processor 1101 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 1102 and calling the data stored in the memory 1102, the various functions of the device and processing data, the device can be monitored as a whole.
[0076] In one possible design, the processor 1101 may include one or more processing units, and the processor 1101 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 1101. In some embodiments, the processor 1101 and the memory 1102 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.
[0077] Processor 1101 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the data processing method disclosed in the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.
[0078] Memory 1102, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory 1102 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic memory, disk, optical disk, etc. Memory 1102 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1102 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.
[0079] By designing and programming the processor 1101, the code corresponding to the data processing method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 1The steps of the data processing method of the embodiment shown are as follows: How to design and program the processor 1101 is a technique known to those skilled in the art and will not be described in detail here.
[0080] Based on the same inventive concept, an embodiment of the present application further provides a storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the data processing method discussed above.
[0081] In some possible implementations, various aspects of the data processing method provided by the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of the data processing method according to various exemplary implementations of the present application described above in this specification.
[0082] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0083] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0084] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0086] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A data processing method, characterized in that: include: Receiving a data operation request sent by a target object, wherein the data operation request includes inventory data to be operated; After sending a data operation start message to the message queue, based on the association relationship between the data identifier of the inventory data and the memory node, determining the target memory node corresponding to the inventory data, and forwarding the data operation request to the target memory node; Determining whether a data operation result returned by the target memory node is received; If yes, sending a data operation success message to the message queue and the target object; If not, a data operation rollback message is sent to the message queue, and a data operation failure message is returned to the target object.
2. The method according to claim 1, characterized in that After forwarding the data operation request to the target memory node, the method further includes: Query whether there is a rollback log through the operation identifier corresponding to the data operation request; If yes, return the data operation rollback message; If not, query whether there is an operation log through the operation identifier. When it is determined that there is no operation log, adjust the relevant information of the inventory data, record the corresponding adjustment log, and return the data operation success message.
3. The method according to claim 1, characterized in that When the data operation start message is consumed from the message queue, the method further includes: Parsing the data operation start message to obtain the operation identifier and operation content corresponding to the data operation request; The operation identifier and the operation content are stored in a disk database according to a preset format, and the operation identifier is added to a list of pending operation requests in the disk database.
4. The method according to claim 1, characterized in that When the data operation success message is consumed from the message queue, the method further includes: Determine whether the current version number of the inventory data in the data operation success message is greater than the original version number of the inventory data in the disk database; If so, synchronizing the data operation result in the data operation success message to the disk database, recording a data operation success log in the disk database, and removing the operation identifier corresponding to the data operation request from the list of pending operation requests in the disk database; If not, a data operation success log is recorded in the disk database, and the operation identifier corresponding to the data operation request is removed from the list of pending operation requests in the disk database.
5. The method according to claim 1, characterized in that When the data operation rollback message is consumed from the message queue, the method further includes: Rolling back the relevant information of the inventory data based on the rollback identifier corresponding to the data operation rollback message; Determine whether the current version number of the inventory data in the rollback success message returned after the rollback is successful is greater than the original version number of the inventory data in the disk database; If yes, the data operation result in the rollback success message is synchronized to the disk database, a data operation rollback log is recorded in the disk database, and the operation identifier corresponding to the data operation request is removed from the list of pending operation requests in the disk database; If not, the data operation rollback log is recorded in the disk database, and the operation identifier corresponding to the data operation request is removed from the list of pending operation requests in the disk database.
6. The method according to claim 3, characterized in that After adding the operation identifier to the list of pending operation requests in the disk database, the method further includes: Scan the list of pending operation requests according to a set time period to determine the operation duration of the inventory data in the list of pending operation requests; When it is determined that the operation duration is greater than the set duration, the relevant information of the inventory data is rolled back based on the operation identifier; Determine whether the current version number of the inventory data in the rollback success message returned after the rollback is successful is greater than the original version number of the inventory data in the disk database; If yes, the data operation result in the rollback success message is synchronized to the disk database, a data operation rollback log is recorded in the disk database, and the operation identifier corresponding to the data operation request is removed from the list of pending operation requests in the disk database; If not, the data operation rollback log is recorded in the disk database, and the operation identifier corresponding to the data operation request is removed from the list of pending operation requests in the disk database.
7. The method according to claim 5, characterized in that The step of rolling back the relevant information of the inventory data based on the rollback identifier corresponding to the data operation rollback message includes: By using the rollback identifier, query whether there is a rollback log in the memory database; If yes, return the rollback success message; If not, the rollback identifier is used to query whether there is an operation log in the memory database. When it is determined that the operation log exists, the relevant information of the inventory data is rolled back, the rollback log is recorded, and the rollback success message is returned.
8. A data processing device, characterized in that: include: A data receiving module, used to receive a data operation request sent by a target object, wherein the data operation request includes inventory data to be operated; A data forwarding module, configured to determine a target memory node corresponding to the inventory data based on an association relationship between a data identifier of the inventory data and a memory node after sending a data operation start message to a message queue, and forward the data operation request to the target memory node; A determination module, used to determine whether a data operation result returned by the target memory node is received; If yes, sending a data operation success message to the message queue and the target object; If not, a data operation rollback message is sent to the message queue, and a data operation failure message is returned to the target object.
9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to implement the method steps of any one of claims 1 to 7 when executing the computer program stored in the memory.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.