Data processing method, apparatus, device, storage medium, and program product

Abstract

This application relates to a data processing method, apparatus, device, storage medium, and program product. The method includes: performing business processing on each piece of business data to obtain a business processing result; wherein the business processing result includes a business identifier of the business data; comparing the business identifier with a current maximum business identifier; wherein the current maximum business identifier is determined based on the business identifier of the business data most recently written to persistent storage; and, in response to a business identifier not exceeding the current maximum business identifier, writing the business processing result to persistent storage; wherein the business processing result and the corresponding business data are stored together in the persistent storage. This method ensures the consistency between the business processing result and the business data storage, effectively avoiding invalid operations and resource waste.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] As the scale of business data continues to expand, distributed collaborative processing technology has gradually emerged. Its features of supporting high-concurrency data generation and multi-module division of labor and cooperation have adapted to the massive business processing needs of industries such as e-commerce and finance.

[0003] In traditional technologies, business data processing mainly involves first storing the business data in the database, then triggering subsequent business processing logic on the business data, and finally writing the business processing results into the database.

[0004] However, this processing method has significant drawbacks: if business data has not been successfully submitted to the database, or if the database fails to update in a timely manner due to network latency or service anomalies, the business processing result may be written to the database under the assumption that the corresponding business data has already been stored, resulting in invalid business processing result write records. This not only consumes storage resources and system computing power, but may also cause problems such as data redundancy and association failure, seriously affecting the effectiveness of business data processing. Summary of the Invention

[0005] Therefore, it is necessary to provide a data processing method, apparatus, device, storage medium, and program product to address the aforementioned technical problems, which can ensure the consistency between business processing results and business data storage, and effectively avoid invalid operations and resource waste.

[0006] In a first aspect, this application provides a data processing method, including:

[0007] For each piece of business data, business processing is performed on the business data to obtain the business processing result; the business processing result includes the business identifier of the business data.

[0008] Compare the business identifier with the current maximum business identifier; the current maximum business identifier is determined based on the business identifier of the business data most recently written to the persistent storage space;

[0009] In response to a business identifier not being greater than the current maximum business identifier, the business processing result is written to the persistent storage space; wherein, the business processing result and the corresponding business data are stored together in the persistent storage space.

[0010] In one embodiment, the method further includes: in response to the service identifier being greater than the current maximum service identifier, writing the service processing result into the cache space; and after the current maximum service identifier is updated, returning the operation of comparing the service identifier with the current maximum service identifier based on the updated current maximum service identifier.

[0011] In one embodiment, for each piece of business data, business processing is performed, including: synchronously writing the business data to persistent storage space and performing business processing on the business data; wherein each piece of business data includes a business identifier.

[0012] In one embodiment, the service identifiers of different service data increment sequentially from earliest to latest as the reception time progresses; the current maximum service identifier is determined as follows: after any service data is written, the service identifier of the service data is determined as the current maximum service identifier.

[0013] In one embodiment, for each piece of business data, the business data is synchronously written to persistent storage space and processed, including: after receiving each piece of business data sent by the business system, the business data is written to a target data queue; the business data is distributed to a first process and a second process through the target data queue, so that the first process writes the business data to persistent storage space and the second process processes the business data.

[0014] In one embodiment, after receiving each piece of business data sent by the business system, writing the business data into the data queue includes: after receiving each piece of business data sent by the business system, in response to a failure of the main data queue, determining the backup data queue as the target data queue, and writing the business data into the target data queue.

[0015] Secondly, this application also provides a data processing apparatus, comprising:

[0016] The business processing module is used to perform business processing on each piece of business data and obtain the business processing result; the business processing result includes the business identifier of the business data.

[0017] The comparison module is used to compare the service identifier with the current maximum service identifier; the current maximum service identifier is determined based on the service identifier of the service data most recently written to the persistent storage space.

[0018] The write module is used to write the business processing result to the persistent storage space in response to the business identifier not being greater than the current maximum business identifier; wherein, the business processing result and the corresponding business data are stored together in the persistent storage space.

[0019] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the various method embodiments provided in the first aspect above.

[0020] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the various method embodiments provided in the first aspect above.

[0021] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the various method embodiments provided in the first aspect above.

[0022] The aforementioned data processing methods, devices, equipment, storage media, and program products, by performing business processing on each piece of business data to obtain a business processing result including a business identifier, ensure a unique binding between the business processing result and the corresponding business data. This provides a core basis for subsequent correlation verification and avoids mismatches between business processing results and business data. By comparing the business identifier with the current maximum business identifier, it can accurately determine whether the business data corresponding to that business identifier has been persistently stored. The current maximum business identifier is a direct mapping of the storage state; if the business identifier is not greater than this value, it means that the data has been safely stored. This verification step logically establishes a synchronous association between the processing result being written to the database and the data being stored. In response to the business identifier not being greater than the current maximum business identifier, the business processing result is written to the persistent storage space and associated with the corresponding business data for storage. This process eliminates invalid operations where processing results are written to the database even though the business data is not stored, while also avoiding problems such as storage resource occupation, computing power consumption, and data redundancy caused by invalid database writes. This ensures the consistency between the business processing result and the business data storage and achieves efficient resource utilization. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0025] Figure 2 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0026] Figure 3A schematic diagram illustrating a process for synchronous business data processing and business data storage provided in an embodiment of this application;

[0027] Figure 4 A flowchart illustrating the most detailed data processing method provided in this application embodiment;

[0028] Figure 5 A structural block diagram of a data processing apparatus provided in an embodiment of this application;

[0029] Figure 6 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0031] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0032] The data processing method provided in this application can be applied to a computer device with data processing capabilities. This computer device can be presented as a server or as a terminal. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0033] In one exemplary embodiment, such as Figure 1 As shown, a data processing method is provided. Taking the application of this method to a server as an example, it includes the following steps S101-S103:

[0034] S101: For each piece of business data, perform business processing to obtain the business processing result.

[0035] The business processing result includes the business identifier of the business data.

[0036] Business data refers to various types of data generated during the operation of a business system that need to be stored and processed, such as order data from e-commerce platforms, transaction data from financial systems, and user operation data from social platforms. It includes core fields such as business identifiers and business attribute information.

[0037] The so-called business processing result refers to the result data obtained after performing business processing (such as the amount calculation of order data (clearing of order data), risk verification of transaction data, behavioral analysis of user operation data, etc.). This result data contains the business identifier of the corresponding business data, and may also contain the processed attribute information, such as the clearing amount and order status in the order clearing result, and the risk level and whether the verification is passed in the transaction risk verification result.

[0038] A business identifier refers to characteristic information used to uniquely identify a specific business data. It can be a number, a string, or a combination of numbers and strings. Its generation rules must ensure uniqueness, such as using a numerical sequence number format for order IDs.

[0039] Optionally, taking the server as an example in a distributed system, this distributed system is connected to the business system to receive and process business data generated by the business system. Specifically, the distributed system may include a business processing subsystem, which is connected to the business system. After generating business data, the business system sends each piece of business data to the business processing subsystem. The business processing subsystem processes each piece of business data by calling a preset business processing algorithm to obtain the business processing result. Taking an order business system as an example, the business data is the order data generated by the order system (i.e., the business system). Each piece of order data may contain information such as order ID (Identifier, business identifier), unit price of goods, quantity purchased, and discount amount. The clearing subsystem (i.e., the business processing subsystem) performs amount clearing (i.e., business processing) on ​​each piece of order data to obtain the order processing result for each piece of order data.

[0040] S102, compare the business identifier with the current maximum business identifier.

[0041] The current maximum business identifier is determined based on the business identifier of the business data most recently written to the persistent storage space.

[0042] The so-called current maximum business identifier refers to the largest business identifier among the currently stored business data, determined based on the business identifier of the latest successfully written business data to the persistent storage space. It is used to reflect the latest storage status of the persistent storage space in real time.

[0043] Persistent storage refers to storage media that can store data for a long time and is not easily lost. It supports data reading, writing, and related queries, such as the MySQL (My Structured Query Language) database.

[0044] Optionally, the current maximum business identifier is determined based on the business identifier of the business data most recently written to the persistent storage space. For example, if the order ID of the most recently stored order is 003, then the current maximum business identifier is 003. The size of the business identifier is compared with the current maximum business identifier. Specifically, if the business identifier is a numeric type, the comparison is made based on the corresponding values ​​of the business identifier and the current maximum business identifier; if the business identifier is a string type, the comparison can be made according to the character encoding order or a preset sorting rule. The string can consist of only letters or a string consisting of letters and numbers. The current maximum business identifier is cached in the cache space to achieve global synchronization and sharing of storage state.

[0045] As an optional implementation, taking the server in the distributed system as an example, based on the distributed system structure mentioned in step S101 above, the distributed system may further include a data persistence subsystem, a caching subsystem, and an asynchronous write-to-database subsystem. The data persistence subsystem is connected to the business system and the caching subsystem, respectively, and the asynchronous write-to-database subsystem is connected to both the data persistence subsystem and the caching subsystem. The data persistence subsystem includes persistent storage space, which can store business data generated by the business system. The caching subsystem includes cache space, which can store the current maximum business identifier and the business processing result. The asynchronous write-to-database subsystem is used to obtain the current maximum business identifier and the business processing result from the caching subsystem and compare the size of the business identifier with that of the current maximum business identifier.

[0046] S103, in response to the business identifier not being greater than the current maximum business identifier, write the business processing result to the persistent storage space.

[0047] In the persistent storage space, the business processing results are stored in association with the corresponding business data.

[0048] Optionally, if the business identifier is not greater than the current maximum business identifier, it means that the business data corresponding to the business identifier has been successfully written to the persistent storage space. In this case, the business processing result corresponding to the business identifier is written to the specified storage location in the persistent storage space, and an association is established between the business identifier and the corresponding business data.

[0049] The aforementioned data processing method ensures a unique binding between the processing result and the corresponding business data by performing business processing on each piece of business data, thus providing a core basis for subsequent correlation verification and avoiding mismatches between processing results and business data. By comparing the business identifier with the current maximum business identifier, it can accurately determine whether the business data corresponding to the business identifier has been persistently stored. The current maximum business identifier is a direct mapping of the storage state; if the business identifier is not greater than this value, it means that the data has been safely stored. This verification step logically establishes a synchronous association between the processing result being written to the database and the data being stored. In response to the business identifier not being greater than the current maximum business identifier, the processing result is written to the persistent storage space and associated with the corresponding business data for storage. This process eliminates invalid operations where processing results are written to the database even though the business data is not stored, and avoids problems such as storage resource occupation, computing power consumption, and data redundancy caused by invalid database writes. It not only ensures the consistency between the processing result and the business data storage but also achieves efficient resource utilization.

[0050] Based on the above embodiments, in an exemplary embodiment, such as Figure 2 As shown, the method may further include the following steps:

[0051] S201, in response to the business identifier being greater than the current maximum business identifier, write the business processing result to the cache space.

[0052] Cache space refers to a storage medium used for temporary data storage with fast access speed. It supports high-frequency data read and write operations, such as Redis (REmote DIctionary Server) cache.

[0053] Optionally, if the business identifier is greater than the current maximum business identifier, it means that the business data corresponding to the business identifier has not yet been stored in the persistent storage space. In this case, the business processing result corresponding to the business identifier will not be written to the database (i.e., written to the persistent storage space), but will be written to the cache space.

[0054] S202, after the current maximum service identifier is updated, return to perform the operation of comparing the service identifier with the current maximum service identifier based on the updated current maximum service identifier.

[0055] Optionally, the system can detect changes in the current maximum service identifier in the cache space in real time. If the current maximum service identifier is found to be greater than the previously recorded current maximum service identifier, it is determined that the current maximum service identifier has been updated. Based on the updated current maximum service identifier, the system can obtain the service identifier corresponding to the service processing result in the cache space and return to perform the operation of comparing the service identifier with the current maximum service identifier.

[0056] As an optional implementation, a retry interval (e.g., 10ms) can be preset, triggering a comparison operation every 10ms. This involves reading the updated current maximum service identifier and the service identifier corresponding to the service processing result from the cache space, and comparing the service identifier with the updated current maximum service identifier. If the service identifier is not greater than the current maximum service identifier, the write operation in step S103 above is performed; if it is still greater, the process continues to wait for the next retry interval until the service processing result corresponding to the service identifier is written to the persistent storage space.

[0057] As another optional implementation, after the business processing subsystem processes each piece of business data, the business processing result corresponding to each piece of business data can be cached in the cache space. For any business processing result, it is compared with the current maximum business identifier; if the business identifier corresponding to the business processing result is not greater than the current maximum business identifier, the business processing result is stored in the persistent storage space; if the business identifier corresponding to the business processing result is greater than the current maximum business identifier, the business processing result can continue to be cached in the cache space, and after waiting for a preset retry time interval, the current maximum business identifier (which may have been updated during this waiting time) and the business identifier of the business processing result are retrieved from the cache space, and the retrieved current maximum business identifier and the business identifier are compared. The write operation in step S103 above is then performed until the business processing result corresponding to the business identifier is written to the persistent storage space.

[0058] In this embodiment, when the business identifier is greater than the current maximum business identifier, the business processing result is first written to the cache space for temporary storage, and then asynchronously written to the database through a timed verification mechanism. This avoids blocking the business process and ensures eventual consistency through retry logic, thus balancing real-time performance and reliability.

[0059] Based on the above embodiments, in an exemplary embodiment, the business processing process in S102 is further refined. Optionally, it may include, for each piece of business data, synchronously writing the business data to persistent storage space, and performing business processing on the business data. Each piece of business data includes a business identifier.

[0060] Optionally, for each piece of business data, two operations can be triggered simultaneously: writing the business data to the persistent storage space and processing the business data. Specifically, after generating business data, the business system in the business system can send each piece of business data to the data persistence subsystem and the business processing subsystem respectively. The data persistence subsystem is used to write the business data to the persistent storage space, and the business processing subsystem is used to process the business data, thereby realizing asynchronous parallel processing of business data.

[0061] In this embodiment, by synchronously executing business data storage and business processing, the processing cycle of business data is shortened and the system throughput is improved. At the same time, the parallel execution mode keeps the progress difference between business data processing and business data storage within a reasonable range, providing a better time window for subsequent identifier comparison and verification, reducing the number of retry verifications, and further improving the system processing efficiency.

[0062] Based on the above embodiments, in an exemplary embodiment, the service identifiers of different service data increment sequentially from earliest to latest receiving time, further refining the process for determining the current maximum service identifier. Optionally, this may include determining the service identifier of any service data as the current maximum service identifier after the data has been written.

[0063] Optionally, the business system receives business data in chronological order, assigning a unique business identifier to each piece of data. The business identifier increments sequentially from earliest to latest received time. The business identifier can be a pure numeric sequence; for example, the identifier for the first piece of business data is 1, the second is 2, and so on. After the data persistence subsystem completes the persistent storage of business data, it can immediately send a storage completion notification to the caching subsystem. This notification includes the business identifier of the data. Upon receiving the notification, the caching subsystem directly sets this business identifier as the new current maximum business identifier and synchronously updates it in the cache space. For example, after the order data with business identifier 1003 is stored, the caching subsystem updates the current maximum business identifier in the cache from 1002 to 1003; subsequently, after the order with business identifier 1004 is stored, it updates it again to 1004, ensuring that the current maximum business identifier always reflects the latest storage status.

[0064] In this embodiment, by generating service identifiers incrementally according to the reception time, the service identifiers acquire a time sequence attribute, simplifying the logic for determining the current maximum service identifier. No complex sorting or query operations are required; only the latest completed service identifier needs to be used as the current maximum service identifier, thus improving identifier update efficiency. Simultaneously, the incremental service identifiers ensure the accuracy of the comparison verification. A larger service identifier indicates a later reception time and a lower probability of completed storage. By using the verification rule that the service identifier is no greater than the current maximum service identifier, it is possible to accurately determine whether service data has been stored, further strengthening the avoidance of invalid data storage.

[0065] Based on the above embodiments, in an exemplary embodiment, the process of synchronous business data processing and business data storage is further refined. Optionally, such as Figure 3 As shown, the following steps may be included:

[0066] S301: After receiving each piece of business data sent by the business system, write the business data into the target data queue.

[0067] The so-called target data queue refers to a message queue used to store and distribute business data, which enables asynchronous distribution of business data.

[0068] Optionally, after receiving each piece of business data sent by the business system, the distributed system writes the business data to a preset target data queue through the queue write interface.

[0069] S302, through the target data queue, distribute business data to the first process and the second process, so that the first process writes the business data into the persistent storage space, and the second process performs business processing on the business data.

[0070] The so-called first process refers to an independent operating unit specifically responsible for writing business data into persistent storage space, such as the data persistence subsystem mentioned above; the so-called second process refers to an independent operating unit specifically responsible for performing preset business logic processing on business data, such as the business processing subsystem mentioned above.

[0071] Optionally, the target data queue distributes the business data to both the first process and the second process simultaneously through a dual subscription mechanism. After receiving the business data, the first process performs persistent storage operations, and after receiving the business data, the second process performs business processing operations.

[0072] As an optional implementation, steps S301-S302 above can be the execution process of using the main data queue as the target data queue when the main data queue is not faulty. After receiving each service data sent by the service system, in response to a main data queue fault, the backup data queue is determined as the target data queue, and the service data is written to the target data queue.

[0073] The so-called master data queue refers to the core queue used by default in the distributed system for business data distribution; the so-called backup data queue refers to the standby queue used to replace the master queue when the master queue fails. The two are configured identically to ensure data distribution logic compatibility.

[0074] Optionally, the operating status of the primary data queue can be monitored in real time. When a failure is detected in the primary data queue (such as connection interruption, write success rate below a threshold, or all nodes becoming unavailable), a failover mechanism is automatically triggered, designating the backup data queue as the new target data queue, and subsequent business data is written to the backup data queue. After the primary data queue is troubleshooted and returns to normal operation, the business data temporarily stored in the backup data queue is synchronized to the primary data queue to ensure data consistency between the two queues. After synchronization is complete, the system automatically switches back to the primary data queue as the target data queue, restoring the normal distribution architecture.

[0075] This embodiment decouples business data storage and business data processing through a data queue distribution mechanism, avoiding direct dependencies between the two operations, improving the flexibility and scalability of the distributed system, ensuring that business data can be synchronously transmitted to the storage process and the processing process, guaranteeing the stability of parallel execution, and providing data distribution support for subsequent identification comparison and accurate database storage. By using a primary / backup queue switching method, the problem of business data distribution interruption caused by data queue failure is solved, ensuring that business data can be continuously and stably distributed to the storage process and the processing process, avoiding system process interruption due to queue failure, and improving the high availability of the entire system.

[0076] Based on the above embodiments, in an exemplary embodiment, the method may further include:

[0077] The caching subsystem is deployed using a Redis master-slave high-availability architecture, which includes one master node and at least one standby slave node. When the master node fails, it automatically switches to the standby slave node, synchronizes the data, and restores the state synchronization function.

[0078] The data persistence subsystem is deployed in a MySQL master-slave cluster architecture, which includes one master node and several read-only replica nodes. When the MySQL master node fails, it switches to a read-only replica and starts the master node recovery process to ensure that data reading is uninterrupted.

[0079] Based on the above embodiments, in an exemplary embodiment, such as Figure 4 As shown, the method may include the following steps:

[0080] S401: Upon receiving each piece of business data sent by the business system, in response to a failure in the main data queue, the backup data queue is identified as the target data queue, and the business data is written to the target data queue.

[0081] S402, through the target data queue, distributes business data to the first process and the second process, so that the first process writes the business data into the persistent storage space, and the second process performs business processing on the business data.

[0082] S403: After any business data is written, the business identifier of the business data is determined as the current maximum business identifier.

[0083] S404, compare the business identifier with the current maximum business identifier.

[0084] S405, in response to the business identifier not being greater than the current maximum business identifier, writes the business processing result to the persistent storage space.

[0085] S406, in response to a business identifier being greater than the current maximum business identifier, writes the business processing result to the cache space.

[0086] S407: After the current maximum service identifier is updated, return to perform the operation of comparing the service identifier with the current maximum service identifier based on the updated current maximum service identifier.

[0087] The specific implementation methods of steps S401-S407 are the same as those in the above method embodiments, and will not be repeated here.

[0088] This data processing method employs a master-slave redundant queue to transmit business data, simultaneously distributing it to both the data processing subsystem and the data persistence subsystem via dual links to prevent business data loss. After business data persistence is complete, the current maximum business identifier is synchronized to Redis, achieving global sharing of storage status. After the asynchronous data persistence subsystem retrieves the business processing results, it verifies the relationship between the corresponding business identifier and the current maximum business identifier, and only performs data persistence on business processing results for which the corresponding business data has already been stored. Redis is used to achieve state communication between the business data storage and business data processing processes, ensuring process progress matching and avoiding invalid operations.

[0089] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0090] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0091] In one exemplary embodiment, such as Figure 5 As shown, a data processing apparatus is provided, including: a business processing module 501, a comparison module 502, and a writing module 503, wherein:

[0092] The business processing module 501 is used to perform business processing on each piece of business data to obtain a business processing result; wherein, the business processing result includes the business identifier of the business data.

[0093] The comparison module 502 is used to compare the service identifier with the current maximum service identifier; wherein, the current maximum service identifier is determined based on the service identifier of the service data most recently written to the persistent storage space;

[0094] The write module 503 is used to write the business processing result to the persistent storage space in response to the business identifier not being greater than the current maximum business identifier; wherein, the business processing result and the corresponding business data are stored together in the persistent storage space.

[0095] In an exemplary embodiment, the apparatus further includes: a cache processing module, configured to write the service processing result into a cache space in response to a service identifier being greater than the current maximum service identifier; and an update processing module, configured to, after the current maximum service identifier is updated, return the operation of comparing the service identifier with the current maximum service identifier based on the updated current maximum service identifier.

[0096] In an exemplary embodiment, the business processing module 501 includes: a synchronization submodule, used to synchronously write business data to persistent storage space for each piece of business data, and to perform business processing on the business data; wherein each piece of business data includes a business identifier.

[0097] In an exemplary embodiment, the service identifiers of different service data increment sequentially from earliest to latest as the reception time progresses; the device further includes a maximum identifier determination module, used to determine the service identifier of any service data as the current maximum service identifier after the writing of any service data is completed.

[0098] In an exemplary embodiment, the synchronization submodule includes: a queue writing unit, configured to write the business data into a target data queue after receiving each business data sent by the business system; and a distribution unit, configured to distribute the business data to a first process and a second process through the target data queue, so that the first process writes the business data into persistent storage space and the second process performs business processing on the business data.

[0099] In an exemplary embodiment, the queue writing unit is specifically configured to: upon receiving each piece of business data sent by the business system, in response to a failure of the main data queue, determine the backup data queue as the target data queue, and write the business data into the target data queue.

[0100] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0101] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores business data and business processing results. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a data processing method.

[0102] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0103] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0104] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0105] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0106] It should be noted that the data involved in this application (including but not limited to data used for analysis, such as business data) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0107] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0108] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0109] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data processing method, characterized in that, The method includes: For each piece of business data, business processing is performed on the business data to obtain a business processing result; wherein, the business processing result includes the business identifier of the business data; Compare the service identifier with the current maximum service identifier; wherein the current maximum service identifier is determined based on the service identifier of the service data most recently written to the persistent storage space; In response to the fact that the service identifier is not greater than the current maximum service identifier, the service processing result is written to the persistent storage space; wherein, the service processing result and the corresponding service data are stored together in the persistent storage space.

2. The method according to claim 1, characterized in that, The method further includes: In response to the fact that the service identifier is greater than the current maximum service identifier, the service processing result is written to the cache space; After the current maximum service identifier is updated, the operation of comparing the service identifier with the current maximum service identifier is returned based on the updated current maximum service identifier.

3. The method according to claim 1 or 2, characterized in that, The step of performing business processing on each piece of business data includes: For each piece of business data, the business data is synchronously written to the persistent storage space, and the business data is processed; wherein, each piece of business data includes a business identifier.

4. The method according to claim 1 or 2, characterized in that, The service identifiers of different service data increment sequentially from earliest to latest as they are received; the current maximum service identifier is determined in the following manner: After any business data is written, the business identifier of the business data is determined as the current maximum business identifier.

5. The method according to claim 3, characterized in that, The step of synchronously writing the business data to the persistent storage space for each piece of business data, and performing business processing on the business data, includes: After receiving each piece of business data sent by the business system, the business data is written into the target data queue; The business data is distributed to a first process and a second process through the target data queue, so that the first process writes the business data into the persistent storage space, and the second process performs business processing on the business data.

6. The method according to claim 5, characterized in that, The step of writing the business data into the data queue after receiving each piece of business data sent by the business system includes: Upon receiving each piece of business data sent by the business system, in response to a failure in the main data queue, the backup data queue is identified as the target data queue, and the business data is written into the target data queue.

7. A data processing apparatus, characterized in that, The device includes: The business processing module is used to perform business processing on each piece of business data to obtain a business processing result; wherein, the business processing result includes the business identifier of the business data; The comparison module is used to compare the service identifier with the current maximum service identifier; wherein, the current maximum service identifier is determined based on the service identifier of the service data most recently written to the persistent storage space; The write module is used to write the business processing result to the persistent storage space in response to the business identifier not being greater than the current maximum business identifier; wherein the business processing result is stored in association with the business data corresponding to the business processing result in the persistent storage space.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images