Method, system and medium for asynchronous data transmission

Abstract

The invention relates to a method, a system and a medium for asynchronous data transmission. According to one aspect of the present disclosure, there is provided a computer-implemented method for asynchronous data transmission, comprising: receiving, by a consumer, at least one message from a producer, the at least one message comprising a summary of message data generated by the producer, a batch number, and a total number of messages; and accessing, by the consumer, the message data in a storage device according to the received at least one message, wherein the message data is stored in the storage device by the producer before sending the at least one message to the consumer. According to the embodiment of the invention, improved asynchronous data transmission between modules can be realized.

Description

Technical Field

[0001] The present disclosure relates to data transmission between modules, and more particularly to a method, system and medium for asynchronous data transmission. Background Art

[0002] In the automotive industry, there are multiple systems to support dealers' reimbursement, refund, and analysis functions for different types of vehicles, policies, activities, and other businesses. The systems are numerous, complex, and inflexible, which leads to a serious separation between business and finance, difficult and inefficient work interaction between corporate departments, and difficulty for employees to get started.

[0003] Taking the above issues into consideration and the ever-changing factors in the future, we hope to establish a unified business-finance integration platform that integrates the functions of all dealer reimbursement-related systems, considers the future flexible and changeable business scenarios and architectural composition in the design, and builds a flexible and efficient business-finance integration platform. Summary of the invention

[0004] Embodiments of the present disclosure relate to methods, systems, and media for asynchronous data transmission.

[0005] According to one aspect of the present disclosure, a computer-implemented method for asynchronous data transmission is provided, comprising: receiving, by a consumer, at least one message from a producer, the at least one message comprising a summary of message data generated by the producer, a batch number, and a total number of messages; and accessing, by the consumer, the message data in a storage device based on the at least one message received, wherein the message data is stored in the storage device by the producer before sending the at least one message to the consumer.

[0006] In some embodiments, the method further includes: creating, by the consumer, a distributed lock based on the summary of the message data and the batch number; and reading, by the consumer in response to successfully creating the distributed lock, task processing information.

[0007] In some embodiments, the summary of the message data includes a transmission type, and the distributed lock is created based on both the transmission type and the batch number.

[0008] In some embodiments, the method further includes: determining, by the consumer, whether to start processing the message data from a breakpoint.

[0009] In some embodiments, the method further includes: in response to determining to start processing the message data from a breakpoint, the consumer continues to process the message data; and updating the task processing information by the consumer.

[0010] In some embodiments, the method further includes: initializing, by the consumer, task processing information in response to determining not to process the message data from a breakpoint; processing, by the consumer, the message data from the beginning; and updating, by the consumer, the task processing information.

[0011] In some embodiments, the method further includes: determining, by the consumer, whether processing of the message data is completed; and updating, by the consumer, a task status in response to determining that processing of the message data is completed.

[0012] In some embodiments, the method further includes: reading the task status by the caller.

[0013] According to another aspect of the present disclosure, a system for asynchronous data transmission implemented by a computer is provided, comprising: a storage device; a producer coupled to the storage device, configured to store generated message data in the storage device and to send at least one message to a consumer after storing the generated message data in the storage device, the at least one message comprising a summary of the message data, a batch number, and a total number of messages; and the consumer coupled to the storage device, configured to access the message data in the storage device according to the at least one message received.

[0014] According to another aspect of the present disclosure, a computer system is provided, comprising: one or more processors, and a memory coupled to the one or more processors, the memory storing computer-readable program instructions, which when executed by the one or more processors perform the method described above.

[0015] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which computer-readable program instructions are stored. When the instructions are executed by the processor, the method described above is performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0017] Figure 1 is a block diagram of a system for asynchronous data transmission.

[0018] Figure 2 is a block diagram of an exemplary system for asynchronous data transmission according to an embodiment of the present disclosure.

[0019] Figure 3 is an example flow chart of a method for asynchronous data transmission implemented by a computer according to an embodiment of the present disclosure.

[0020] Figure 4 is an example block diagram of an exemplary system for asynchronous data transmission implemented by a computer according to an embodiment of the present disclosure.

[0021] Figure 5 is a flowchart of an exemplary method for asynchronous data transmission implemented by a computer according to an embodiment of the present disclosure.

[0022] Figure 6 FIG. 4 is a schematic diagram showing a general hardware environment in which a device according to an embodiment of the present disclosure can be implemented. DETAILED DESCRIPTION

[0023] The following description is provided to enable those skilled in the art to make and use the embodiments, and is provided in the context of a specific application and its requirements. Various modifications will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the embodiments. Therefore, the embodiments are not limited to the embodiments shown, but are to be given the widest scope consistent with the principles and features disclosed herein.

[0024] It is expected to build a business and financial integration platform, which will involve data integration, collect data from many data source systems, build data transmission channels, and achieve timely, accurate and secure data transmission. In the database, data is managed at different levels.

[0025] The platform needs to manage data in different fields according to the different characteristics and requirements of the services to achieve efficient and orderly technological development.

[0026] The platform will involve many calculation or processing modules. For example, the platform may include policy and project management module, provision management module, rebate management module, payment application module, KPI assessment module, rebate provisional estimation module, support document management module, penalty management module, etc. These modules can implement policy and activity management to support business departments to release corresponding activity policies according to market conditions and corporate strategies; can automatically organize and split project dimensions according to policies and activities and in combination with business and financial needs; can implement corporate financial management, automatically implement financial actions such as provision, write-off, and clearing accounts, and timely and accurately reflect the financial status of the company; can be a calculation platform for dealer reimbursement and rebate, and can be combined with auxiliary modules such as dealer batch, retail assessment, dealer quality assessment, and support documents to calculate the final reimbursement and rebate data; can implement payment management, and dealers can obtain reimbursement and rebates through red tickets, blue tickets, and prepaid payment methods.

[0027] The platform can be based on microservices and cloud platform technologies. These modules can be distributed on different devices in a computer network and on different devices on a cloud platform.

[0028] Some modules need to provide messages to other modules for consumption. Usually, the module that provides messages encapsulates the message data to be transmitted into the message body and transmits the message data to the module that is the message consumer in the message queue. When a large amount of message data needs to be transmitted between two modules, it often involves the transmission of hundreds of thousands or even millions of messages between the two modules.

[0029] It is expected to achieve safer, more efficient and reliable message data transmission between modules.

[0030] Figure 1 is a block diagram of a system for asynchronous data transmission.

[0031] like Figure 1 As shown, the system includes producers, consumers, callers, and message queues transmitted between producers and consumers. Producers, consumers, and callers are, for example, functional modules implemented by software and / or hardware of one or more computers. For example, a producer can be a message generation module that generates business and / or financial related messages. Consumers are, for example, modules that implement specific computing / processing functions, such as financial accounting modules. Callers can be modules that are further downstream of consumers, which, for example, call processing results and / or processed data from consumers to perform the next step of transmission / computation / processing. In some embodiments, consumers and callers can be different routines.

[0032] Since the size of a message and the size of the message queue are limited, when the amount of message data to be transmitted is large, a large number of messages (such as Figure 1 The message data is transmitted via the message queue (1...n) in the message queue. Usually, hundreds of thousands or millions of messages may need to be transmitted.

[0033] Message data may refer to data associated with business and / or finance. Message data may be any data used for calculation and / or processing defined according to design requirements.

[0034] In this case, multiple (e.g., dozens) parallel consumer computing / processing modules are often involved, each of which receives a portion of the messages and processes them. For example, when a processing task involving a consumer receives a large amount of message data, multiple parallel consumer modules are required to receive the message data from the producer and process the corresponding portions of the message data respectively. Figure 1 In FIG. 4 , multiple consumer modules are shown, which are associated with the same specific task, for example.

[0035] In this case, since each consumer module only processes a portion of the message data that it receives, it is often difficult to verify when there is loss, error or retransmission in the message data, especially it is difficult to verify the entire message data. Even if each consumer module can determine that there is an error or loss in the portion it receives, and provide feedback to the producer in the reverse direction and request retransmission, the entire system will be complicated and inefficient. In addition, it is also difficult for each consumer module to confirm whether the message data has been received. At this time, it is often necessary to set up a special monitoring module to verify the reception of the entire message data.

[0036] In addition, message data is prone to loss and errors during transmission, and the integrity and reliability of the message data cannot be well guaranteed.

[0037] When a system platform involves many modules and needs to transmit complex and changeable messages between these modules, and the system has high requirements for data integrity and reliability, this will cause problems. For example, for an email transmission / processing system, if some data is lost or wrong when transmitting a large amount of data between modules, it will not affect user use. However, for platforms such as financial systems, the transmitted data must be accurate to ensure the accuracy of the subsequent calculation / processing results.

[0038] like Figure 1 As shown, the system also includes a database, and consumers and regulators can initialize the message processing information for a specific task, update the processing information, query the processing status, and clean up data regularly through interaction with the database. The processing information here refers to the message processing information associated with a specific task, and the processing status refers to the status of the specific task. Figure 2 and Figure 3 Introduce in more detail.

[0039] Figure 2 is a block diagram of an exemplary system for asynchronous data transmission according to an embodiment of the present disclosure.

[0040] like Figure 2 As shown, the system includes a producer, a consumer, a caller, and a message queue transmitted between the producer and the consumer. In particular, the system includes a cache coupled to the producer and the consumer.

[0041] In some embodiments, Redis cache technology can be used to increase the speed and performance of queries and loading to improve access speed and reduce database load.

[0042] Those skilled in the art will appreciate that other types of storage devices (such as databases, storage files, etc.) may also be provided to replace the cache.

[0043] By setting up the cache, the producer does not need to transmit a large amount of message data to the consumer through the message queue, but can directly package the message data into the cache. After storing the message data into the cache, the producer generates a message, for example, which may include information such as the summary, batch number, and total number of the message data stored in the cache. The producer sends the generated message to the consumer. The consumer can access the corresponding consumption data from the cache based on the received message.

[0044] Those skilled in the art will appreciate that, in some cases, multiple messages may be generated. Figure 1 The transmission of hundreds of thousands or millions of messages per day, as shown in Figure 2 The system in only needs a very small amount of message transmission. Figure 1 There may be dozens of consumer modules required. Figure 2 The system in may only need one consumer module for message processing.

[0045] Message data is directly stored in the cache and accessed by consumers based on the summary information contained in the received message. It also allows for convenient verification of the message data as a whole and avoids possible loss and errors during message data transmission, ensuring data integrity and reliability. This is especially useful in application scenarios that require data to be absolutely reliable and error-free.

[0046] When a system involves many modules and complex and changeable message data transmission is required between the modules, the method according to the embodiment of the present disclosure provides a cache accessible to all modules, allowing any producer module to store consumption data in the cache before transmitting data to the consumer module, and then send information such as the consumption data summary to the consumer. The consumer only needs to receive and process information such as the consumption data summary, and can access the required consumption data more quickly. Overall, this not only increases the reliability of platform data transmission but also greatly improves the efficiency and speed of platform response.

[0047] In addition, the message data transmitted between modules may have a certain degree of repetitiveness. The producer module can store the data that needs to be sent frequently within a period of time in the cache, and only send the summary of the message data and other information to one or more consumers in need once or multiple times, thereby avoiding repeated transmission of the message data itself to one or more consumers. In addition, since multiple consumers access the consumption data from the cache or one consumer accesses the consumption data from the cache each time, strong consistency between the accessed consumption data can be guaranteed. After the predetermined period of time ends, the stored message data can be cleared from the cache to free up space for message data transmission between other modules. These can greatly improve the resource utilization of the system and improve the operating efficiency of the system.

[0048] In addition, the cache setting also reduces the load on the database, so that each module can access the database only when it needs to initialize, update or query task processing information, which can allow for faster system response.

[0049] Figure 3 is an example flow chart of a method for asynchronous data transmission implemented by a computer according to an embodiment of the present disclosure.

[0050] like Figure 3 As shown, the method 300 includes operation 301, in which a consumer receives at least one message from a producer, and the at least one message includes a summary of message data generated by the producer, a batch number, and a total number of messages.

[0051] The message summary may include a brief description of the message data, such as a producer ID, a consumer ID, a brief description of the content of the consumption data, processing requirements, etc. The message summary may have various forms, as long as it can at least identify the type (transmission type) of the message data. For example, the summary of the message data may identify which producer sends which consumer the message data for which specific task.

[0052] Multiple batches of message data can be stored in the buffer, and each batch of message data can be indexed by a batch number. In some embodiments, a specific batch of message data can be uniquely indexed by the batch number and other information such as the type (transmission type) of the message data.

[0053] The total number of messages in the message data can be used to verify the integrity of the message data.

[0054] The method 300 may further include an operation 302, in which the consumer accesses the message data in the storage device according to the received at least one message, wherein the message data is stored in the storage device by the producer before sending the at least one message to the consumer.

[0055] The storage device may include any device capable of implementing a data storage function, such as a database, a cache, etc. In some embodiments, a cache technology such as Redis may be used to implement data access.

[0056] Figure 4 is an example block diagram of an exemplary system 400 for asynchronous data transmission implemented by a computer according to an embodiment of the present disclosure.

[0057] like Figure 4 As shown, the system 400 may include a storage device 401 , a producer 402 coupled to the storage device 401 , and a consumer 403 coupled to the storage device 401 .

[0058] The producer 402 may be configured to store the generated message data in the storage device 401 and send at least one message to the consumer 403 after storing the generated message data in the storage device 401 , the at least one message including a summary of the message data, a batch number, and a total number of messages.

[0059] The consumer 403 may be configured to receive the at least one message and access the message data in the storage device 403 according to the received at least one message.

[0060] Figure 5 is a flow chart of an exemplary method 500 for asynchronous data transmission implemented by a computer according to an embodiment of the present disclosure.

[0061] like Figure 5 As shown, method 500 includes operation 501, in which the producer writes the consumption data into the cache. Message data is, for example, a batch of data associated with a specific processing task.

[0062] The method 500 includes an operation 502, in which the producer sends at least one message to the consumer. The at least one message includes a summary of message data generated by the producer and written into a cache, a batch number, and a total number of messages.

[0063] Table 1 shows the structure of an exemplary message body.

[0064] summary batch total

[0065] Table 1

[0066] The message body includes, for example, a summary of message data (or a portion of the summary), a batch, and a total number of messages.

[0067] In some embodiments, the summary may include at least the transmission type type shown in Table 2 below. The transmission type and the batch number together may serve as a unique index for the message data. The transmission type may be any identifier or identification information for identifying the type of the message data. It will be appreciated by those skilled in the art that the message body may also include other information, such as additional information for verifying the message data, a specific storage location in the cache, etc.

[0068] Method 500 may include operation 503, in which the consumer may obtain / create a distributed lock. This may avoid repeated operations caused by retries. Because the entire consumption data may be stored in the cache, the consumer is able to apply a distributed lock to the entire consumption data to avoid repeated operations.

[0069] In some embodiments, a distributed lock may be constructed based on both a summary in the message (eg, the transmission type contained in the summary) and a batch number.

[0070] If the distributed lock is successfully created in operation 503 , method 500 proceeds to operation 504 , where the consumer may read the task processing information.

[0071] Task processing information can be accessed through interaction with the database.

[0072] Table 2 shows exemplary task processing information.

[0073] Fields type Remark id BigInt Auto-increment primary key type int Transmission Type batch_no varchar(40) Batch number total_record int Total records process_record int Number of records processed status int Status: 1 is in progress, 9 is completed create_time datetime Creation time update_time datetime Update time

[0074] Table 2

[0075] Among them, the type and batch_no fields can construct a joint unique index.

[0076] In some embodiments, it may also be uniquely indexed by batch number and / or other identification information.

[0077] In some embodiments, the task processing information may include breakpoint information. For example, if a batch of message data includes 10,000 pieces of data, a breakpoint may be set for every 1,000 pieces of data. If only the first 5,000 pieces of data were processed due to a failure during the last processing, the information of the breakpoint processed last time may be recorded. The number of processed records process_record in Table 2 may also be considered as a kind of breakpoint information. This can avoid having to restart task processing in the event of a system failure or a consumer module failure.

[0078] If the creation of the distributed lock in operation 503 fails, the consumer operation ends.

[0079] The method 500 may further include an operation 505 , in which the consumer may determine whether it is necessary to start processing from the breakpoint.

[0080] If it is determined at operation 505 that the processing needs to be started from the breakpoint, that is, the task processing information includes the breakpoint information, the method 500 proceeds to operation 506 , in which the message data is continuously processed from the breakpoint.

[0081] If it is determined at operation 505 that it is not necessary to start processing from the breakpoint, ie, a new unprocessed task is involved, the method 500 proceeds to operation 507 where processing information is initialized.

[0082] Table 3 shows input parameters of the initialization process information.

[0083] Fields Remark type Transmission Type batchNo Batch number totalRecord Total records

[0084] Table 3

[0085] Method 500 also includes operation 508, in which the consumer updates the processing information.

[0086] Table 4 shows input parameters of the update process information.

[0087] Fields Remark type Transmission Type batchNo Batch number processRecord Number of records processed status state

[0088] Table 4

[0089] This allows a batch of data to be processed multiple times by dividing it into a certain number of small batches, and each small batch processing will be recorded in the database.

[0090] Method 500 then proceeds to operation 509 where the consumer determines whether processing of all message data is complete.

[0091] If it is determined at operation 509 that processing of all message data has been completed, method 500 proceeds to operation 510 where the consumer updates the status of the task.

[0092] If it is determined at operation 509 that the processing of all message data has not been completed, the method 500 returns to step 506 to continue processing the remaining message data.

[0093] In the context of the present disclosure, consumer processing of message data refers to the consumer accessing the message data stored in the cache by the producer from the cache and performing related processing.

[0094] Method 500 may also include operation 511 , in which the caller may read the task status.

[0095] Table 5 and Table 6 respectively show the input parameters and return values ​​of reading task status.

[0096] Fields Remark must type Transmission Type yes batchNo Batch number. If this parameter is not passed, the record with the largest ID under type will be taken. no

[0097] Table 5

[0098] Fields Remark processRecord Number of records processed status state

[0099] Table 6

[0100] The solution according to the embodiment of the present disclosure allows each caller to read a consistent task status through interaction with a database.

[0101] Method 500 may further include operation 512, in which the message data may be cleaned up according to the scheduled task, so that unused message data may be cleaned up in a timely manner to provide space for data transmission between other modules.

[0102] In some embodiments, the scheduled task can be set based on the time period when the message data is accessed. For example, if a batch of message data is only accessed in a first time period, the batch of message data will be automatically cleared after the first time period.

[0103] In other embodiments, the scheduled task may be set based on the frequency with which the message data is accessed. For example, message data that is frequently accessed may have a longer storage time, while message data that is not frequently accessed may have a shorter storage time.

[0104] According to the method of the embodiment of the present disclosure, information related to the storage and acquisition of message data can be recorded in the database, which can ensure the strong consistency of system data.

[0105] An example pseudo code for implementing the method for asynchronous data transmission according to an embodiment of the present disclosure is shown below.

[0106]

[0107]

[0108] Figure 6 FIG. 4 is a schematic diagram showing a general hardware environment in which a device according to an embodiment of the present disclosure can be implemented.

[0109] Reference now Figure 6 , a schematic diagram of an example of a computing node 600 is shown. Computing node 600 is only one example of a suitable computing node and is not intended to imply any limitation on the scope of use or functionality of the embodiments described herein. Regardless, computing node 600 is capable of implementing and / or performing any of the functions set forth above.

[0110] In computing node 600, there is a computer system / server 6012, which can operate with numerous other general or special computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that can be suitable for use with computer system / server 6012 include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems or devices, etc.

[0111] Computer system / server 6012 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. In general, program modules can include routines, programs, objects, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. Computer system / server 6012 can be practiced in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on both local or remote computer system storage media including memory storage devices.

[0112] like Figure 6 As shown, the computer system / server 6012 in the computing node 600 is shown in the form of a general computing device. The components of the computer system / server 6012 may include, but are not limited to: one or more processors or processing units 6016, a system memory 6028, and a bus 6018 that couples different system components including the system memory 6028 to the processing unit 6016.

[0113] Bus 6018 represents any one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example and not limitation, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCIe), and an Advanced Microcontroller Bus Architecture (AMBA).

[0114] The computer system / server 6012 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer system / server 6012, including volatile and nonvolatile media, removable and non-removable media.

[0115] System memory 6028 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 6032. Computer system / server 6012 may also include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 6034 may be provided for reading from a non-removable non-volatile magnetic medium (not shown, and commonly referred to as a "hard drive") and writing to a non-removable non-volatile magnetic medium. Although not shown, a disk drive for reading from a removable non-volatile disk (e.g., a "floppy disk") and writing to a removable non-volatile disk, and an optical drive for reading from a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical medium) and writing to a removable non-volatile optical disk may be provided. In these cases, each may be connected to bus 6018 via one or more data medium interfaces. As will be further depicted and described below, memory 6028 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of embodiments of the present disclosure.

[0116] By way of example and not limitation, a program / utility 6040 having a set (at least one) of program modules 6042 and an operating system, one or more application programs, other program modules, and program data may be stored in memory 6028. Each of the operating system, one or more application programs, other program modules, and program data, or some combination thereof, may include an implementation of a network environment. Program modules 6042 generally perform functions and / or methods as described herein in the embodiments.

[0117] The computer system / server 6012 may also communicate with one or more external devices 6014 (such as a keyboard, pointing device, display 6024, etc.), one or more devices that enable a user to interact with the computer system / server 6012, and / or any device that enables the computer system / server 6012 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may occur via an input / output (I / O) interface 22. Also, the computer system / server 6012 may communicate with one or more networks (such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet)) via a network adapter 20. As depicted, the network adapter 20 communicates with other components of the computer system / server 6012 via a bus 6018. It should be understood that, although not shown, other hardware and / or software components may be used in conjunction with the computer system / server 6012. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0118] The present disclosure may be implemented as a system, method, and / or computer program product. The computer program product may include (one or more) computer-readable storage media having computer-readable program instructions thereon for causing a processor to execute aspects of the present disclosure.

[0119] Computer readable storage medium can be a tangible device that can keep and store instructions for being used by instruction execution device.Computer readable storage medium can be, for example, (but not limited to) electronic storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device or any suitable combination of the above.The non-exhaustive list of more specific examples of computer readable storage medium includes the following: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device (such as punch card or convex structure in groove with instruction stored thereon) and any suitable combination of the above.Computer readable storage medium as used herein is not interpreted as instantaneous signal itself, such as radio wave or other free propagating electromagnetic wave, electromagnetic wave propagated by waveguide or other transmission medium (for example, light pulse by optical fiber cable) or electrical signal transmitted by wire.

[0120] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards these computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0121] The computer-readable program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as "C" programming languages ​​or similar programming languages). The computer-readable program instructions may be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or may be connected to an external computer (such as using an Internet service provider via the Internet). In some embodiments, an electronic circuit may be customized by utilizing the state information of a computer-readable program instruction, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), and the electronic circuit may execute a computer-readable program instruction to perform aspects of the present disclosure.

[0122] Aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It will be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0123] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / behaviors specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these computer-readable program instructions can guide the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium in which the instructions are stored includes a manufactured product, which includes instructions for implementing aspects of the functions / behaviors specified in one or more boxes in the flowchart and / or block diagram.

[0124] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operating steps are performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / behaviors specified in one or more boxes in the flowchart and / or block diagram.

[0125] The flow chart and block diagram in the figure show the possible architecture, function and operation of the system, method and computer program product according to each embodiment of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, segment or instruction, and a part of the module, segment or instruction contains one or more executable instructions for realizing the specified (one or more) logical functions. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the figure. For example, depending on the functions involved, two continuous boxes can actually be executed substantially in parallel, or these boxes can sometimes be executed in the opposite order. It will also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a system based on special hardware that performs a specified function or behavior or performs a combination of special hardware and computer instructions.

[0126] It should also be understood by those skilled in the art that the various operations illustrated in order in the embodiments of the present disclosure are not necessarily performed in the illustrated order. Those skilled in the art may adjust the order of the operations as needed. Those skilled in the art may also add more operations or omit some of the operations as needed.

[0127] The description of various embodiments of the present disclosure has been given for the purpose of illustration, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements over technologies found in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for asynchronous data transmission implemented by a computer, comprising: receiving, by a consumer from a producer, at least one message including a summary of message data generated by the producer, a batch number, and a total number of messages; and The message data in the storage device is accessed by the consumer according to the received at least one message, wherein the message data is stored in the storage device by the producer before sending the at least one message to the consumer.

2. The method according to claim 1, further comprising: Creating a distributed lock by a consumer based on the summary of the message data and the batch number; In response to the successful creation of the distributed lock, the consumer reads the task processing information.

3. The method according to claim 1, wherein: The summary of the message data includes a transmission type, and the distributed lock is created based on both the transmission type and the batch number.

4. The method according to claim 2, further comprising: It is up to the consumer to determine whether to start processing the message data from the breakpoint.

5. The method according to claim 3, further comprising: In response to the determination, the consumer starts processing the message data from the breakpoint, and continues processing the message data from the breakpoint; and The task processing information is updated by the consumer.

6. The method according to claim 3, further comprising: In response to determining not to start processing the message data from the breakpoint, the consumer initializes task processing information; The consumer processes the message data from the beginning; as well as The task processing information is updated by the consumer.

7. The method according to claim 5 or 6, further comprising: The consumer determines whether the processing of the message data is completed; and The consumer updates the task status in response to determining that the processing of the message data is completed.

8. The method according to claim 7, further comprising: The task status is read by the caller.

9. A computer-implemented system for asynchronous data transmission, comprising: Storage device; a producer coupled to the storage device, configured to store the generated message data in the storage device and to send at least one message to the consumer after storing the generated message data in the storage device, the at least one message including a summary of the message data, a batch number, and a total number of messages; and The consumer coupled to the storage device is configured to receive the at least one message and access the message data in the storage device according to the received at least one message.

10. A computer system comprising: one or more processors; and A memory coupled to the one or more processors, the memory storing computer-readable program instructions, which, when executed by the one or more processors, perform the method of any one of claims 1-8.

11. A computer-readable storage medium storing computer-readable program instructions, wherein the instructions, when executed by a processor, perform the method according to any one of claims 1 to 8.

Images