Message delivery method and device, medium, electronic equipment and program product
By locating the message producer and the first message file system on the same physical machine and transmitting the message to the elastic block storage for storage, the message delivery delay and reliability problems in the microservice system are solved, and efficient and low-latency message delivery and data reliability are achieved.
Patent Information
- Application Number
- CN202510128720.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-27
- Publication Date
- 2025-05-02
AI Technical Summary
In the prior art, the message delivery delay in the microservice system is large and the data reliability is weak, making it difficult to meet the performance requirements of large-scale microservice systems.
By locating the message producer and the first message file system on the same physical machine and transmitting the messages to the elastic block storage for storage, the message delivery path is simplified, the transmission delay is reduced, and the data reliability is ensured using the multiple copies and high-performance characteristics of the elastic block storage.
It realizes very short message delivery delay, supports the performance requirements of massive data delivery, and ensures high data reliability, so as to directly carry services that have high requirements for data reliability.
Smart Images
Figure CN119922206A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular, to a message delivery method, device, medium, electronic device and program product. Background Art
[0002] At present, most large-scale business systems are deployed using a microservice architecture. Such business systems usually have many types of microservices (for example, 50 or even more) and a large number of microservice instances. Therefore, the delivery of messages generated by microservices becomes a performance bottleneck on the main business path. However, the message delivery delay of mainstream message delivery tools in the industry (for example, Rocket MQ, Kafka) is relatively large, and the reliability of message data is relatively weak. Summary of the invention
[0003] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.
[0004] In a first aspect, the present disclosure provides a message delivery method, comprising: The message producer transmits the message generated by the microservice to the first message file system, wherein the message producer and the first message file system are located on the same physical machine; The first message file system transmits the message transmitted by the message producer to the elastic block storage; The elastic block storage stores the received messages.
[0005] In a second aspect, the present disclosure provides a message delivery device, including a message producer, a first message file system, and an elastic block storage, wherein: The message producer is used to transmit the message generated by the microservice to the first message file system, wherein the message producer and the first message file system are located on the same physical machine; The first message file system is used to transmit the message transmitted by the message producer to the elastic block storage; The elastic block storage is used to store received messages.
[0006] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which implements the steps of any one of the methods in the first aspect when executed by a processing device.
[0007] In a fourth aspect, the present disclosure provides an electronic device, including: a storage device having a computer program stored thereon; A processing device is used to execute a computer program in a storage device to implement the steps of any method in the first aspect.
[0008] In a fifth aspect, the present disclosure provides a computer program product, comprising a computer program, which implements the steps of any one of the methods in the first aspect when executed by a processor.
[0009] By adopting the above technical solution, the message producer transmits the message generated by the microservice to the first message file system, wherein the message producer and the first message file system are located on the same physical machine, and then the first message file system transmits the message transmitted by the message producer to the elastic block storage, and then the elastic block storage stores the received message. Therefore, on the one hand, since the message delivery path is "message producer → first message file system → elastic block storage", the message producer and the first message file system are located on the same physical machine, and the transmission delay between them is very short, and the elastic block storage has the inherent advantages of being close to the business service, high performance and multiple copies, and can provide low-latency storage services, so the entire message delivery link is very short, and can provide a very short message delivery delay, thereby being able to support the performance requirements of massive data delivery; on the other hand, since the elastic block storage can be used to achieve persistent storage, data reliability can be ensured, so that it can directly carry businesses with high requirements for data reliability.
[0010] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 A schematic diagram showing the message delivery process of the existing RMQ is shown.
[0012] Figure 2 A schematic diagram showing the existing Kafka message delivery process.
[0013] Figure 3 It is a flowchart of a message delivery method according to an embodiment of the present disclosure.
[0014] Figure 4 This is another flow chart of a message delivery method according to an embodiment of the present disclosure.
[0015] Figure 5This is another flow chart of a message delivery method according to an embodiment of the present disclosure.
[0016] Figure 6 This is another flow chart of a message delivery method according to an embodiment of the present disclosure.
[0017] Figure 7 It is an architectural diagram of a message delivery architecture according to an embodiment of the present disclosure.
[0018] Figure 8 It is a schematic block diagram of a message delivery device according to an embodiment of the present disclosure.
[0019] Fig. 9 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION
[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0021] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0022] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.
[0023] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0024] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0025] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0026] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0027] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
[0028] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0029] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0030] At the same time, it is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.
[0031] The mainstream message delivery tools in the industry mainly include Rocket Message Queue (RMQ), Kafka, etc.
[0032] Figure 1 FIG. 1 shows a schematic diagram of the message delivery process of the existing RMQ. Figure 1As shown in the figure, the message producer (Producer) transmits the message generated by the microservice to the RMQ proxy (Proxy) through the general hardware of the Transfer Control Protocol (TCP) / Internet Protocol (IP). Then, the RMQ proxy passes the message to the message server (Broker) through the general hardware of the TCP / IP network protocol, and the message is persistently stored on the message server. The delay overhead of the entire message delivery process . The delay for a message producer to send a message to a message server, which mainly includes the delay from the message producer to the RMQ agent and the delay from the RMQ agent to the message server. The empirical average value is between 5ms and 30ms. The delay in persisting a message after the message server receives it. For a complex microservice system, the impact of introducing an RMQ message queue on system performance depends on the scale of the system. The larger the scale, the greater the impact. For latency-sensitive businesses, the above delay is unacceptable.
[0033] In addition, the data received by RMQ is stored in a local single copy and master-slave asynchronous synchronization mode, which has low data reliability and its recovery point objective (RPO) is greater than 0. In other words, RMQ's asynchronous replication cannot guarantee data reliability with RPO=0.
[0034] Figure 2 The following is a schematic diagram showing the message delivery process of existing Kafka. Figure 2 As shown in the figure, the message producer transmits the message generated by the microservice to the Kafka agent through the general TCP / IP network protocol general hardware, and then the Kafka agent passes the message to the Kafka server as the leader through the general TCP / IP network protocol general hardware. Then, the Kafka server as the leader copies the message to the Kafka server as the follower, where there are generally two Kafka servers as followers. The delay overhead of the entire message delivery process . The latency for a message producer to send a message to the leader Kafka server. This latency mainly includes the latency from the message producer to the Kafka proxy and the latency from the Kafka proxy to the leader Kafka server. The empirical value is similar to that of RMQ, also 5ms to 30ms. The delay for the Kafka server as the leader to persist the message after receiving it. ,in, It is the latency overhead of the local persistent storage of the Kafka server as the leader. It is the latency overhead caused by the Kafka server as the leader replicating messages to the Kafka server as the follower. The empirical value is 5ms to 30ms. Therefore, the total latency is 8ms to 45ms. For complex microservice systems, the impact of introducing Kafka message queues on system performance depends on the scale of the system. The larger the scale, the greater the impact. For latency-sensitive businesses, a latency of 8ms to 45ms is unacceptable.
[0035] In addition, Kafka enhances data reliability through Y-type Qourum writing, but because it does not use dedicated network protocols and dedicated high-speed hardware, the data reliability is low.
[0036] Therefore, whether it is RMQ or Kafka, the delivery of its messages needs to go through "message producer → message agent → message server → disk persistence", so it cannot meet the requirements of large-scale microservice systems in terms of message delivery delay performance and data reliability.
[0037] Figure 3 1 is a flow chart of a message delivery method according to an embodiment of the present disclosure. The message delivery method can be applied to scenarios requiring massive data delivery, such as large-scale microservice systems. Figure 3 As shown, the message delivery method may include the following steps S31 to S33.
[0038] In step S31, the message producer transmits the message generated by the microservice to the first message file system, wherein the message producer and the first message file system are located on the same physical machine.
[0039] The message producer can be integrated and used by the microservice process, for example, embedded in the microservice process to provide message delivery capabilities. The message producer can be visible to the user.
[0040] The first message file system is used to provide read and write capabilities, such as sequential read and write capabilities. The first message file system may be a file system constructed based on block storage and object storage.
[0041] The message producer and the first message file system can be located on the same virtual machine of the physical machine where they are located. The message producer and the first message file system can communicate through a domain socket, that is, they can use the built-in transmission channel of the operating system to communicate, so that the transmission delay between the message producer and the first message file system is at the level of 5μs.
[0042] In step S32, the first message file system transmits the message transmitted by the message producer to the elastic block storage.
[0043] Elastic Block Storage is a public cloud Infrastructure as a Service (IaaS) product that provides low-latency (e.g., hundreds of microseconds) and multi-replica block storage services. Elastic Block Storage can be used for persistent storage of various data, such as persistent storage of message Write-Ahead Logging (WAL).
[0044] The first message file system can connect to the elastic block storage through the block storage driver on the physical machine where it is located, so that the message transmitted by the message producer can be transmitted to the elastic block storage. The block storage driver can be connected to the elastic block storage through a dedicated high-performance network protocol and dedicated high-performance hardware. For example, the first message file system can directly write messages to the elastic block storage through the block device in the virtual machine where it is located, so as to achieve persistent storage of messages, and the transmission delay is at the level of 200μs. The standard POSIX semantic file reading and writing method can be used between the first message file system and the block storage driver, and the delay is very short.
[0045] In step S33, the elastic block storage stores the received message, for example, in a persistent storage.
[0046] As the persistence layer for message delivery, elastic block storage can provide read and write latency of hundreds of microseconds and 99.9999999999% data reliability.
[0047] After receiving the message, the elastic block storage may return confirmation information to the message producer, for example, by sequentially returning the confirmation information to the message producer via the block storage driver and the first message file system. This operation is optional.
[0048] By adopting the above technical solution, the message producer transmits the message generated by the microservice to the first message file system, wherein the message producer and the first message file system are located on the same physical machine, and then the first message file system transmits the message transmitted by the message producer to the elastic block storage, and then the elastic block storage stores the received message. Therefore, on the one hand, since the message delivery path is "message producer → first message file system → elastic block storage", the message producer and the first message file system are located on the same physical machine, and the transmission delay between them is very short. Elastic block storage has the inherent advantages of being close to business services, high performance and multiple copies, and can provide low-latency storage services. Therefore, the entire message delivery link is very short, and can provide very short message delivery delay, thereby supporting the performance requirements of massive data delivery. For example, in the present disclosure, the delay between the message producer and the first message file system is about 5μs, and the delay between the first message file system and the elastic block storage is about 200μs. Therefore, the end-to-end message delivery delay can be controlled at 300μs, which is 16-100 times shorter than 5ms~30ms in the related art. On the other hand, since elastic block storage can be used to achieve persistent storage, data reliability can be ensured, so that services with high requirements for data reliability can be directly carried. In addition, after the elastic block storage is bound to the virtual machine, a fate sharing architecture can be formed, and there is no need to consider the disaster recovery and migration of messages separately.
[0049] Figure 4 This is another flow chart of a message delivery method according to an embodiment of the present disclosure.
[0050] like Figure 4 As shown, first, the message producer transmits the message generated by the microservice to the first message file system.
[0051] Then, the first message file system requests a unique sequence number for the message transmitted by the message producer from the sequence number generator.
[0052] The sequence number generator can provide a number generator service, which can provide globally consistent sequence number generation capabilities based on, for example, the raft protocol. With the help of unique sequence numbers, the order-preserving message delivery mode can be implemented.
[0053] The sequence number generator can be located on the same physical machine as the first message file system, or can be located on the same virtual machine of the same physical machine as the first message file system. The sequence number generator and the first message file system can communicate with each other through domain sockets. This can make the communication delay between them very short. Of course, it is also feasible for the sequence number generator and the first message file system to be located on different physical machines.
[0054] Then, the sequence number generator generates a unique sequence number based on the request of the first message file system, and sends the generated unique sequence number to the first message file system.
[0055] Then, the first message file system transmits the unique sequence number sent by the sequence number generator and the message transmitted by the message producer to the elastic block storage.
[0056] The elastic block storage then stores the received message together with the unique sequence number, such as in persistent storage.
[0057] In addition, after receiving the message, the elastic block storage can return confirmation information to the message producer. This operation is optional.
[0058] By adopting the above technical solution, on the one hand, the entire message delivery link is very short, which can provide a very short message delivery delay, and thus can support the performance requirements of massive data delivery; on the other hand, since elastic block storage can be used to achieve persistent storage, data reliability can be ensured, so that businesses with high requirements for data reliability can be directly carried; on the other hand, since each message has its own unique serial number, an order-preserving message delivery mode can be implemented.
[0059] Figure 5 This is another flow chart of a message delivery method according to an embodiment of the present disclosure.
[0060] like Figure 5 As shown, first, the message merger sends a first message reading request to the second message file system.
[0061] The message merger is used to provide message merger services, for example, to merge messages of the same stream in the background. The message merger can be invisible to users.
[0062] The message merger can be a separate process running in the container.
[0063] The message merger and the second message file system may be located on the same physical machine, or on the same virtual machine of the same physical machine. The message merger and the second message file system may communicate with each other through a domain socket.
[0064] The second message file system can be the same as or different from the first message file system. If the second message file system is the same as the first message file system, that is, they are the same message file system, the message merger and the message producer can share the message file system service of the same virtual machine of the same physical machine. For the message merger and the message producer, the corresponding message file systems are mounted on elastic block storage in a read-write manner.
[0065] The first message reading request may carry information about the message file to be read and information about the messages in the message file, so that the second message file system can know which messages in which message file need to be read.
[0066] Then, the second message file system reads the corresponding message from the elastic block storage based on the first message read request, and returns the read message to the message merger.
[0067] The message merger then merges the returned messages and stores the merged messages in the object store.
[0068] For example, the message merger can merge the messages that belong to the same stream in the returned messages.
[0069] Object storage is a public cloud IaaS product that can provide low-cost, unstructured, and massive storage services. By persistently storing merged messages in object storage, it can provide massive and low-cost distributed storage capabilities.
[0070] The message merger can store the merged message in the object storage in the following way: the message merger sends an archiving request to the second message file system, and the archiving request carries the merged message. Then the second message file system sends the received merged message to the object storage, so that the merged message can be persistently stored in the object storage.
[0071] In addition, after receiving the merged message, the object storage can return a confirmation message to the message merger. This operation is optional.
[0072] By adopting the above technical solution, the messages can be merged and the merged messages can be persistently stored in the object storage, which can reduce the storage cost. In addition, since the entire link is very short, the message delivery delay is very short.
[0073] Figure 6 This is another flow chart of a message delivery method according to an embodiment of the present disclosure.
[0074] like Figure 6 As shown, first, the message consumer sends a second message reading request to the third message file system.
[0075] Message consumers can be integrated and used by microservice processes to provide message reading capabilities. Message consumers can be visible to users.
[0076] The message consumer and the third message file system may be located on the same physical machine, or on the same virtual machine of the same physical machine. The message consumer and the third message file system may communicate with each other through a domain socket.
[0077] The third message file system can be the same as or different from the first message file system. If the third message file system is the same as the first message file system, that is, they are the same message file system, the message consumer and the message producer can share the message file system service of the same virtual machine of the same physical machine. For the message consumer, the corresponding message file system is mounted in elastic block storage in read-only mode.
[0078] The third message file system then determines whether the object store is empty.
[0079] Then, if the object storage is not empty, the third message file system reads the message from the object storage based on the second message read request, and returns the read message to the message consumer. If the object storage is empty, the third message file system reads the message from the elastic block storage based on the second message read request, and returns the read message to the message consumer.
[0080] Then, the message consumer consumes the message returned by the third message file system.
[0081] By adopting the above technical solution, the consumption of persistently stored messages can be realized. Moreover, since the entire link is very short, the overall delay is very short.
[0082] Figure 7 is an architectural diagram of a message delivery architecture according to an embodiment of the present disclosure. Figure 7 As shown in FIG. 1 , the message delivery architecture includes a message producer, a message merger, a message consumer, a message file system, a block storage driver, an elastic block storage, an object storage, a sequence number generator, a local file system, and a local solid-state drive (SSD). The message producer and the message merger are located on the same virtual machine of the same physical machine; the message consumer and the sequence number generator are located on physical machines different from the physical machine where the message producer is located.
[0083] The specific implementation methods of the operations performed by the above-mentioned components have been described in detail in the previous text and will not be repeated here.
[0084] Through this message delivery architecture, on the one hand, the entire message delivery link is very short, which can provide a very short message delivery delay, and thus can support the performance requirements of massive data delivery; on the other hand, since elastic block storage can be used to achieve persistent storage, data reliability can be ensured, so that businesses with high requirements for data reliability can be directly carried; on the other hand, since each message has its own unique serial number, an order-preserving message delivery mode can be implemented.
[0085] Figure 8 1 is a schematic block diagram of a message delivery device according to an embodiment of the present disclosure. The message delivery device can be applied to scenarios requiring massive data delivery, such as large-scale microservice systems. Figure 8 As shown, the message delivery device 80 includes a message producer 81, a first message file system 82 and an elastic block storage 83, wherein: the message producer 81 is used to transmit the message generated by the microservice to the first message file system 82, wherein the message producer 81 and the first message file system 82 are located on the same physical machine; the first message file system 82 is used to transmit the message transmitted by the message producer to the elastic block storage 83; the elastic block storage 83 is used to store the received message.
[0086] By adopting the above technical solution, the message producer transmits the message generated by the microservice to the first message file system, wherein the message producer and the first message file system are located on the same physical machine, and then the first message file system transmits the message transmitted by the message producer to the elastic block storage, and then the elastic block storage stores the received message. Therefore, on the one hand, since the message delivery path is "message producer → first message file system → elastic block storage", the message producer and the first message file system are located on the same physical machine, and the transmission delay between them is very short. Elastic block storage has the inherent advantages of being close to business services, high performance and multiple copies, and can provide low-latency storage services. Therefore, the entire message delivery link is very short, and can provide very short message delivery delay, thereby supporting the performance requirements of massive data delivery. For example, in the present disclosure, the delay between the message producer and the first message file system is about 5μs, and the delay between the first message file system and the elastic block storage is about 200μs. Therefore, the end-to-end message delivery delay can be controlled at 300μs, which is 16-100 times shorter than 5ms~30ms in the related art. On the other hand, since elastic block storage can be used to achieve persistent storage, data reliability can be ensured, so that services with high requirements for data reliability can be directly carried. In addition, after the elastic block storage is bound to the virtual machine, a fate sharing architecture can be formed, and there is no need to consider the disaster recovery and migration of messages separately.
[0087] Optionally, the first message file system 82 also requests a unique serial number for the message transmitted by the message producer from a serial number generator (not shown); the serial number generator generates a unique serial number based on the request and sends the generated unique serial number to the first message file system 82; the first message file system 82 also transmits the unique serial number sent by the serial number generator and the message transmitted by the message producer to the elastic block storage 83; the elastic block storage 83 stores the received message and the unique serial number together.
[0088] Optionally, the message delivery device 80 also includes a message merger (not shown), wherein the message merger sends a first message reading request to a second message file system (not shown), wherein the message merger and the second message file system are located on the same physical machine, and the second message file system is the same as or different from the first message file system 82; the second message file system reads the corresponding message from the elastic block storage 83 based on the first message reading request, and returns the read message to the message merger; the message merger merges the returned messages and stores the merged messages in the object storage.
[0089] Optionally, the message merger further merges messages belonging to the same stream among the returned messages.
[0090] Optionally, the message delivery device further includes a message consumer (not shown), wherein: The message consumer sends a second message read request to a third message file system (not shown), wherein the message consumer and the third message file system are located on the same physical machine, and the third message file system is the same as or different from the first message file system 82; The third message file system determines whether the object storage is empty; If the object storage is not empty, the third message file system reads the message from the object storage based on the second message read request, and returns the read message to the message consumer; The message consumer consumes the message returned by the third message file system.
[0091] Optionally, if the object storage is empty, the third message file system reads the message from the elastic block storage 83 based on the second message read request, and returns the read message to the message consumer.
[0092] Optionally, the message producer and the first message file system 82 are located on the same virtual machine of the physical machine where they are located; the message merger and the second message file system are located on the same virtual machine of the physical machine where they are located; the message consumer and the third message file system are located on the same virtual machine of the physical machine where they are located.
[0093] The specific implementation methods of the operations performed by each module in the message delivery device according to the embodiment of the present disclosure have been described in detail in the relevant methods and will not be repeated here.
[0094] The present disclosure also provides a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processing device, the steps of any method in the present disclosure are implemented.
[0095] The present disclosure also provides an electronic device, comprising: a storage device having a computer program stored thereon; A processing device is used to execute a computer program in a storage device to implement the steps of any method in the present disclosure.
[0096] The present disclosure also provides a computer program product, comprising a computer program, which implements the steps of any method in the present disclosure when executed by a processor.
[0097] Reference below Fig. 9 , which shows a schematic diagram of the structure of an electronic device 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0098] like Fig. 9 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0099] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.
[0100] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0101] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0102] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0103] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0104] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: the message producer transmits the message generated by the microservice to the first message file system, wherein the message producer and the first message file system are located on the same physical machine; the first message file system transmits the message transmitted by the message producer to the elastic block storage; the elastic block storage stores the received message.
[0105] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0106] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0107] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of the module does not limit the module itself in some cases. For example, the message producer may also be described as "a module that transmits the message generated by the microservice to the first message file system".
[0108] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0109] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0110] According to one or more embodiments of the present disclosure, Example 1 provides a message delivery method, including: The message producer transmits the message generated by the microservice to the first message file system, wherein the message producer and the first message file system are located on the same physical machine; The first message file system transmits the message transmitted by the message producer to the elastic block storage; The elastic block storage stores the received messages.
[0111] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein the method further comprises: the first message file system requests a sequence number generator for a unique sequence number for a message transmitted by the message producer; the sequence number generator generates a unique sequence number based on the request and sends the generated unique sequence number to the first message file system; The first message file system transmits the message transmitted by the message producer to the elastic block storage, including: The first message file system transmits the unique sequence number sent by the sequence number generator and the message transmitted by the message producer to the elastic block storage; The elastic block storage stores the received message, including: The elastic block storage stores the received message together with the unique sequence number.
[0112] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 1 or 2, wherein the method further includes: The message merger sends a first message read request to a second message file system, wherein the message merger and the second message file system are located on the same physical machine, and the second message file system is the same as or different from the first message file system; The second message file system reads the corresponding message from the elastic block storage based on the first message read request, and returns the read message to the message merger; The message merger merges the returned messages and stores the merged messages in the object storage.
[0113] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 3, wherein the message merger merges the returned messages, including: The message merger merges the messages belonging to the same stream among the returned messages.
[0114] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 3, wherein the method further includes: The message consumer sends a second message read request to a third message file system, wherein the message consumer and the third message file system are located on the same physical machine, and the third message file system is the same as or different from the first message file system; The third message file system determines whether the object storage is empty; If the object storage is not empty, the third message file system reads the message from the object storage based on the second message read request, and returns the read message to the message consumer; The message consumer consumes the message returned by the third message file system.
[0115] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 5, wherein the method further includes: If the object storage is empty, the third message file system reads the message from the elastic block storage based on the second message read request, and returns the read message to the message consumer.
[0116] According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 5, wherein: The message producer and the first message file system are located on the same virtual machine of the physical machine where they are located; The message merger and the second message file system are located on the same virtual machine of the physical machine where they are located; The message consumer and the third message file system are located on the same virtual machine of the physical machine where they are located.
[0117] According to one or more embodiments of the present disclosure, Example 8 provides a message delivery device, including a message producer, a first message file system, and an elastic block storage, wherein: The message producer is used to transmit the message generated by the microservice to the first message file system, wherein the message producer and the first message file system are located on the same physical machine; The first message file system is used to transmit the message transmitted by the message producer to the elastic block storage; The elastic block storage is used to store received messages.
[0118] According to one or more embodiments of the present disclosure, Example 9 provides a computer-readable medium having a computer program stored thereon, which implements the steps of any of the methods described in Examples 1-7 when executed by a processing device.
[0119] According to one or more embodiments of the present disclosure, Example 10 provides an electronic device, including: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of any one of the methods described in Examples 1-7.
[0120] According to one or more embodiments of the present disclosure, Example 11 provides a computer program product, including a computer program, which implements the steps of any one of the methods of Examples 1-7 when executed by a processor.
[0121] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.
[0122] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0123] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Claims
1. A message delivery method, characterized in that: include: The message producer transmits the message generated by the microservice to the first message file system, wherein the message producer and the first message file system are located on the same physical machine; The first message file system transmits the message transmitted by the message producer to the elastic block storage; The elastic block storage stores the received messages.
2. The method according to claim 1, characterized in that: The method further comprises: The first message file system requests a unique sequence number for a message transmitted by the message producer from a sequence number generator; The sequence number generator generates a unique sequence number based on the request and sends the generated unique sequence number to the first message file system; The first message file system transmits the message transmitted by the message producer to the elastic block storage, including: The first message file system transmits the unique sequence number sent by the sequence number generator and the message transmitted by the message producer to the elastic block storage; The elastic block storage stores the received message, including: The elastic block storage stores the received message together with the unique sequence number.
3. The method according to claim 1 or 2, characterized in that: The method further comprises: The message merger sends a first message read request to a second message file system, wherein the message merger and the second message file system are located on the same physical machine, and the second message file system is the same as or different from the first message file system; The second message file system reads the corresponding message from the elastic block storage based on the first message read request, and returns the read message to the message merger; The message merger merges the returned messages and stores the merged messages in the object storage.
4. The method according to claim 3, characterized in that The message merger merges the returned messages, including: The message merger merges the messages belonging to the same stream among the returned messages.
5. The method according to claim 3, characterized in that: The method further comprises: The message consumer sends a second message read request to a third message file system, wherein the message consumer and the third message file system are located on the same physical machine, and the third message file system is the same as or different from the first message file system; The third message file system determines whether the object storage is empty; If the object storage is not empty, the third message file system reads the message from the object storage based on the second message read request, and returns the read message to the message consumer; The message consumer consumes the message returned by the third message file system.
6. The method according to claim 5, characterized in that The method further comprises: If the object storage is empty, the third message file system reads the message from the elastic block storage based on the second message read request, and returns the read message to the message consumer.
7. The method according to claim 5, characterized in that The message producer and the first message file system are located on the same virtual machine of the physical machine where they are located; The message merger and the second message file system are located on the same virtual machine of the physical machine where they are located; The message consumer and the third message file system are located on the same virtual machine of the physical machine where they are located.
8. A message delivery device, characterized in that: The system includes a message producer, a first message file system and elastic block storage, wherein: The message producer is used to transmit the message generated by the microservice to the first message file system, wherein the message producer and the first message file system are located on the same physical machine; The first message file system is used to transmit the message transmitted by the message producer to the elastic block storage; The elastic block storage is used to store received messages.
9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 7 are implemented.
10. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 7.
11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.