Construction method and system supporting multi-region distributed message queue

Abstract

The invention discloses a construction method and system supporting a multi-region distributed message queue, and the construction method comprises the steps: respectively deploying a plurality of message queue instances MQ and a message queue proxy MQ-Proxy in different region clusters; the message queue proxy MQ-Proxy of each regional cluster receives a production message delivered by a producer in a corresponding region, and delivers the production message to a corresponding message queue instance MQ for storage according to a preset rule; and the message queue proxy MQ-Proxy pulls the production message from the corresponding message queue instance MQ according to the consumption / subscription information of the consumer and returns the production message to the consumer for consumption. According to the method and the device, the production messages of producer service can be stored in the message queue instance MQ of each region in a balanced manner through the message queue proxy MQ-Proxy, and the storage pressure caused by partial data offset is avoided; and the message consumption requirements of consumer services in each region can be met, and the problem of message queue flow management in a multi-cloud and multi-region scene is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of message queues, and specifically provides a method and system for constructing a message queue that supports multi-region distributed messaging. Background Art

[0002] With the development of the Internet and the expansion of business, the current service architecture has evolved into a multi-cloud, multi-region distributed architecture system. In addition, in service communication, message queues are a crucial link. Message queues are a solution for transmitting messages between applications. They store messages in a queue and ensure the order and reliability of messages. Services need to rely on message queues to complete asynchronous, peak-shaving, queuing and other operations.

[0003] The mainstream message queue solutions on the market are RocketMQ and Kafka, which only support production and consumption functions within a single cluster and do not support message routing capabilities between multiple clusters.

[0004] In multi-cloud and multi-region scenarios, the production and consumption relationship under the message queue has also evolved to become increasingly complex. The producer of a message may be distributed in multiple clusters, and its consumers may also be distributed in multiple clusters. There may be a situation where there are producers but no consumers in a cluster, or there may be consumers but no producers.

[0005] In addition, if the consumers of a certain cluster are unavailable, consumers in other clusters will need to process the messages in the failed cluster. In order to improve the overall availability of the service, we need to schedule consumers in other clusters to process the traffic of the failed cluster.

[0006] In view of this, the present invention patent is proposed. Summary of the invention

[0007] In response to the above technical problems, the present invention proposes a method and system for building a distributed message queue that supports multiple regions, so as to solve the problem of message queue traffic management in multi-cloud and multi-region scenarios.

[0008] Specifically, the following technical solutions are adopted:

[0009] In a first aspect, the present invention provides a method for constructing a message queue supporting multiple regions, comprising:

[0010] Deploy multiple message queue instances MQ and a message queue agent MQ-Proxy in different regional clusters;

[0011] The message queue agent MQ-Proxy of each regional cluster receives the production messages delivered by the producers in the corresponding region, and delivers the production messages to the corresponding message queue instance MQ for storage according to the preset rules;

[0012] The message queue agent MQ-Proxy pulls the production message from the corresponding message queue instance MQ according to the consumer's consumption / subscription information and returns it to the consumer for consumption.

[0013] As an optional embodiment of the present invention, in a method for constructing a multi-region distributed message queue supporting the present invention, the message queue agent MQ-Proxy of each regional cluster receives the production message delivered by the producer in the corresponding region, and delivers the production message to the corresponding message queue instance MQ for storage according to preset rules, including:

[0014] The message queue agent MQ-Proxy of each regional cluster receives the production messages delivered by the producers in the corresponding region and arranges the received production messages into one or more data queues;

[0015] Divide the production message data in each data queue into multiple production message data blocks;

[0016] The production message data block block is delivered to the corresponding message queue instance MQ for storage according to the preset rules.

[0017] As an optional implementation of the present invention, in a method for constructing a multi-region distributed message queue supporting the present invention, the production message data block block is delivered to the corresponding message queue instance MQ for storage according to preset rules, wherein the preset rules are random strategies or hash strategies to satisfy the balance and sequentiality of the storage of the production message data.

[0018] As an optional implementation of the present invention, a method for constructing a message queue supporting multiple regions of the present invention includes:

[0019] Record the message queue instance MQ corresponding to each production message data block block in the data queue queue through the index service, and generate a production message data storage index;

[0020] The message queue agent MQ-Proxy obtains the message queue instance MQ storing the production message data block block in the data queue queue through the index service query.

[0021] As an optional implementation of the present invention, in a method for constructing a multi-region distributed message queue supporting the present invention, the message queue agent MQ-Proxy pulls the production message from the corresponding message queue instance MQ according to the consumer's consumption / subscription information and returns it to the consumer for consumption, including:

[0022] The message queue agent MQ-Proxy obtains consumption / subscription information within the same region cluster;

[0023] Get the target production message based on the consumer's consumption / subscription information;

[0024] Obtaining the target message queue instance MQ stored in the production message data block block containing the target production message in the data queue queue through the index service query;

[0025] The production message is pulled from the corresponding target message queue instance MQ and returned to the consumer for consumption.

[0026] As an optional implementation of the present invention, in a method for constructing a multi-region distributed message queue supporting the present invention, pulling the production message from the corresponding target message queue instance MQ and returning it to the consumer for consumption includes:

[0027] The message queue agent MQ-Proxy determines whether the target message queue instance MQ and the consumer belong to the same regional cluster according to the target message queue instance MQ stored in the acquired production message data block containing the target production message;

[0028] If the judgment result is yes, the message queue agent MQ-Proxy pulls the production message from the corresponding message queue instance MQ and returns it to the consumer for consumption;

[0029] If the judgment result is no, the message queue proxy MQ-Proxy of the regional cluster to which the consumer belongs sends a request to obtain the target message queue instance MQ to the message queue proxy MQ-Proxy of the regional cluster to which the corresponding target message queue instance MQ belongs;

[0030] The message queue agent MQ-Proxy of the regional cluster to which the target message queue instance MQ belongs obtains the production information data of the corresponding target message queue instance MQ according to the request, and returns it to the consumer for consumption through the message queue agent MQ-Proxy of the regional cluster to which the consumer belongs.

[0031] As an optional implementation of the present invention, a method for constructing a message queue supporting multiple regions of the present invention includes:

[0032] The message queue agent MQ-Proxy of each regional cluster records all regional information of data queue deployment;

[0033] The message queue agents MQ-Proxy in each regional cluster maintain communication and synchronize producer and subscriber information with each other regularly.

[0034] As an optional embodiment of the present invention, in a method for constructing a multi-region distributed message queue supporting the present invention, the message queue agents MQ-Proxy of each regional cluster maintain communication with each other, and synchronize producer and subscriber information with each other on a regular basis, including:

[0035] If the message queue proxy MQ-Proxy of regional cluster A obtains that there is a subscriber in the regional cluster that only consumes production messages stored in the message queue instance MQ of the regional cluster, the message queue proxy MQ-Proxy of regional cluster A adjusts the routing information of the data node and adds the data node information of the message queue instance MQ in other regional clusters;

[0036] Subscribers in regional cluster A can discover data nodes of message queue instances MQ in other regional clusters through new routing information in the message queue agent MQ-Proxy and initiate subscriptions.

[0037] In a second aspect, the present invention provides a system for supporting multi-region distributed message queues, comprising:

[0038] A message queue instance MQ module is deployed in different regional clusters, and multiple message queue instance MQ modules are deployed in each regional cluster;

[0039] A message queue proxy MQ-Proxy module is deployed in different regional clusters, and one message queue proxy MQ-Proxy is deployed in each regional cluster;

[0040] The message queue proxy MQ-Proxy module of each regional cluster receives the production messages delivered by the producers in the corresponding region, and delivers the production messages to the corresponding message queue instance MQ module for storage according to the preset rules;

[0041] The message queue agent MQ-Proxy module pulls the production message from the corresponding message queue instance MQ module according to the consumer's consumption / subscription information and returns it to the consumer for consumption.

[0042] In a third aspect, the present invention provides a computer-readable recording medium storing a computer-executable program, which, when executed, implements the method for constructing a message queue that supports multiple regions.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention provides a construction method that supports multi-region distributed message queues. Multiple message queue instances MQ and a message queue agent MQ-Proxy are deployed in different regional clusters. The message queue agent MQ-Proxy is responsible for accepting all message production and consumption operations. The producer service is connected to the message queue agent MQ-Proxy and delivers the message to the message queue agent MQ-Proxy. The message queue agent MQ-Proxy then delivers the message to the corresponding MQ data instance according to preset rules. Similarly, the consumer service is also connected to the message queue agent MQ-Proxy and pulls subsequent message data from the corresponding message queue instance MQ according to the current consumption position of the consumer and returns it to the consumer.

[0045] Therefore, the present invention provides a method for constructing a multi-region distributed message queue that supports the balanced storage of production messages of producer services in message queue instances MQ in each region through the message queue proxy MQ-Proxy, without causing storage pressure due to partial data offset; it can also meet the message consumption needs of consumer services in each region, and solve the problem of message queue traffic management in multi-cloud and multi-region scenarios.

[0046] The present invention provides a method for constructing a multi-region distributed message queue, wherein the message queue agent MQ-Proxy of each regional cluster respectively records all regional information of data queue queue deployment; the message queue agents MQ-Proxy of each regional cluster maintain communication and synchronize producer and subscriber information with each other at regular intervals.

[0047] Therefore, a construction method of the present invention that supports multi-region distributed message queues can realize production message routing capabilities and support automatic message closure in scenarios where service topology is incomplete without service awareness. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A system architecture diagram of a multi-region distributed message queue supporting embodiment 1 of the present invention;

[0049] Figure 2 A schematic diagram of a first embodiment of the present invention supporting the storage of production messages of multi-region distributed message queues in multiple message queue instances MQ;

[0050] Figure 3 A schematic diagram of a method of supporting multi-region distributed message queues to automatically route production messages in a multi-region cluster scenario according to a first embodiment of the present invention;

[0051] Figure 4 A schematic structural diagram of an electronic device according to a second embodiment of the present invention;

[0052] Figure 5 A schematic diagram of a computer-readable recording medium according to a second embodiment of the present invention. DETAILED DESCRIPTION

[0053] To make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be described clearly and completely in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them.

[0054] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the invention claimed for protection, but merely represents some embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions in the embodiments may be combined with each other.

[0056] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0057] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the invention product is usually placed when in use, or the orientation or positional relationship commonly understood by those skilled in the art. Such terms are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0058] Embodiment 1

[0059] See also Figure 1 As shown, a method for constructing a multi-region distributed message queue in this embodiment includes:

[0060] Deploy multiple message queue instances MQ and a message queue agent MQ-Proxy in different regional clusters;

[0061] The message queue agent MQ-Proxy of each regional cluster receives the production messages delivered by the producers in the corresponding region, and delivers the production messages to the corresponding message queue instance MQ for storage according to the preset rules;

[0062] The message queue agent MQ-Proxy pulls the production message from the corresponding message queue instance MQ according to the consumer's consumption / subscription information and returns it to the consumer for consumption.

[0063] A method for constructing a multi-region distributed message queue in this embodiment is to deploy multiple message queue instances MQ and a message queue agent MQ-Proxy in different regional clusters. The message queue agent MQ-Proxy is responsible for accepting all message production and consumption operations. The producer service will connect to the message queue agent MQ-Proxy and deliver the message to the message queue agent MQ-Proxy. The message queue agent MQ-Proxy will then deliver the message to the corresponding MQ data instance according to preset rules. Similarly, the consumer service will also connect to the message queue agent MQ-Proxy and pull subsequent message data from the corresponding message queue instance MQ according to the consumer's current consumption position and return it to the consumer.

[0064] Therefore, this embodiment provides a method for constructing a multi-region distributed message queue that supports the balanced storage of production messages of producer services in message queue instances MQ in each region through the message queue proxy MQ-Proxy, without causing storage pressure due to partial data offset; it can also meet the message consumption needs of consumer services in each region, and solve the problem of message queue traffic management in multi-cloud and multi-region scenarios.

[0065] As an optional implementation of this embodiment, see Figure 2 As shown, in a method for constructing a multi-region distributed message queue supporting the present embodiment, the message queue agent MQ-Proxy of each regional cluster receives the production message delivered by the producer in the corresponding region, and delivers the production message to the corresponding message queue instance MQ for storage according to the preset rules, including:

[0066] The message queue agent MQ-Proxy of each regional cluster receives the production messages delivered by the producers in the corresponding region, and arranges the received production messages into one or more data queues. When there are multiple data queues, the data volume of the production messages in the multiple data queues is the same or similar.

[0067] Divide the production message data in each data queue into multiple production message data blocks;

[0068] The production message data block block is delivered to the corresponding message queue instance MQ for storage according to the preset rules.

[0069] In this embodiment, a method for constructing a message queue that supports multi-region distributed messages is introduced to ensure the order of production message storage. The production messages are arranged in order to form a data queue. The production messages in the data queue are evenly divided into a number of production message data blocks, which are then allocated to the message queue instances MQ in each region for storage. Therefore, the data queue of this embodiment is a logical concept, which is composed of all production messages in the same queue, and the actual data is scattered in the message queue instances MQ in multiple regions.

[0070] Therefore, the method for constructing a multi-region distributed message queue in this embodiment allows the production messages to be evenly distributed on the data nodes through the above-mentioned design method, and will not cause uneven node data due to excessive production of message data in some queues.

[0071] As an optional implementation of this embodiment, in a method for constructing a multi-region distributed message queue supporting this embodiment, the production message data block block is delivered to the corresponding message queue instance MQ for storage according to preset rules, wherein the preset rules are random strategies or hash strategies to satisfy the balance and sequentiality of the storage of the production message data.

[0072] The random strategy of this embodiment randomly distributes the production message to the message queue instance MQ for storage each time. The random strategy needs to consider the balance of the production message data storage.

[0073] The hash strategy of this embodiment is mainly used for message routing and partitioning in the message queue. The key value of the message is calculated by the hash function, and the production message is distributed to different partitions. This ensures that the production messages are evenly distributed on the message queue instances MQ in different regional clusters, thereby improving the system's processing capacity and load balancing.

[0074] Specifically, the hash strategy of this embodiment can select a consistent hash strategy, a random hash strategy, and a range hash strategy:

[0075] Consistent hashing divides the hash ring into multiple virtual nodes, each node represents a partition. The key value of the new message is calculated through the hash function, and then mapped to a position on the hash ring. The corresponding partition is found based on the position. Consistent hashing can better handle the increase and decrease of nodes and reduce data migration caused by node changes.

[0076] When allocating messages, the random hash strategy performs hash calculation on the key value of each message and randomly selects a partition for storage. The random hash strategy is simple to implement and can better balance the load.

[0077] The range hash strategy maps the hash value to a specific range, and each range corresponds to a partition. The range hash strategy can ensure that key values ​​in the same range are always mapped to the same partition, which is easy to manage and query.

[0078] As an optional implementation of this embodiment, a method for constructing a message queue supporting multiple regions is provided in this embodiment, including:

[0079] Record the message queue instance MQ corresponding to each production message data block block in the data queue queue through the index service, and generate a production message data storage index;

[0080] The message queue agent MQ-Proxy obtains the message queue instance MQ storing the production message data block block in the data queue queue through the index service query.

[0081] This embodiment provides a storage index for production message data through an index service. In this way, when the consumer service consumes production messages, it can determine the message queue instance MQ where the production messages are actually stored through the index service, and thus pull the production messages from the corresponding message queue instance MQ for consumption.

[0082] Specifically, in a method for constructing a multi-region distributed message queue supporting the present embodiment, the message queue agent MQ-Proxy pulls the production message from the corresponding message queue instance MQ according to the consumer's consumption / subscription information and returns it to the consumer for consumption, including:

[0083] The message queue agent MQ-Proxy obtains consumption / subscription information within the same region cluster;

[0084] Get the target production message based on the consumer's consumption / subscription information;

[0085] Obtaining the target message queue instance MQ stored in the production message data block block containing the target production message in the data queue queue through the index service query;

[0086] The production message is pulled from the corresponding target message queue instance MQ and returned to the consumer for consumption.

[0087] Furthermore, in a method for constructing a multi-region distributed message queue supporting the present embodiment, the step of pulling the production message from the corresponding target message queue instance MQ and returning it to the consumer for consumption includes:

[0088] The message queue agent MQ-Proxy determines whether the target message queue instance MQ and the consumer belong to the same regional cluster according to the target message queue instance MQ stored in the acquired production message data block containing the target production message;

[0089] If the judgment result is yes, the message queue agent MQ-Proxy pulls the production message from the corresponding message queue instance MQ and returns it to the consumer for consumption;

[0090] If the judgment result is no, the message queue proxy MQ-Proxy of the regional cluster to which the consumer belongs sends a request to obtain the target message queue instance MQ to the message queue proxy MQ-Proxy of the regional cluster to which the corresponding target message queue instance MQ belongs;

[0091] The message queue agent MQ-Proxy of the regional cluster to which the target message queue instance MQ belongs obtains the production information data of the corresponding target message queue instance MQ according to the request, and returns it to the consumer for consumption through the message queue agent MQ-Proxy of the regional cluster to which the consumer belongs.

[0092] In this embodiment, a method for constructing a multi-region distributed message queue is provided. When a consumer service consumes production information, if the production information is stored in a message queue instance MQ in a regional cluster where the consumer service is deployed, the message queue proxy MQ-Proxy of the regional cluster can directly pull the production message in the corresponding message queue instance MQ and return it to the consumer service for consumption. If the production information is stored in a message queue instance MQ in a regional cluster other than the consumer service, the message queue proxy MQ-Proxy of the regional cluster can obtain the corresponding production message through the message queue proxy MQ-Proxy of the regional cluster where the message queue instance MQ is deployed and return it to the consumer service for consumption.

[0093] Therefore, the method for constructing a multi-region distributed message queue in this embodiment solves the problem of message queue traffic management in a multi-cloud and multi-region scenario.

[0094] As an optional implementation of this embodiment, a method for constructing a message queue supporting multiple regions is provided in this embodiment, including:

[0095] The message queue agent MQ-Proxy of each regional cluster records all regional information of data queue deployment;

[0096] The message queue agents MQ-Proxy in each regional cluster maintain communication and synchronize producer and subscriber information with each other regularly.

[0097] Furthermore, in a method for constructing a multi-region distributed message queue supporting the present embodiment, the message queue proxies MQ-Proxy of each regional cluster maintain communication with each other, and synchronize producer and subscriber information with each other on a regular basis, including:

[0098] If the message queue proxy MQ-Proxy of regional cluster A obtains that there is a subscriber in the regional cluster that only consumes production messages stored in the message queue instance MQ of the regional cluster, the message queue proxy MQ-Proxy of regional cluster A adjusts the routing information of the data node and adds the data node information of the message queue instance MQ in other regional clusters;

[0099] Subscribers in regional cluster A can discover data nodes of message queue instances MQ in other regional clusters through new routing information in the message queue agent MQ-Proxy and initiate subscriptions.

[0100] For specific examples, see Figure 3 As shown, a method for constructing a multi-region distributed message queue in this embodiment, how to realize automatic routing of messages in a regional cluster scenario:

[0101] Services will be deployed in multiple regions and clusters, but due to resource and dependency constraints, all services cannot be deployed equally in all regions and clusters. Figure 3 As shown in the figure, the consumer is deployed in regional cluster A, but the service is not deployed in regional cluster B. As a result, the production message will not be processed and will be lost.

[0102] To address this problem, our message queue implements the automatic routing capability of messages between regional clusters. On the message queue proxy MQ-Proxy of regional cluster A, we record all regions where the data queue is deployed, and maintain communication with the message queue proxy MQ-Proxy in other regions to synchronize producer and subscriber information regularly.

[0103] If the message queue agent MQ-Proxy of regional cluster A finds that some consumers only exist in this cluster and do not exist in other clusters, it will adjust the routing information of the data node and add the data node information of other clusters. Consumers can use the new routing information to find the data node of the message queue instance MQ in regional cluster B and initiate subscription.

[0104] Therefore, a method for constructing a multi-region distributed message queue in this embodiment can realize production message routing capabilities and support automatic message closure in scenarios where the service topology is incomplete without service awareness.

[0105] See also Figure 1As shown, this embodiment also provides a system supporting multi-region distributed message queues, including:

[0106] A message queue instance MQ module is deployed in different regional clusters, and multiple message queue instance MQ modules are deployed in each regional cluster;

[0107] A message queue proxy MQ-Proxy module is deployed in different regional clusters, and one message queue proxy MQ-Proxy is deployed in each regional cluster;

[0108] The message queue proxy MQ-Proxy module of each regional cluster receives the production messages delivered by the producers in the corresponding region, and delivers the production messages to the corresponding message queue instance MQ module for storage according to the preset rules;

[0109] The message queue agent MQ-Proxy module pulls the production message from the corresponding message queue instance MQ module according to the consumer's consumption / subscription information and returns it to the consumer for consumption.

[0110] A system supporting multi-region distributed message queues in this embodiment deploys multiple message queue instance MQ modules and a message queue agent MQ-Proxy module in different regional clusters. The message queue agent MQ-Proxy module is responsible for accepting all message production and consumption operations. The producer service will connect to the message queue agent MQ-Proxy module and deliver the message to the message queue agent MQ-Proxy module. The message queue agent MQ-Proxy module will then deliver the message to the corresponding message queue instance MQ module according to preset rules. Similarly, the consumer service will also connect to the message queue agent MQ-Proxy module and pull subsequent message data from the corresponding message queue instance MQ according to the consumer's current consumption position and return it to the consumer.

[0111] Therefore, a system supporting multi-region distributed message queues in this embodiment can realize balanced storage of production messages of producer services in message queue instance MQ modules in each region through the message queue proxy MQ-Proxy module, without causing storage pressure due to partial data offset; it can also meet the message consumption needs of consumer services in each region, solving the problem of message queue traffic management in multi-cloud and multi-region scenarios.

[0112] As an optional implementation of this embodiment, a system of this embodiment supporting multi-region distributed message queues, wherein the message queue agent MQ-Proxy module of each regional cluster receives the production message delivered by the producer in the corresponding region, and delivers the production message to the corresponding message queue instance MQ module for storage according to preset rules, including:

[0113] The message queue proxy MQ-Proxy module of each regional cluster receives the production messages delivered by the producers in the corresponding region, and arranges the received production messages into one or more data queues. When there are multiple data queues, the data volume of the production messages in the multiple data queues is the same or similar.

[0114] Divide the production message data in each data queue into multiple production message data blocks;

[0115] The production message data block is delivered to the corresponding message queue instance MQ module for storage according to the preset rules.

[0116] In this embodiment, a system supporting multi-region distributed message queues introduces the concept of data queues to arrange production messages in order to ensure the order of production message storage. Production messages in the data queues are evenly divided into several production message data blocks, which are then allocated to the message queue instance MQ modules of each region for storage. Therefore, the data queue of this embodiment is a logical concept, which is composed of all production messages in the same queue, and the actual data is scattered in the message queue instance MQ modules of multiple regions.

[0117] Therefore, a system supporting multi-region distributed message queues in this embodiment allows production messages to be evenly distributed on data nodes through the above-mentioned design method, and will not cause uneven node data due to excessive production of message data in some queues.

[0118] As an optional implementation of this embodiment, a system supporting multi-region distributed message queues in this embodiment includes:

[0119] An index service module records the message queue instance MQ module corresponding to each production message data block in the data queue queue, and generates a production message data storage index;

[0120] The message queue agent MQ-Proxy module queries and obtains the message queue instance MQ module storing the production message data block block in the data queue queue through the index service module.

[0121] This embodiment provides a storage index for production message data through an index service module. In this way, when the consumer service consumes production messages, it can determine the message queue instance MQ module where the production messages are actually stored through the index service module, and thus pull the production messages from the corresponding message queue instance MQ module for consumption.

[0122] Embodiment 2

[0123] The following describes an electronic device embodiment of the present invention, which can be regarded as a specific physical implementation of the method and device embodiments of the present invention. The details described in the electronic device embodiment of the present invention should be regarded as a supplement to the above method or device embodiments; details not disclosed in the electronic device embodiment of the present invention can be implemented with reference to the above method or device embodiments.

[0124] Figure 4 It is a structural diagram of an electronic device of an embodiment of the present invention, the electronic device includes a processor and a memory, the memory is used to store a computer executable program, when the computer program is executed by the processor, the processor executes a method for constructing a multi-region distributed message queue supporting embodiment one.

[0125] like Figure 4 As shown, the electronic device is presented in the form of a general computing device. The processor may be one or more and work in coordination. The present invention does not exclude distributed processing, that is, the processor may be dispersed in different physical devices. The electronic device of the present invention is not limited to a single entity, but may also be the sum of multiple physical devices.

[0126] The memory stores a computer executable program, which is usually a machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least part of the steps in the method.

[0127] The memory includes a volatile memory, such as a random access memory unit (RAM) and / or a cache memory unit, and may also be a non-volatile memory, such as a read-only memory unit (ROM).

[0128] Optionally, in this embodiment, the electronic device further includes an I / O interface, which is used for the electronic device to exchange data with an external device. The I / O interface can represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0129] It should be understood that Figure 4 The electronic device shown is only an example of the present invention, and the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as display screens, and some electronic devices also include human-computer interaction elements such as buttons, keyboards, etc. As long as the electronic device can execute the computer-readable program in the memory to implement the method of the present invention or at least part of the steps of the method, it can be considered as an electronic device covered by the present invention.

[0130] Figure 5 Schematic diagram of a computer readable recording medium according to an embodiment of the present invention. Figure 5 As shown, a computer executable program is stored in a computer-readable recording medium, and when the computer executable program is executed, a method for constructing a multi-region distributed message queue supporting the first embodiment of the present invention is implemented. The computer-readable recording medium may include a data signal propagated in a baseband or as part of a carrier, which carries a readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable recording medium may also be any readable medium other than a readable recording medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or device. The program code contained on the readable recording medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0131] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0132] Through the above description of the implementation mode, it is easy for those skilled in the art to understand that the present invention can be implemented by hardware capable of executing a specific computer program, such as the system of the present invention, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. contained in the system. The present invention can also be implemented by computer software that executes the method of the present invention, such as control software executed by a microprocessor, an electronic control unit, a client, a server, etc. However, it should be noted that the computer software that executes the method of the present invention is not limited to being executed by one or a specific hardware entity, and it can also be implemented in a distributed manner by unspecified specific hardware. For computer software, the software product can be stored in a computer-readable recording medium (which can be a CD-ROM, a USB flash drive, a mobile disk, etc.), and can also be distributed and stored on the network, as long as it enables the electronic device to execute the method according to the present invention.

[0133] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described in the present invention. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the above specific implementation methods. Therefore, any modification or equivalent replacement of the present invention; and all technical solutions and improvements thereof that do not depart from the spirit and scope of the invention are included in the scope of the claims of the present invention.

Claims

1. A method for constructing a distributed message queue supporting multiple regions, characterized in that: include: Deploy multiple message queue instances MQ and a message queue agent MQ-Proxy in different regional clusters; The message queue agent MQ-Proxy of each regional cluster receives the production messages delivered by the producers in the corresponding region, and delivers the production messages to the corresponding message queue instance MQ for storage according to the preset rules; The message queue agent MQ-Proxy pulls the production message from the corresponding message queue instance MQ according to the consumer's consumption / subscription information and returns it to the consumer for consumption.

2. According to claim 1, a method for constructing a message queue supporting multiple regions is characterized in that: The message queue agent MQ-Proxy of each regional cluster receives the production message delivered by the producer in the corresponding region, and delivers the production message to the corresponding message queue instance MQ for storage according to the preset rules, including: The message queue agent MQ-Proxy of each regional cluster receives the production messages delivered by the producers in the corresponding region and arranges the received production messages into one or more data queues; Divide the production message data in each data queue into multiple production message data blocks; The production message data block block is delivered to the corresponding message queue instance MQ for storage according to the preset rules.

3. According to claim 2, a method for constructing a message queue supporting multiple regions is characterized in that: The production message data block block is delivered to the corresponding message queue instance MQ for storage according to a preset rule, wherein the preset rule is a random strategy or a hash strategy to meet the balance and order of the production message data storage.

4. According to claim 2, a method for constructing a message queue supporting multiple regions is characterized in that: include: Record the message queue instance MQ corresponding to each production message data block block in the data queue queue through the index service, and generate a production message data storage index; The message queue agent MQ-Proxy obtains the message queue instance MQ storing the production message data block block in the data queue queue through the index service query.

5. According to claim 4, a method for constructing a message queue supporting multiple regions is characterized in that: The message queue agent MQ-Proxy pulls the production message from the corresponding message queue instance MQ according to the consumer's consumption / subscription information and returns it to the consumer for consumption, including: The message queue agent MQ-Proxy obtains consumption / subscription information within the same region cluster; Get the target production message based on the consumer's consumption / subscription information; Obtaining the target message queue instance MQ stored in the production message data block block containing the target production message in the data queue queue through the index service query; The production message is pulled from the corresponding target message queue instance MQ and returned to the consumer for consumption.

6. A method for constructing a message queue supporting multiple regions according to claim 5, characterized in that: Pulling the production message from the corresponding target message queue instance MQ and returning it to the consumer for consumption includes: The message queue agent MQ-Proxy determines whether the target message queue instance MQ and the consumer belong to the same regional cluster according to the target message queue instance MQ stored in the acquired production message data block containing the target production message; If the judgment result is yes, the message queue agent MQ-Proxy pulls the production message from the corresponding message queue instance MQ and returns it to the consumer for consumption; If the judgment result is no, the message queue proxy MQ-Proxy of the regional cluster to which the consumer belongs sends a request to obtain the target message queue instance MQ to the message queue proxy MQ-Proxy of the regional cluster to which the corresponding target message queue instance MQ belongs; The message queue agent MQ-Proxy of the regional cluster to which the target message queue instance MQ belongs obtains the production information data of the corresponding target message queue instance MQ according to the request, and returns it to the consumer for consumption through the message queue agent MQ-Proxy of the regional cluster to which the consumer belongs.

7. The method for constructing a message queue supporting multiple regions according to claim 5, characterized in that: include: The message queue agent MQ-Proxy of each regional cluster records all regional information of data queue deployment; The message queue agents MQ-Proxy in each regional cluster maintain communication and synchronize producer and subscriber information with each other regularly.

8. The method for constructing a message queue supporting multiple regions according to claim 7, characterized in that: The message queue agents MQ-Proxy of each regional cluster maintain communication with each other and synchronize producer and subscriber information with each other regularly, including: If the message queue proxy MQ-Proxy of regional cluster A obtains that there is a subscriber in the regional cluster that only consumes production messages stored in the message queue instance MQ of the regional cluster, the message queue proxy MQ-Proxy of regional cluster A adjusts the routing information of the data node and adds the data node information of the message queue instance MQ in other regional clusters; Subscribers in regional cluster A can discover data nodes of message queue instances MQ in other regional clusters through new routing information in the message queue agent MQ-Proxy and initiate subscriptions.

9. A system supporting multi-region distributed message queues, characterized in that: include: A message queue instance MQ module is deployed in different regional clusters, and multiple message queue instance MQ modules are deployed in each regional cluster; A message queue proxy MQ-Proxy module is deployed in different regional clusters, and one message queue proxy MQ-Proxy is deployed in each regional cluster; The message queue proxy MQ-Proxy module of each regional cluster receives the production messages delivered by the producers in the corresponding region, and delivers the production messages to the corresponding message queue instance MQ module for storage according to the preset rules; The message queue agent MQ-Proxy module pulls the production message from the corresponding message queue instance MQ module according to the consumer's consumption / subscription information and returns it to the consumer for consumption.

10. A computer-readable recording medium storing a computer-executable program, characterized in that: When the computer executable program is executed, a method for constructing a message queue supporting multiple regions as described in any one of claims 1 to 8 is implemented.

Images