Message processing method and device and machine readable storage medium

Abstract

The invention discloses a message processing method and device and a machine readable storage medium, and relates to the technical field of computers. Determining a master node from all target consumption nodes based on a preset master node determination rule, the target consumption nodes being all consumption nodes that have subscribed to the target theme in the message-oriented middleware; the to-be-consumed message in the target theme is consumed through the main node; determining slave nodes in all consumption nodes based on the node registration information and a distribution rule in the to-be-consumed message; and sending data including the to-be-consumed message to each slave node through the master node, so that each slave node consumes the to-be-consumed message. The to-be-consumed message of the message middleware is consumed through the master node, and the to-be-consumed message is distributed to the slave node through the master node, so that parallel consumption can be realized without specifying different consumption group IDs for each node, the accuracy of message consumption is improved, repeated consumption is avoided, and the efficiency and flexibility of message processing are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a message processing method, apparatus, and machine-readable storage medium. Background Art

[0002] In a distributed architecture, in order to reduce coupling and improve system concurrency, a message middleware is usually introduced, and data interaction between services is asynchronously processed through the message middleware. In a cluster deployment, a service can include multiple identical running nodes, and the message middleware will ensure that the same message is consumed by only one of the running nodes, thus avoiding duplicate consumption. However, in some special scenarios, for example, subscribing to database change events to refresh the local cache, multiple running nodes need to consume the same message in parallel.

[0003] The message middleware consumes messages using the consumer group ID (Identity document) as the uniqueness identifier, that is, the same message is consumed only once under a consumer group. Usually, multiple nodes deployed for a service are configured with the same consumer group ID to ensure that a message is consumed only once. If parallel consumption is required, different consumer group IDs need to be assigned to each node. Assigning different consumer group IDs to each node can be achieved by manual specification, random generation, or incremental sequence, etc. However, the manual specification method in the above methods cannot achieve automated deployment, and the random generation and incremental sequence methods are prone to causing a node to consume the same message repeatedly. Therefore, how to optimize the multi-node parallel consumption method and avoid duplicate consumption by nodes has become a widely concerned issue currently. Summary of the Invention

[0004] In view of the above deficiencies in the prior art, the purpose of the embodiments of this application is to provide a message processing method, apparatus, and machine-readable storage medium.

[0005] To achieve the above purpose, the first aspect of this application provides a message processing method, including:

[0006] Determine a master node from all target consumer nodes based on a preset master node determination rule, where the target consumer nodes are all consumer nodes that have subscribed to the target topic in the message middleware;

[0007] Consume the message to be consumed in the target topic through the master node;

[0008] Determine slave nodes among all consumer nodes based on the node registration information and the distribution rule in the message to be consumed;

[0009] Send data including the message to be consumed to each slave node through the master node, so that each slave node consumes the message to be consumed.

[0010] In the embodiments of the present application, before the step of determining the master node based on the preset master node determination rule, the following steps are further included:

[0011] Determine the target consumer nodes and the corresponding first target topic and second target topic based on the preset message processing requirements, where the first target topic is the topic in the message middleware, and the second target topic is the topic in the local memory;

[0012] Control each target consumer node to subscribe to the first target topic and the second target topic simultaneously.

[0013] In the embodiments of the present application, sending data including the message to be consumed to each slave node through the master node to enable each slave node to consume the message to be consumed includes:

[0014] Send data including the message to be consumed to the message receiving interfaces of each slave node through the master node;

[0015] Trigger the corresponding local memory through the message receiving interface to publish the message to be consumed to the second target topic, so that each slave node consumes the message to be consumed.

[0016] In the embodiments of the present application, determining the slave nodes among all consumer nodes based on the distribution rule and node registration information in the message to be consumed includes:

[0017] Determine the first consumer nodes in the same consumer group as the master node among all consumer nodes based on the node registration information;

[0018] Determine the slave nodes in the first consumer nodes based on the distribution rule in the message to be consumed, where the distribution rule includes the demand for slave nodes and / or load balancing.

[0019] In the embodiments of the present application, sending data including the message to be consumed to each slave node through the master node to enable each slave node to consume the message to be consumed includes:

[0020] Send data including the message to be consumed to each slave node through the master node;

[0021] For each slave node, determine whether the master node and the slave node are in the same consumer group based on the node registration information and the master node information in the data;

[0022] If the master node and the slave node are in the same consumer group, consume the message to be consumed through the slave node.

[0023] In the embodiments of the present application, after the step of sending data including the message to be consumed to each slave node through the master node to enable each slave node to consume the message to be consumed, the following steps are further included:

[0024] Monitor whether each slave node has fed back the reception success information based on the preset delay duration;

[0025] If there is a target slave node that has not fed back the successful reception information, the master node resends the data to the target slave node until the number of transmissions reaches a preset number of times.

[0026] In the embodiments of the present application, the message processing method further includes:

[0027] In the case where the master node goes offline, a master node is re-determined based on a preset master node determination rule, where the preset master node determination rule includes load balancing.

[0028] In the embodiments of the present application, the message processing method further includes:

[0029] In the case where there is a new addition or offline of a consumption node, the node registration information is updated.

[0030] A second aspect of the present application provides a message processing apparatus, including:

[0031] A memory configured to store instructions;

[0032] A processor configured to call instructions from the memory and capable of implementing the message processing method as described in the above embodiments when executing the instructions.

[0033] A third aspect of the present application provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to cause a machine to execute the message processing method as described in the above embodiments.

[0034] Through the above technical solutions, a master node is determined from all target consumption nodes based on a preset master node determination rule, where the target consumption nodes are all consumption nodes that have subscribed to the target topic in the message middleware; the master node consumes the messages to be consumed in the target topic; slave nodes among all consumption nodes are determined based on the node registration information and the distribution rule in the messages to be consumed; the master node sends data including the messages to be consumed to each slave node so that each slave node consumes the messages to be consumed. The master node consumes the messages to be consumed in the message middleware and distributes the messages to be consumed to the slave nodes through the master node, enabling parallel consumption without specifying different consumption group IDs for each node, improving the accuracy of message consumption, avoiding repeated consumption, and improving the efficiency and flexibility of message processing.

[0035] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation part. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used to explain the embodiments of the present application together with the following specific implementation, but do not constitute a limitation to the embodiments of the present application. In the drawings:

[0037] Figure 1 Schematically shows a flowchart of a message processing method according to an embodiment of the present application;

[0038] Figure 2 Schematically shows a schematic diagram of an application scenario of a message processing method according to an embodiment of the present application;

[0039] Figure 3 Schematically shows a block diagram of a message processing device according to an embodiment of the present application. Detailed implementation manners

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the specific implementation manners described herein are only for explaining and illustrating the embodiments of the present application, and are not used to limit the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts fall within the scope of protection of the present application.

[0041] It should be noted that the acquisition, transmission, storage, use, processing, etc. of data in the technical solutions of the present application all comply with the relevant regulations of national laws and regulations. In the embodiments of the present application, certain existing solutions in the industry such as software, components, models, etc. may be mentioned, and they should be regarded as exemplary. The purpose is only to illustrate the feasibility in the implementation of the technical solutions of the present application, but it does not mean that the applicant has already or necessarily used this solution.

[0042] It should be noted that if there are directional indications (such as up, down, left, right, front, back...) involved in the embodiments of the present application, the directional indications are only used to explain the relative position relationship and movement conditions between components in a specific posture (as shown in the drawings). If the specific posture changes, the directional indications will also change accordingly.

[0043] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement them. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0044] In this embodiment, it should be noted that the message middleware is a software architecture used to asynchronously transmit messages between different components or applications in a distributed system. This architecture allows the message sender and the message receiver not to be online or running simultaneously. They can store and forward messages through the message middleware to achieve decoupling and asynchronous communication. The message middleware can handle various different application scenarios. For example, in the application scenario of increasing user points, the producer produces the message "user points + 1" and sends it to the message middleware. Since the message middleware consumes messages using the consumer group ID as the unique identifier, that is, the same message will only be consumed once under a consumer group. In this application scenario, by configuring the same consumer group ID for multiple nodes deployed by a service, it can be ensured that a message in the message middleware is only consumed once. Thus, it can be avoided that the user points will be incremented by more than one because multiple nodes in the consumer service all consume the message "user points + 1", resulting in the inability to meet the actual requirements. Another example is the application scenario where the consumer service needs to refresh the local cache entirely based on the producer service. The producer produces the message "refresh local cache" and sends it to the message middleware. Since the message middleware consumes messages using the consumer group ID as the unique identifier, that is, the same message will only be consumed once under a consumer group, it cannot meet the scenario that requires parallel consumption. In some technologies, by configuring different consumer group IDs for multiple nodes deployed by a service respectively, it can be achieved that the same message in the message middleware is consumed by all nodes once. Thus, it can be avoided that the local cache of an individual node is not refreshed because the message is not consumed by that node, resulting in the inability to meet the actual requirements. Although different consumer group IDs can be assigned to each node by means such as manual specification, random generation, or auto-increment sequence. However, in the above methods, the manual specification method cannot achieve automated deployment, and the random generation and auto-increment sequence methods are prone to causing a node to repeatedly consume the same message. Therefore, in this embodiment, a message processing method is proposed to optimize the parallel consumption method of multiple nodes and avoid repeated consumption by nodes.

[0045] Figure 1 Schematically shows a flowchart of a message processing method according to an embodiment of the present application. As Figure 1 shown, the embodiment of the present application provides a message processing method, which may include the following steps:

[0046] Step 100, determine a master node from all target consumer nodes based on a preset master node determination rule, where the target consumer nodes are all consumer nodes that have subscribed to the target topic in the message middleware;

[0047] Step 200, consume the message to be consumed in the target topic through the master node;

[0048] In this embodiment, the design specifications of the message middleware are retained. That is, even in the scenario of parallel consumption, multiple nodes under the same service are configured with the same consumer group ID. Using the consumer group ID as the unique identifier, all nodes under the same consumer group for the same message will only consume it once in the message middleware. It should be noted that asynchronous processing of messages can be achieved between different services through the message middleware. The message producer can send messages to the message middleware, and the message consumer subscribes to the topic in the message middleware to consume the messages in that topic. Specifically, the target consumption nodes are all consumption nodes that have subscribed to the target topic in the message middleware. All target consumption nodes are in the same consumer group and have the same consumer group ID. The target topic refers to the topic determined by each consumption node based on the current application requirements. Since the consumer group ID is the unique identifier, all nodes under the same consumer group for the same message will only consume it once in the message middleware. In this embodiment, a primary node is selected from all the target consumption nodes to consume the messages in the message middleware. That is, only the primary node in the same consumer group consumes the messages in the message middleware. Specifically, the primary node is determined from all the target consumption nodes based on a preset primary node determination rule. Among them, the preset primary node determination rule can be determined based on various factors, such as the processing capacity of the node, network latency, stability, etc. The preset primary node determination rule is preset in advance and can be dynamically adjusted during the system operation to adapt to changes in the network environment. After the primary node is determined, it will be responsible for consuming the messages to be consumed in the target topic.

[0049] It can be understood that before message consumption, the consumer needs to subscribe to the topic in the message middleware first to obtain the relevant messages required in the actual demand. Specifically, in one embodiment, before the step of determining the primary node based on the preset primary node determination rule, it further includes:

[0050] Determine the target consumption nodes and the corresponding first target topic and second target topic based on the preset message processing requirements, where the first target topic is the topic in the message middleware and the second target topic is the topic in the local memory;

[0051] Control each target consumption node to subscribe to the first target topic and the second target topic simultaneously.

[0052] It should be noted that the preset message processing requirements include the priority of messages, processing speed, storage location, etc. Based on the preset message processing requirements, it can be determined which consumer nodes will participate in message processing and the types of messages that the consumer nodes need to consume. Based on the message types, the target topics to be subscribed can be further determined. In this embodiment, the local memory can implement functions similar to those of the message middleware. By subscribing to the topics in the local memory, the consumer nodes can consume the messages to be consumed when the local memory publishes the messages to be consumed to the topics. Specifically, according to the preset message processing requirements, it is determined which consumer nodes will participate in message processing, and the first target topics corresponding to these consumer nodes in the message middleware and the second target topics corresponding to these consumer nodes in the local memory are identified. After the target consumer nodes and target topics are determined, all the target consumer nodes will be controlled to subscribe to the second target topics while subscribing to the first target topics. It should be noted that the types of messages to be consumed included in the first target topic and the second target topic are the same.

[0053] In this embodiment, by determining the target consumer nodes and topics based on the preset message processing requirements and controlling the target consumer nodes to subscribe to the topics, the message processing flow becomes more complete and flexible.

[0054] In one embodiment, the message processing method further includes:

[0055] In the case where the master node goes offline, a new master node is re-determined based on the preset master node determination rules, where the preset master node determination rules include load balancing.

[0056] In this embodiment, it should be noted that to ensure the stability and continuity of message processing, when the master node goes offline due to faults, maintenance, or other reasons, this change will be quickly detected. Specifically, it can be achieved through heartbeat detection, health checks, or node status monitoring, etc. Once it is detected that the master node has gone offline, the process of re-determining the master node will be triggered. During the process of re-determining the master node, the preset master node determination rules need to be followed, where load balancing is an important consideration. Load balancing aims to ensure that all target consumer nodes can evenly share the load of message processing, thereby avoiding single-point overload and performance bottlenecks.

[0057] Specifically, the information of all target consumer nodes can be collected, including the processing capabilities of the nodes, current load, network latency, etc. Based on the collected node information, the load balancing metrics of each node are calculated. The load balancing metrics can include the remaining processing capabilities of the nodes, current connection numbers, response times, etc. According to the load balancing metrics, the system will select a most suitable node as the new master node to resume message processing operations. The new master node will start receiving and processing the messages to be consumed published by the message middleware.

[0058] In this embodiment, by quickly detecting the offline of the master node, selecting a new master node based on the load balancing rule, performing master node switching and message processing recovery, etc., the effectiveness of coping with the master node failure is improved.

[0059] Step 300, determine the slave nodes among all consumer nodes based on the node registration information and the distribution rule in the message to be consumed;

[0060] It should be noted that during the process of the master node processing messages, the slave nodes can be determined according to the node registration information and the distribution rule in the message to be consumed. Among them, the distribution rule is the information carried in the message to be consumed, and can include the number of nodes that need to consume the message to be consumed. For example, one target consumer node, 50% of all target consumer nodes, all target consumer nodes, etc. The distribution rule can also include the priority of the message to be consumed, processing requirements, processing capabilities of consumer nodes, etc. The distribution rule can be adaptively adjusted based on actual application requirements. When a consumer node starts to join the service to execute message processing, it needs to first register the node in the registration center to generate the node registration information of this consumer node. The node registration information includes the IP (Internet Protocol) address, port number, consumer group ID, processing capabilities, subscribed topic messages, etc. of the node.

[0061] Furthermore, in one embodiment, the message processing method further includes:

[0062] Update the node registration information in the case of new consumer nodes being added or offline.

[0063] In this embodiment, it should be noted that in order to improve the ability of the message processing process to accurately track and manage the status of all consumer nodes, so as to effectively distribute and process messages. In the case of detecting new consumer nodes being added or offline, the node registration information needs to be updated. Specifically, when a new consumer node joins, the new node needs to send a registration request to the registration center. The registration request includes the basic information of the new node, such as IP address, port number, processing capabilities, etc. and the subscribed topic information, so as to update the node registration information. When a consumer node needs to go offline due to failure, maintenance or other reasons, the offline node needs to send a cancellation request to the registration center, or the offline status of the node can be automatically detected through mechanisms such as heartbeat detection, so as to update the node registration information. When new consumer nodes are added or offline, the load balancing strategy needs to be re-evaluated to adapt to the new node distribution situation.

[0064] In this embodiment, the status of all consumer nodes is accurately tracked and managed through the node registration information, so as to effectively distribute and process messages, ensuring the stability and continuity of the message processing system.

[0065] Step 400: The master node sends data including the message to be consumed to each slave node, so that each slave node consumes the message to be consumed.

[0066] It should be noted that the master node sends data including the message to be consumed to each slave node, so that each slave node consumes the message to be consumed. Thus, the problem that multiple consumer nodes in the same consumer group cannot consume the same message in the message middleware in the parallel consumption scenario is solved. The master node consumes the message to be consumed in the message middleware, and distributes the message to be consumed through the master node to enable the remaining nodes in the consumer group to consume the message to be consumed. While achieving parallel consumption, it avoids duplicate consumption by nodes.

[0067] In this embodiment, the master node is determined from all target consumer nodes based on a preset master node determination rule, where the target consumer nodes are all consumer nodes that have subscribed to the target topic in the message middleware; the master node consumes the message to be consumed in the target topic; the slave nodes among all consumer nodes are determined based on the node registration information and the distribution rule in the message to be consumed; the master node sends data including the message to be consumed to each slave node, so that each slave node consumes the message to be consumed. By consuming the message to be consumed in the message middleware through the master node and distributing the message to be consumed to the slave nodes through the master node, parallel consumption can be achieved without specifying different consumer group IDs for each node, and the accuracy of message consumption is improved, duplicate consumption is avoided, and the efficiency and flexibility of message processing are enhanced.

[0068] In one embodiment, sending data including the message to be consumed from the master node to each slave node so that each slave node consumes the message to be consumed includes:

[0069] The master node sends data including the message to be consumed to the message receiving interfaces of each slave node;

[0070] Trigger the corresponding local memory through the message receiving interface to publish the message to be consumed to the second target topic, so that each slave node consumes the message to be consumed.

[0071] In this embodiment, it should be noted that after determining the slave nodes, when the master node distributes the data including the message to be consumed obtained from the message middleware to each slave node, it does not directly give the message to be consumed to the message processing class of the slave node to enable the slave node to process the message to be consumed. Instead, the message to be consumed is published to the second target topic through the local memory of the slave node, so that the slave node obtains and consumes the message to be consumed. Refer to Figure 2 , Figure 2Among them, it includes Service A that produces messages, a message middleware, and Service B that consumes messages. Among them, Service B includes three nodes, namely Node 1, Node 2, and Node 3. These Node 1, Node 2, and Node 3 belong to the same consumer group, and the consumer group ID is all b. It is assumed that Node 1 in Service B is the master node. It should be noted that the nodes within the service can include a message receiving interface and a message processing class. The message processing class can be used to process the data to be consumed. The message processing class of the master node can also be used to send the data including the message to be consumed to the message receiving interfaces of each slave node, so that the message receiving interfaces of each slave node can receive and parse the data.

[0072] Specifically, after the master node obtains the message to be consumed, it encapsulates the message to be consumed, the IP address and port number of the master node, etc. into a data packet and sends it to the message receiving interface of each slave node. Each slave node has a message receiving interface for listening to and receiving messages from the master node. When receiving a message, the message receiving interface will parse the data packet and extract the message to be consumed. And after the message receiving interface of the slave node parses the message, it will trigger a local memory event, and the local memory event will publish the message to be consumed to the second target topic inside the slave node. Since the slave node subscribes to the second target topic, when there is a new message in the second target topic, the message processing class in the slave node will start consuming the message to be consumed.

[0073] In this embodiment, the message to be consumed is published inside the slave node through local memory, so that the slave node can also consume the message to be consumed published by the message middleware in parallel, avoiding duplicate consumption of messages, improving the message processing efficiency, and improving the adaptability of the message processing method to different application scenarios.

[0074] In one embodiment, after the step of sending the data including the message to be consumed by the master node to each slave node so that each slave node consumes the message to be consumed, it further includes:

[0075] Monitoring whether each slave node has fed back the successful reception information based on a preset delay duration;

[0076] If there is a target slave node that has not fed back the successful reception information, the master node will resend the data to the target slave node until the number of sending times reaches the preset number of times.

[0077] It should be noted that, in order to reduce the occurrence of consumption failures, in this embodiment, a retry mechanism is adopted to improve the fault tolerance of message processing. Specifically, after the master node sends data including messages to be consumed to the message receiving interfaces of each slave node, it starts monitoring the feedback of each slave node based on a preset delay duration. It can be understood that the preset delay duration can be adaptively adjusted according to the actual application scenario, ensuring that the preset delay duration is sufficient for the slave node to receive data, process the message, and feedback the successful reception information. The target slave node refers to the slave node that has not fed back the successful reception information after the preset delay duration ends. Specifically, after the preset delay duration ends, the master node checks whether all slave nodes have fed back the successful reception information. If there are target slave nodes that have not fed back the successful reception information, the master node will resend the data to the target slave node and record the number of times the data is resent to the target slave node. It checks whether the number of resends has reached the preset upper limit of the number of times. If not, the master node will continue to send data to the target slave node and record the number of times the data is resent to the target slave node. For the target slave nodes that have reached the preset number of sends but still have not received feedback, the master node can take further exception handling measures, such as logging and sending alarm notifications. When all slave nodes have successfully fed back the successful reception information, or all target slave nodes have reached the preset upper limit of the number of sends and exception handling has been performed, the entire process ends.

[0078] In this embodiment, the retry mechanism ensures that the master node can efficiently send data to each slave node and properly handle possible reception problems of the slave nodes. At the same time, the preset delay duration and the upper limit of the number of sends also provide the necessary flexibility and robustness for the process.

[0079] In one embodiment, determining the slave nodes among all consumption nodes based on the distribution rule and node registration information in the message to be consumed includes:

[0080] Determining the first consumption nodes in all consumption nodes that are in the same consumption group as the master node based on the node registration information;

[0081] Determining the slave nodes among the first consumption nodes based on the distribution rule in the message to be consumed, where the distribution rule includes the demand for slave nodes and / or load balancing.

[0082] In this embodiment, it should be noted that the master node can learn about the identity identifiers, affiliated consumer groups, processing capabilities, current loads, etc. of all slave nodes based on the node registration information. According to the node registration information, the master node filters out the first consumer nodes that are in the same consumer group as itself. These nodes are potential candidates for slave nodes because they share the same consumption logic or goal with the master node. The master node parses the distribution rules carried in the message to be consumed. The distribution rules can include the demand of the slave nodes, such as the ability or preference to process specific types of messages, the number of slave nodes required to complete the processing of the message to be consumed, etc.; the distribution rules can also include load balancing requirements, such as avoiding overloaded nodes or preferentially using low-load nodes. For the filtered first consumer nodes, the master node further evaluates whether they are suitable as slave nodes according to the distribution rules. Specifically, factors such as the processing capabilities, current loads, and matching degrees with the message to be consumed of each first consumer node can be considered based on the distribution rules. Based on the evaluation results, the master node selects the slave nodes that meet the distribution rules. These nodes will be responsible for consuming the message to be consumed. It can be understood that for some special cases, such as when no suitable slave nodes are found during the evaluation process, such as all candidates being overloaded or not meeting the message type requirements, etc., the master node can take other measures, such as temporarily storing the message, notifying the administrator, etc.

[0083] In this embodiment, the qualified slave nodes are accurately and efficiently selected according to the distribution rules and node registration information, thereby optimizing the message distribution and consumption process.

[0084] In one embodiment, sending data including the message to be consumed from the master node to each slave node to enable each slave node to consume the message to be consumed includes:

[0085] Sending data including the message to be consumed from the master node to each slave node;

[0086] For each slave node, determining whether the master node and the slave node are in the same consumer group based on the node registration information and the master node information in the data;

[0087] If the master node and the slave node are in the same consumer group, the slave node consumes the message to be consumed.

[0088] It should be noted that the message receiving interface of the slave node may be connected by illegal users and illegal messages may be sent. Moreover, only the slave node will receive and process the data sent by the master node within the same consumer group. To improve the legitimacy of message processing, in this embodiment, after the master node sends the data including the messages to be consumed to each slave node, each slave node will perform security verification on the sent data. Specifically, since the data packed by the master node includes the IP address and port number of the master node itself, for each slave node, after receiving the data including the messages to be consumed sent by the master node, by parsing the master node information in the data, the IP address and port number of the master node can be obtained. Based on the IP address and port number of the master node, the slave node can query in the node registration information whether it is in the same consumer group as the master node. If the master node and the slave node are in the same consumer group, it is determined that the data received by the slave node is secure, and the messages to be consumed corresponding to the data can be consumed.

[0089] In this embodiment, by identifying the consumer group to ensure the security of the messages consumed by the slave node, the reliability of message consumption is improved.

[0090] It should be noted that embodied intelligence, also known as concrete intelligence, is a comprehensive concept involving multiple fields such as artificial intelligence, robotics, and cognitive science. It emphasizes that intelligent agents such as robots can achieve the ability of learning and reasoning through interaction with the environment and by using the perceptual and motor capabilities of the body. In one embodiment, embodied intelligence can be combined with message processing to achieve efficient, accurate, and flexible message processing. By combining embodied intelligence with the message processing mechanism, a highly intelligent, adaptive, and efficient distributed message processing system can be constructed. In this system, the concept of embodied intelligence is not only reflected in the intelligent processing capabilities of each node, but also in how they interact with the environment through the body and how to achieve higher-level intelligent behaviors through this interaction. Here, the body refers to the physical or virtual nodes in the network, and the environment includes the message middleware and the message flow therein. Specifically, the master node has powerful computing capabilities and intelligent decision-making capabilities, is responsible for consuming the messages to be consumed in the target topic, and intelligently identifies and assigns tasks to the slave nodes according to the node registration information and the distribution rules in the messages. The slave nodes also have intelligent processing capabilities. They can receive the data sent by the master node and consume it according to their own processing capabilities and business requirements. By combining embodied intelligence with the message processing mechanism, not only can accurate distribution and parallel consumption of messages be achieved, but also dynamic adjustment and optimization can be performed according to changes in the system environment, thereby significantly improving the efficiency and flexibility of message processing.

[0091] Figure 3 Schematically shows a structural block diagram of a message processing device according to an embodiment of the present application. AsFigure 3 As shown in Figure 3 , an embodiment of the present application provides a message processing device 1000, which may include:

[0092] A memory 1001, configured to store instructions;

[0093] A processor 1002, configured to call instructions from the memory 1001 and be able to implement the above-mentioned message processing method when executing the instructions.

[0094] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to cause a machine to execute the above-mentioned message processing device method.

[0095] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0096] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, so that the instructions executed by the processors of the computer or other programmable data processing devices generate a device for implementing the functions specified in one Figure 1 one or more flows and / or Figure 1 more blocks

[0097] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements the functions specified in one Figure 1 one or more flows and / or Figure 1 more blocks

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 steps of the functions specified in one block or multiple blocks.

[0099] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0100] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0101] Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media do not include transitory media, such as modulated data signals and carrier waves.

[0102] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, commodity or device including the element.

[0103] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A message processing method, characterized in that, Including: Determine a master node from all target consumer nodes based on a preset master node determination rule, where the target consumer nodes are all consumer nodes that have subscribed to a target topic in the message middleware; Consume the messages to be consumed in the target topic through the master node; Determine slave nodes among all the consumer nodes based on node registration information and the distribution rule in the messages to be consumed; Send data including the messages to be consumed to each of the slave nodes through the master node, so that each of the slave nodes consumes the messages to be consumed.

2. The message processing method according to claim 1, wherein Before the step of determining the master node based on the preset master node determination rule, it further includes: Determine target consumer nodes and corresponding first target topic and second target topic based on preset message processing requirements, where the first target topic is a topic in the message middleware, and the second target topic is a topic in local memory; Control each of the target consumer nodes to subscribe to the first target topic and the second target topic simultaneously.

3. The message processing method according to claim 2, wherein The step of sending data including the messages to be consumed to each of the slave nodes through the master node, so that each of the slave nodes consumes the messages to be consumed, includes: Send data including the messages to be consumed to the message receiving interfaces of each of the slave nodes through the master node; Trigger the corresponding local memory to publish the messages to be consumed to the second target topic through the message receiving interface, so that each of the slave nodes consumes the messages to be consumed.

4. The message processing method according to claim 1, wherein The step of determining slave nodes among all the consumer nodes based on the distribution rule in the messages to be consumed and node registration information includes: Determine the first consumer nodes in the same consumer group as the master node among all the consumer nodes based on the node registration information; Determine the slave nodes in the first consumer nodes based on the distribution rule in the messages to be consumed, where the distribution rule includes the required number of slave nodes and / or load balancing.

5. The message processing method according to claim 1, wherein The step of sending data including the messages to be consumed to each of the slave nodes through the master node, so that each of the slave nodes consumes the messages to be consumed, includes: Send data including the messages to be consumed to each of the slave nodes through the master node; For each of the slave nodes, determine whether the master node and the slave node are in the same consumer group based on the node registration information and the master node information in the data; If the master node and the slave node are in the same consumer group, consume the messages to be consumed through the slave node.

6. The message processing method according to claim 1, characterized in that After the step of sending data including the messages to be consumed to each of the slave nodes through the master node, so that each of the slave nodes consumes the messages to be consumed, it further includes: Monitor whether each of the slave nodes has fed back a successful reception message based on a preset delay duration; If there is a target slave node that has not fed back the successful reception message, resend the data to the target slave node through the master node until the number of sending times reaches a preset number of times.

7. The message processing method according to claim 1, characterized in that It further includes: In the case of the master node going offline, re-determine the master node based on the preset master node determination rule, where the preset master node determination rule includes load balancing.

8. The message processing method according to claim 1, wherein It further includes: Update the node registration information in the case of addition or deactivation of a consumption node.

9. A message processing device, characterized in that Including: A memory configured to store instructions; A processor configured to call the instructions from the memory and capable of implementing the message processing method according to any one of claims 1 to 8 when executing the instructions.

10. A machine-readable storage medium, characterized in that, Instructions are stored on the machine-readable storage medium, and the instructions are used to cause the machine to execute the message processing method according to any one of claims 1 to 8.

Images