Message processing method and device and electronic equipment

Abstract

The invention relates to the technical field of data processing, in particular to a message processing method and device and electronic equipment. Through the partition configuration information based on the message middleware, directional sending of the gray scale message of the message producer is controlled, the gray scale partition is allocated to the gray scale consumer, and the gray scale consumer is controlled to directionally obtain the message from the gray scale partition. According to the method and the device, the grayscale message is accurately delivered from the message producer to the grayscale consumer through the message middleware, the decoupling capability and the grayscale capability of the message middleware for the producer and the consumer are kept, meanwhile, the consumer does not need to classify the message, and the message processing efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data processing, and in particular, to a message processing method, apparatus, and electronic device. Background Art

[0002] Gray release means gradually providing the functions or services of the new version to a small number of users in the production environment to verify the stability and performance of the new version, and then gradually expanding the release scope. Therefore, in the gray release scenario, it is necessary for the producer to accurately deliver the gray messages to the gray consumers in order to achieve the effect of gray verification.

[0003] The message middleware realizes data communication through an efficient and reliable message delivery mechanism, thereby promoting the integration of distributed systems. It can store the messages sent by the upstream message producers and provide these messages to the downstream consumers for consumption, realizing the decoupling between the producer and the consumer, but at the same time losing the ability of gray verification. Usually, it is necessary for the downstream consumer to obtain the messages from the message middleware and then classify them, and provide the gray messages among them to the gray consumers for consumption.

[0004] However, this method adds an additional message classification operation between the message middleware and the consumer, reducing the message processing efficiency. Summary of the Invention

[0005] In order to solve the above technical problems, the present disclosure provides a message processing method, apparatus, and electronic device to ensure the gray ability of the message middleware and improve the message processing efficiency.

[0006] In a first aspect, an embodiment of the present disclosure provides a message processing method, including:

[0007] Obtain the partition configuration information of the partitions in the message middleware, where the partitions in the message middleware include at least one gray partition;

[0008] Based on the partition configuration information, allocate the gray partition to the gray consumer;

[0009] Control the gray consumer to obtain the gray message from the gray partition.

[0010] In some embodiments, the allocating the gray partition to the gray consumer based on the partition configuration information includes:

[0011] Identify the gray consumers in the consumer list according to the gray identifiers of the consumers;

[0012] Based on the partition configuration information, obtain the gray partition number of the gray partition;

[0013] Bind the gray level partition number to the gray level consumer.

[0014] In some embodiments, the controlling the gray level consumer to obtain gray level messages from the gray level partition includes:

[0015] Authenticate the current consumer according to the binding relationship between the gray level partition number and the gray level consumer;

[0016] When the identity information of the current consumer conforms to the binding relationship with the gray level partition number, determine that the current consumer is the gray level consumer bound to the gray level partition;

[0017] Allow the gray level consumer to obtain gray level messages from the gray level partition.

[0018] In some embodiments, the partition in the message middleware further includes a plurality of baseline partitions. After obtaining the partition configuration information of the partition in the message middleware, the method further includes:

[0019] Based on the partition configuration information, allocate the plurality of baseline partitions to a plurality of baseline consumers;

[0020] If there is no such gray level consumer, allocate the gray level partition to the first baseline consumer.

[0021] In some embodiments, the allocating the plurality of baseline partitions to a plurality of baseline consumers based on the partition configuration information includes:

[0022] Obtain the number of baseline partitions and the number of baseline consumers;

[0023] Calculate the average number of partitions per baseline consumer and the number of baseline consumers with additional partitions according to the number of baseline partitions and the number of baseline consumers;

[0024] Determine the start and end numbers of the baseline partitions for each baseline consumer according to the average number of partitions per baseline consumer and the number of baseline consumers with additional partitions;

[0025] Based on the start and end numbers of the baseline partitions, allocate the plurality of baseline partitions to a plurality of baseline consumers.

[0026] In some embodiments, the start and end numbers of the baseline partitions include the start number of the baseline partition. The determining the start and end numbers of the baseline partitions for each baseline consumer according to the average number of partitions per baseline consumer and the number of baseline consumers with additional partitions includes:

[0027] For each baseline consumer, if the baseline consumer belongs to the baseline consumers with additional partitions, determine the starting number of the baseline partitions of the baseline consumer according to the product of the average number of partitions and the serial number of the baseline consumer plus the serial number of the baseline consumer; or,

[0028] if the baseline consumer does not belong to the baseline consumers with additional partitions, determine the starting number of the baseline partitions of the baseline consumer according to the product of the average number of partitions and the serial number of the baseline consumer plus the number of baseline consumers with additional partitions.

[0029] In some embodiments, the method further includes:

[0030] Controlling a message producer to send baseline messages to the baseline partitions in the message middleware, where the baseline messages are messages that do not contain gray scale tags;

[0031] Controlling a baseline consumer to obtain the baseline messages from at least one baseline partition assigned to the baseline consumer.

[0032] In some embodiments, the controlling the message producer to send baseline messages to the baseline partitions in the message middleware, where the baseline messages are messages that do not contain gray scale tags, includes:

[0033] Taking the remainder of the number of the baseline partitions according to the eigenvalue of the baseline message to obtain the target baseline partition number corresponding to the baseline message;

[0034] Controlling the message producer to send the baseline message to the target baseline partition corresponding to the target baseline partition number.

[0035] In a second aspect, an embodiment of the present disclosure provides a message processing device, including:

[0036] An obtaining module, configured to obtain partition configuration information of partitions in a message middleware, where the partitions in the message middleware include at least one gray scale partition;

[0037] An allocation module, configured to allocate the gray scale partitions to gray scale consumers based on the partition configuration information;

[0038] A first control module, configured to control the gray scale consumers to obtain the gray scale messages from the gray scale partitions.

[0039] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0040] A memory;

[0041] A processor; and

[0042] A computer program;

[0043] Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect.

[0044] The message processing method, apparatus, and electronic device provided by the embodiments of the present disclosure control the directional sending of gray-scale messages of message producers based on the partition configuration information of the message middleware, and allocate gray-scale partitions to gray-scale consumers to control the gray-scale consumers to obtain messages directionally from the gray-scale partitions, realizing the precise delivery of gray-scale messages from message producers via the message middleware to gray-scale consumers, taking into account the decoupling ability of the message middleware for producers and consumers and the maintenance of gray-scale capabilities, and at the same time eliminating the need for consumers to classify messages, improving the message processing efficiency. Description of the Drawings

[0045] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0046] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0047] Figure 1 It is a flowchart of the message processing method provided by the embodiments of the present disclosure;

[0048] Figure 2 It is a schematic diagram of an application scenario provided by the embodiments of the present disclosure;

[0049] Figure 3 It is a schematic diagram of an application scenario provided by another embodiment of the present disclosure;

[0050] Figure 4 It is a schematic structural diagram of the message processing apparatus provided by the embodiments of the present disclosure;

[0051] Figure 5 It is a schematic structural diagram of a vehicle provided by the embodiments of the present disclosure;

[0052] Figure 6 It is a schematic structural diagram of the electronic device provided by the embodiments of the present disclosure. Detailed Embodiments

[0053] In order to better understand the above-mentioned objects, features, and advantages of the present disclosure, the following will further describe the solutions of the present disclosure. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

[0054] In the following description, many specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, rather than all the embodiments.

[0055] Apache Kafka is a distributed stream processing platform with characteristics such as high throughput, persistence, and scalability, and is commonly used to build real-time data pipelines and stream applications. Gray release refers to gradually providing the functions or services of the new version to a small number of users in the production environment to verify the stability and performance of the new version, and then gradually expanding the release scope.

[0056] Currently, during the gray release process, it is necessary to verify the gray functions of the downstream gray service Pods, and the gray service Pods implement their functional services through corresponding consumers. The gray messages published by the upstream producer will lose their gray capabilities when passing through the message middleware Kafka and cannot be accurately delivered to the consumers of the downstream gray service Pods, resulting in the inability to perform full-link gray verification on the message link.

[0057] In view of the above problems, the embodiments of the present disclosure provide a message processing method, and the method will be introduced below in combination with specific embodiments.

[0058] Figure 1 It is a flowchart of the message processing method provided by the embodiments of the present disclosure. This method can be applied to Figure 2 the application scenario shown in the figure. In this application scenario, the message processing system 20 includes a message producer 21, a message middleware 22, a baseline consumer 23, and a gray consumer 24. The message middleware includes multiple topics, and each topic serves as a basic unit in the message middleware to organize and classify messages, and each message belongs to a specific topic. At the same time, the partitions of the message middleware are the physical parts of the topics and are used to horizontally split the topics. Each topic includes one or more partitions, and each partition corresponds to a file on the disk. The following embodiments will be described by taking multiple partitions under a single topic as an example.

[0059] It can be understood that the message processing method provided by the embodiments of the present disclosure can also be applied in other scenarios.

[0060] The following combines Figure 2 the application scenario shown in the figure to introduce Figure 1 the message processing method shown in the figure. The specific steps included in this method are as follows:

[0061] S101. Obtain the partition configuration information of the partitions in the message middleware, and the partitions in the message middleware include at least one gray partition.

[0062] The message middleware 22 is divided into multiple partitions, including a gray partition and a baseline partition.

[0063] The gray partition is specifically used to store the gray messages sent by the upstream message producer 21 and is consumed by the downstream gray consumer 24.

[0064] The partition configuration information records the nature of each partition in the message middleware 22 and the corresponding partition number.

[0065] The partition number is the unique identifier of each partition, and the specified partition can be determined according to the partition number.

[0066] S102. Allocate the gray partition to the gray consumer based on the partition configuration information.

[0067] In the gray release scenario, the downstream message consumers are divided into baseline consumers and gray consumer 24. Among them, the baseline consumers process messages using the business logic that has been stably running in the normal production environment. They process messages according to the established rules and processes to ensure the daily stable operation of the system.

[0068] The gray consumer 24 is responsible for verifying new functions, new versions, or changed business logic. They receive and process messages in a small range, observe the stability and compatibility of new functions, and reduce the risks brought by full-scale release.

[0069] In the message processing system, according to the nature and partition number of the partitions recorded in the partition configuration information, the gray partition among them is allocated to the downstream gray consumer 24, so that the downstream gray consumer 24 is specified to obtain messages from the gray partition for consumption.

[0070] Specifically, according to the gray identifier of the consumer, the gray consumers in the consumer list are identified; based on the partition configuration information, the gray partition number of the gray partition is obtained; and the gray partition number is bound to the gray consumer.

[0071] When the consumer instance starts, the consumer list including the downstream baseline consumer 23 and the gray consumer 24 is synchronized to the message middleware 22, and the gray consumer 24 has a special gray identifier for it to be recognized.

[0072] The partition configuration information records the nature of each partition in the message middleware 22 and the corresponding partition number, which includes the gray partition number of the gray partition.

[0073] The gray partition number refers to the partition number specifically used for gray message transmission in the message middleware 22. The gray partition is usually the partition with the lowest serial number, such as partition number 0.

[0074] Further, bind the gray partition to the gray consumer 24 so that the messages in the gray partition are specified to be retrieved and processed by the gray consumer 24.

[0075] Optionally, after allocating the gray partition to the gray consumer, control the message producer to send the gray message to the gray partition in the message middleware, where the gray message is a message containing a gray label.

[0076] The message producer is the generator and sender of messages, responsible for encapsulating data or events into messages and publishing them to the message link, such as sensors, control units, or other functional modules that actively initiate communication in the vehicle system.

[0077] When controlling the message producer 21 to send messages, it is first necessary to determine the nature of the message. The nature of the message represents whether the message belongs to a gray message or a baseline message. Among them, gray messages need to be processed and consumed by the gray consumer 24, and baseline messages need to be consumed and processed by the baseline consumer 23.

[0078] Among them, the gray message is a message with a gray label. When the message producer produces a message, it determines whether the message is used for gray verification. If the message is used for gray verification, a gray label will be added to the message, such as the Key of the message, or specific gray header information of the message, etc., which is specifically set according to business requirements.

[0079] In the message middleware, the gray partition is dedicated to storing gray messages, and the partition configuration information of the message middleware is pre-negotiated with the message producer, and the message producer already knows the gray partition and its partition number in the message middleware. Therefore, when sending messages, it is possible to control the message producer to specify and send the gray message with a gray label to the gray partition in the message middleware.

[0080] S103. Control the gray consumer to retrieve the gray message from the gray partition.

[0081] The gray consumer 24 is designated to process the gray message to implement gray verification of the message processing system.

[0082] In the above steps, the gray partition in the message middleware 22 has been allocated to the gray consumer 24; when the gray consumer 24 needs to retrieve the gray message, it retrieves the message from the gray partition in the message middleware 22, and the gray partition in the message middleware 22 contains the gray messages sent by the message producer 21 in a directed manner. That is, the gray message retrieved by the gray consumer 24 from the gray partition in the message middleware 22 is the gray message sent by the message producer 21, realizing the directed publishing of gray messages from the message producer 21 via the message middleware 22 to the gray consumer 24.

[0083] Optionally, the gray-scale partition in the message middleware can be a special topic (referred to as the gray-scale topic), which controls the message producer to send gray-scale messages to this topic, and correspondingly controls the gray-scale consumer to subscribe to the gray-scale topic and obtain the gray-scale messages therein.

[0084] In the embodiment of the present disclosure, by obtaining the partition configuration information of the partitions in the message middleware, where the partitions in the message middleware include at least one gray-scale partition; based on the partition configuration information, allocating the gray-scale partition to the gray-scale consumer; controlling the message producer to send gray-scale messages to the gray-scale partition in the message middleware; controlling the gray-scale consumer to obtain the gray-scale messages from the gray-scale partition; by controlling the directional sending of the gray-scale messages of the message producer based on the partition configuration information of the message middleware, and allocating the gray-scale partition to the gray-scale consumer, and controlling the gray-scale consumer to obtain messages directionally from the gray-scale partition, the accurate delivery of the gray-scale messages from the message producer via the message middleware to the gray-scale consumer is realized, taking into account the decoupling ability of the message middleware for the producer and the consumer and the maintenance of the gray-scale ability, and at the same time, there is no need for the consumer to classify the messages, improving the message processing efficiency.

[0085] Based on the above embodiment, the controlling the gray-scale consumer to obtain gray-scale messages from the gray-scale partition includes: authenticating the identity of the current consumer according to the binding relationship between the gray-scale partition number and the gray-scale consumer; when the identity information of the current consumer conforms to the binding relationship with the gray-scale partition number, determining that the current consumer is the gray-scale consumer bound to the gray-scale partition; and allowing the gray-scale consumer to obtain gray-scale messages from the gray-scale partition.

[0086] After the gray-scale consumer 24 is bound to the gray-scale partition in the message middleware, their binding relationship is stored in the coordination service of the entire message link.

[0087] The current consumer is the consumer that currently needs to obtain messages from the gray-scale partition, and the identity information of the consumer includes metadata such as consumer ID, IP address, and gray-scale version number.

[0088] Optionally, the message middleware 22 authenticates the identity information of the current consumer to determine whether the current consumer is a gray-scale consumer bound to the gray-scale partition based on the partition configuration information. Specifically, first verify whether the current consumer has a gray-scale identifier; if so, further verify whether the attribute information such as the ID and IP address of the current consumer conforms to the binding relationship in the coordination service. Correspondingly, if the current consumer does not have a gray-scale identifier, or the attribute information such as the ID and IP address of the current consumer does not conform to the binding relationship in the coordination service, it is determined that the current consumer is not a gray-scale consumer bound to the gray-scale partition.

[0089] When the identity information of the current consumer conforms to the binding relationship with the gray partition number, it means that the current consumer passes the identity verification and is determined to be a gray consumer bound to the gray partition. Further, the gray consumer can be allowed to obtain messages from the gray partition.

[0090] Optionally, after the current consumer passes the identity verification and is determined to be a gray consumer bound to the gray partition, an authorization token is sent to the gray consumer. When the gray consumer reads messages from the gray partition next time, it can be allowed to obtain gray messages from the gray partition based on the token without performing identity verification again, effectively improving the message acquisition efficiency.

[0091] In the embodiment of the present disclosure, when the current consumer obtains messages from the gray partition, the identity of the current consumer is first verified. Only when the current consumer is a gray consumer bound to the gray partition is it allowed to obtain messages in the gray partition, avoiding the inflow of gray messages to the baseline consumer and affecting the gray capability of the message link, further improving the reliability and accuracy of message processing.

[0092] Based on the above embodiment, the partition in the message middleware further includes a plurality of baseline partitions. After obtaining the partition configuration information of the partitions in the message middleware, the method further includes: allocating the plurality of baseline partitions to a plurality of baseline consumers based on the partition configuration information; if there is no gray consumer, allocating the gray partition to the first baseline consumer.

[0093] The baseline partition is used to store baseline messages without gray labels for consumption by baseline consumers.

[0094] The baseline consumer processes messages using the business logic that has been stably running in the normal production environment. Before the system officially processes messages, the baseline consumer registers with the message processing system and obtains the topics subscribed to by each baseline consumer.

[0095] In the message middleware, there are multiple baseline partitions under the same topic, and these baseline partitions need to be allocated to multiple baseline consumers who have subscribed to the topic.

[0096] For the gray partition under the specified topic in the message middleware, if there is no gray consumer who subscribes to the topic, the gray partition is allocated to the first baseline consumer who subscribes to the topic, and the first baseline consumer who subscribes to the topic consumes the messages in the gray partition.

[0097] Specifically, allocating the multiple baseline partitions to multiple baseline consumers based on the partition configuration information includes: obtaining the number of baseline partitions and the number of baseline consumers; calculating the average number of partitions per baseline consumer and the number of baseline consumers with extra partitions according to the number of baseline partitions and the number of baseline consumers; determining the start and end numbers of the baseline partitions for each baseline consumer according to the average number of partitions per baseline consumer and the number of baseline consumers with extra partitions; and allocating the multiple baseline partitions to multiple baseline consumers based on the start and end numbers of the baseline partitions.

[0098] The number of baseline partitions refers to the number of baseline partitions under a specific topic.

[0099] Both baseline consumers and gray consumers need to be registered in the message processing system first and obtain the topics subscribed by each baseline consumer or gray consumer. After the registration of baseline consumers and gray consumers is completed, based on the registration information of each baseline consumer or gray consumer, the topics subscribed by each baseline consumer or gray consumer can be obtained, and then the partitions under the topic in the message middleware are allocated to the baseline consumers or gray consumers that have subscribed to the topic.

[0100] A consumer subscribing to a topic means that the consumer declares to the message middleware that it is interested in one or more topics and hopes to receive the messages under these topics. The subscription mechanism allows consumers to dynamically select the types of messages to be processed without caring about the message production process.

[0101] In the gray test scenario, for multiple partitions under a specified topic, there are usually multiple baseline partitions. Pull the list of baseline consumers subscribing to the topic to obtain the number of baseline consumers subscribing to the topic, that is, the number of baseline consumers waiting to be allocated the baseline partitions under the topic.

[0102] Based on the number of baseline partitions under a specific topic and the number of baseline consumers subscribing to the topic, making an as-average-as-possible allocation to the baseline consumers subscribing to the topic can obtain the average number of partitions that can be evenly allocated to each baseline consumer and the number of baseline consumers with extra partitions.

[0103] For example, if there are 10 baseline partitions and 3 baseline consumers under a specific topic, the average number of partitions that can be evenly allocated to each baseline consumer is 3, and the remaining 1 baseline partition cannot be evenly allocated. There is 1 baseline consumer that is additionally allocated one baseline partition, that is, the 3 baseline consumers are respectively allocated 4, 3, and 3 baseline partitions. Among them, the average number of partitions per baseline consumer is 3 partitions, and the number of baseline consumers with extra partitions is 1.

[0104] Further, after determining the number of baseline partitions assigned to each baseline consumer, calculate the start and end numbers of the baseline partitions assigned to each baseline consumer, so that each baseline consumer can directly obtain the baseline partitions assigned to it according to the start and end numbers of the baseline partitions.

[0105] For example, if the start number of the baseline partition assigned to a certain baseline consumer is 3 and the end number is 5, then the baseline consumer can directly obtain multiple baseline partitions with partition numbers from 3 to 5 according to the start and end numbers of the baseline partitions.

[0106] In some embodiments, the start and end numbers of the baseline partitions include the start number of the baseline partition. Determining the start and end numbers of the baseline partitions for each baseline consumer according to the average number of partitions of each baseline consumer and the number of baseline consumers with additional partitions includes: for each baseline consumer, if the baseline consumer belongs to the baseline consumers with additional partitions, then determine the start number of the baseline partition of the baseline consumer according to the product of the average number of partitions and the serial number of the baseline consumer plus the serial number of the baseline consumer; or, if the baseline consumer does not belong to the baseline consumers with additional partitions, then determine the start number of the baseline partition of the baseline consumer according to the product of the average number of partitions and the serial number of the baseline consumer plus the number of baseline consumers with additional partitions.

[0107] It should be noted that the serial numbers and partition numbers of the baseline consumers in the embodiments of the present disclosure are all digital serial numbers that increase sequentially from small to large. In actual situations, if the definitions of the serial numbers and partition numbers of the baseline consumers do not conform to this rule, serial numbers and partition numbers of baseline consumers that conform to the above rules can be assigned to them for calculating the start and end numbers of the baseline partitions in the baseline partition allocation process. For example, assign serial numbers to the baseline consumers in the baseline consumer list in sequence.

[0108] The basic logic for calculating the start number of the baseline partition is: add one to the last baseline partition number that has been assigned.

[0109] Based on the principle of round-robin allocation of baseline partitions, a baseline consumer belonging to the baseline consumers with additional partitions means that the serial number of the baseline partition is less than the number of baseline consumers with additional partitions. That is, if the number of baseline consumers with additional partitions is denoted as X, then the first X baseline consumers in the baseline consumer list are the baseline consumers with additional partitions.

[0110] Based on this logic, it is possible to determine whether the current baseline consumer number belongs to the baseline consumers with additional partitions according to the size relationship between the current baseline consumer number and the number of baseline consumers with additional partitions. Specifically, if the current baseline consumer number is less than or equal to the number of baseline consumers with additional partitions, it is determined that the current baseline consumer belongs to the number of baseline consumers with additional partitions; if the current baseline consumer number is greater than the number of baseline consumers with additional partitions, it is determined that the current baseline consumer does not belong.

[0111] If the baseline consumer currently assigned a baseline partition belongs to the baseline consumers with additional partitions, then all the baseline consumers before the baseline consumer currently assigned a baseline partition are baseline consumers with additional partitions, and the number of baseline partitions that have been currently assigned is the sum of the product of the average number of partitions and the number of the baseline consumer and the number of the baseline consumer.

[0112] If the baseline consumer currently assigned a baseline partition does not belong to the baseline consumers with additional partitions, then before the baseline consumer currently assigned a baseline partition, the baseline partitions of the baseline consumers with additional partitions have been completely assigned, and the number of baseline partitions that have been currently assigned is the sum of the product of the average number of partitions and the number of the baseline consumer and the number of baseline consumers with additional partitions.

[0113] The above calculation logic for the starting number of the baseline partition can be expressed as:

[0114] start = CP * i + min(i, EC) + 1

[0115] Where start is the starting number of the baseline partition, CP is the average number of partitions, i is the number of the baseline partition, and EC is the number of baseline consumers with additional partitions.

[0116] Furthermore,

[0117] CP = (N - h) / C

[0118] EC = (N - h) % C

[0119] Where N is the total number of partitions under the current topic, h is the number of gray-scale partitions, and C is the number of baseline consumers.

[0120] Further, the start and end numbers of the baseline partitions include the end number of the baseline partitions. Determining the start and end numbers of the baseline partitions for each baseline consumer according to the average number of partitions of each baseline consumer and the number of baseline consumers with additional partitions includes: if the baseline partition belongs to a baseline consumer with additional partitions, the number of baseline partitions assigned to the baseline consumer is the average number of partitions plus one; if the baseline partition does not belong to a baseline consumer with additional partitions, the number of baseline partitions assigned to the baseline consumer is the average number of partitions; determining the end number of the baseline partition according to the start number of the baseline partition and the number of baseline partitions assigned to the baseline consumer.

[0121] The number of baseline partitions assigned to a baseline consumer can be expressed as:

[0122] length = CP + (i > EC? 0 : 1)

[0123] Further, select a baseline partition sublist [start, start + length] from the baseline partition list and assign it to the corresponding baseline consumer.

[0124] Optionally, after the registration of the baseline consumers is completed, select one consumer from the registered baseline consumers as the leader (referred to as the leader consumer), and then control the leader consumer to calculate the start and end numbers of the baseline partitions for each registered baseline consumer according to the above logic and synchronize them to each baseline consumer to complete the assignment of the baseline partitions.

[0125] By calculating the start and end numbers of the baseline partitions assigned to each baseline consumer in the embodiments of the present disclosure, each baseline consumer can directly obtain the baseline partitions assigned to it, without sequentially assigning each baseline partition, improving the efficiency of the assignment of the baseline partitions.

[0126] In addition, in the embodiments of the present disclosure, by calculating the average number of partitions of each baseline consumer and the number of baseline consumers with additional partitions, multiple partitions under a single topic are distributed as evenly as possible to multiple baseline consumers subscribing to the topic, ensuring that the possibility of messages on each partition being consumed by baseline consumers is as balanced as possible, and avoiding performance problems caused by overloading of a single partition.

[0127] In some embodiments, the method further includes: controlling a message producer to send baseline messages to the baseline partitions in the message middleware, where the baseline messages are messages that do not contain gray scale labels; controlling baseline consumers to obtain the baseline messages from at least one baseline partition assigned to the baseline consumers.

[0128] Specifically, the control message producer sends the baseline message to the baseline partition in the message middleware. The baseline message is a message without a gray label, including: taking the remainder of the number of baseline partitions based on the eigenvalue of the baseline message to obtain the target baseline partition number corresponding to the baseline message; controlling the message producer to send the baseline message to the target baseline partition corresponding to the target baseline partition number.

[0129] Generally, there is at least one gray partition and multiple baseline partitions under a single topic in the message middleware. For the message producer, it doesn't care which consumer consumes the message it produces. Therefore, it is sufficient for the control message producer to send the baseline message to a certain baseline partition.

[0130] The message producer adds a gray label to the gray message during message production, and the message without the gray label is the baseline message for the baseline consumer to consume.

[0131] When the control message producer sends a message, if the message does not contain a gray label, a certain baseline partition in the message middleware is selected according to the logical algorithm of round-robin average distribution.

[0132] Specifically, first obtain the eigenvalue of the baseline message. This eigenvalue can be obtained by taking the hash modulo of a pre-specified key in the baseline message. If the baseline message does not contain the pre-specified key, the message producer can be controlled to assign a unique special identifier to the baseline message, and then convert this special identifier into a positive integer as the eigenvalue of the baseline message.

[0133] In the embodiment of the present disclosure, the eigenvalue of the baseline message obtained after hash modulo or the eigenvalue of the baseline message obtained by converting the special identifier into a positive integer is used to take the remainder of the number of baseline partitions under this topic in the message middleware. Taking the eigenvalue M of the baseline message obtained after hash modulo as an example, the calculation process is expressed as:

[0134] partition = M % (ap - h) + 1

[0135] Where partition is the target baseline partition number corresponding to the baseline message, ap is the total number of partitions under this topic in the message middleware, and h is the number of gray partitions under this topic in the message middleware.

[0136] After obtaining the target baseline partition number corresponding to the baseline message, the control message producer sends the baseline message to the target baseline partition corresponding to the target baseline partition number.

[0137] In the embodiments of the present disclosure, when a control message producer sends a message to a message middleware, a target partition number is determined according to a message feature value, so that multiple baseline partitions in the message middleware achieve load balancing, and performance problems caused by overloading of a single partition are avoided.

[0138] Figure 3 This is another application scenario provided by the embodiments of the present disclosure. As Figure 3 shown, this application scenario includes a message producer (Producer1), a message middleware cluster (Kafka Cluster), baseline consumers (Consumer1, Consumer2), and a gray consumer (GreyConsumer1). In the embodiments of the present disclosure, by using the SpringBootStarter tool, a Starter dependency is introduced to use components to implement a message processing method.

[0139] Producer1 refers to a component or service that sends messages to the Kafka cluster. Among them, send(topic, key, data) represents a function or method for sending messages, where topic is the topic, key is the key of the message, and data is the data to be sent. KafkaTemplate is a template class used to simplify Kafka operations in the Spring framework. Greypartitioner is used to determine the target partition for message sending. ProducerRecord(topic, partition, key, value) is a record class used by the message producer to send messages, which contains information about the topic, partition, key, and value, and the Producer performs the sending action.

[0140] The message middleware cluster (Kafka Cluster) is a distributed system composed of multiple Kafka Brokers. Kafka Broker 1 / 2 represents a single server node in the Kafka cluster. partition0, 1, 2, 3, 4, 5 represent partitions 0 to 5. There can be multiple partitions under each topic. Among them, partition0 is defined as the gray partition, and the remaining partitions are baseline partitions.

[0141] Each Consumer includes an assignment list assign list, which represents the partitions assigned to the consumer. Among them, the gray partition partition0 is assigned to the gray consumer (GreyConsumer1). Each consumer is a service Pod respectively.

[0142] Gray Range Partition Assignor is a gray range partition assigner that assigns partitions in the gray release scenario.

[0143] Based on Figure 3 In the application scenario shown, the Kafka message producer sends messages to specific partitions through a self-developed partitioning algorithm, and the Kafka message consumer assigns the specific Partition0 partition to the grayscale service Pod through a self-developed partition assignment algorithm. This process does not intrude on the business service code and does not affect or change the processing logic of each node for business data.

[0144] Specifically, sending grayscale messages to specific grayscale partitions, the content includes: when the message producer produces messages, it tags the messages, so that the grayscale messages are added with grayscale tags. The grayscale tags can be the specified Key in the message, or specific grayscale header information of the message, etc., which are set according to business requirements.

[0145] An exception is that when the number of partitions in the message middleware is less than 2, there is no grayscale partition, and there is no need to determine whether the message includes a grayscale tag.

[0146] Normally, there is at least one grayscale partition in the message middleware, that is, the number of partitions in the message middleware is not less than 2. At this time, it is judged whether the message sent by the producer hits the grayscale rule, that is, it is judged whether the message sent by the producer contains a grayscale tag.

[0147] If the message sent by the producer contains a grayscale tag, it hits the grayscale rule, and the grayscale message is sent to a specific grayscale partition (partition0) in the message middleware.

[0148] If the message sent by the producer does not contain a grayscale tag, it does not hit the grayscale rule, and a partition other than the specific grayscale partition (i.e., the baseline partition) is selected as the target partition of the message according to the logical algorithm of round-robin average distribution.

[0149] The specific logical algorithm is as follows:

[0150] Based on the Key specified in advance in the baseline message, after hashing, take the modulus, and use the number of baseline partitions to determine the partition number. Calculate the Murmur2 hash value m of the Key; convert the hash value m to a positive number M; obtain the number of all available partitions ap under the message topic in the message middleware, ap > 1, and one of them contains a grayscale partition. Use the following formula to calculate the partition number partition:

[0151] partition = M % (ap - 1) + 1

[0152] Alternatively, if the pre-specified Key does not exist in the baseline message, the message producer assigns a unique identifier nextValue to it. After converting nextValue to a positive number and taking the modulus by the number of partitions, the partition number is determined. Convert nextValue to a positive number V; obtain the number of all available partitions ap under this message topic in the message middleware, ap > 1, and it includes a gray partition. Calculate the partition number partition using the following formula:

[0153] partition = V % (ap - 1) + 1

[0154] Assign the gray partition to the gray service Pod for consumption, which includes: obtaining the number of all partitions N under the specified topic; obtaining the list of baseline consumers subscribing to this topic, where the number of baseline consumers is C, and the number of gray consumers in the gray consumer list is G.

[0155] Calculate the average number of partitions for each baseline consumer:

[0156] CP = (N - 1) / C

[0157] Calculate the number of baseline consumers with extra partitions:

[0158] EC = (N - 1) % C

[0159] Traverse the list of baseline consumers and assign partitions according to the following formula:

[0160] start = CP * i + min(i, EC) + 1

[0161] ②length = CP + (i > EC? 0 : 1)

[0162] where i is the serial number of the baseline consumer.

[0163] Select the partition sublist [start, start + length] from the baseline partition list and assign it to the corresponding baseline consumer.

[0164] Optionally, if G is empty, assign the gray partition to the first baseline consumer.

[0165] In some embodiments, the above partition algorithm is integrated into the business in the form of components without intruding into the business code, which includes:

[0166] Non-intrusive. By introducing the Starter dependency to use the component, there is no need for users to manually configure or modify the business code, thus achieving that Spring Boot Starter does not intrude into the business code.

[0167] Automatic configuration. It utilizes the Spring Boot Starter to automatically configure the Beans required by the component and load them according to conditions.

[0168] Simplified configuration. Through the Starter, a set of related configurations are encapsulated, simplifying the management and adjustment of these configurations by users.

[0169] Business code is unaware. By modifying the injection of the original Kafka producer and consumer factory classes into the Spring Beans, the gray scale capability can be enhanced while retaining the original business-related configurations and logics.

[0170] Pluggability. By defining appropriate interfaces and extension points, the Spring Boot Starter supports plug-in development, enabling the functions of the component to be extended and replaced according to requirements.

[0171] In summary, in the embodiments of the present disclosure, through the combination of the self-developed partitioning algorithm and the self-developed partition allocation algorithm, the gray scale capability integration of Kafka is realized; relying on the Spring's Bean management mechanism, the factory classes of the producers and consumers of the business Kafka are enhanced, and at the same time, based on the design pattern of the Spring boot Starter, it is possible to achieve zero damage to the original business, no code intrusion, out-of-the-box use, automatic assembly, and pluggability.

[0172] The embodiments of the present disclosure implement the gray scale function of the message middleware Kafka in the full-link gray scale link, and achieve a non-code-intrusive method, which has no intrusion, no damage, and little impact on the existing business code, thus realizing the single-vehicle verification method in the production environment and reducing the release risk and the scope of problem impact.

[0173] Figure 4 It is a schematic structural diagram of the message processing device provided by the embodiments of the present disclosure. The message processing device provided by the embodiments of the present disclosure can execute the processing flow provided by the embodiments of the message processing method, as Figure 4 shown, the message processing device 40 includes: an acquisition module 41, an allocation module 42, and a first control module 43; the acquisition module 41 is used to acquire the partition configuration information of the partitions in the message middleware, and the partitions in the message middleware include at least one gray scale partition; the allocation module 42 is used to allocate the gray scale partition to the gray scale consumers based on the partition configuration information; the first control module 43 is used to control the gray scale consumers to acquire gray scale messages from the gray scale partition.

[0174] Optionally, the allocation module 42 is used to identify the gray scale consumers in the consumer list according to the gray scale identifiers of the consumers; acquire the gray scale partition numbers of the gray scale partitions based on the partition configuration information; and bind the gray scale partition numbers to the gray scale consumers.

[0175] Optionally, the first control module 43 includes an authentication unit 431, a first determination unit 432, and a first acquisition unit 433. The authentication unit 431 is configured to authenticate the identity of the current consumer according to the binding relationship between the gray-scale partition number and the gray-scale consumer. The first determination unit 432 is configured to determine that the current consumer is a gray-scale consumer bound to the gray-scale partition when the identity information of the current consumer conforms to the binding relationship with the gray-scale partition number. The first acquisition unit 433 is configured to allow the gray-scale consumer to obtain gray-scale messages from the gray-scale partition.

[0176] Optionally, the partitions in the message middleware further include a plurality of baseline partitions, and the allocation module 42 is further configured to allocate the plurality of baseline partitions to a plurality of baseline consumers based on the partition configuration information. If there is no gray-scale consumer, the gray-scale partition is allocated to the first baseline consumer.

[0177] Optionally, the allocation module 42 includes a second acquisition unit 421, a calculation unit 422, a second determination unit 423, and an allocation unit 424. The second acquisition unit 421 is configured to acquire the number of baseline partitions and the number of baseline consumers. The calculation unit 422 is configured to calculate the average number of partitions per baseline consumer and the number of baseline consumers with additional partitions according to the number of baseline partitions and the number of baseline consumers. The second determination unit 423 is configured to determine the start and end numbers of the baseline partitions for each baseline consumer according to the average number of partitions per baseline consumer and the number of baseline consumers with additional partitions. The allocation unit 424 is configured to allocate the plurality of baseline partitions to a plurality of baseline consumers based on the start and end numbers of the baseline partitions.

[0178] Optionally, the start and end numbers of the baseline partitions include a baseline partition start number. For each baseline consumer, the second determination unit 423 is configured to determine the baseline partition start number of the baseline consumer according to the product of the average number of partitions and the serial number of the baseline consumer plus the serial number of the baseline consumer if the baseline consumer belongs to the baseline consumer with additional partitions; or, determine the baseline partition start number of the baseline consumer according to the product of the average number of partitions and the serial number of the baseline consumer plus the number of baseline consumers with additional partitions if the baseline consumer does not belong to the baseline consumer with additional partitions.

[0179] Optionally, the message processing device 40 further includes a second control module 44, which is configured to control the message producer to send baseline messages to the baseline partitions in the message middleware, and the baseline messages are messages that do not contain gray-scale tags. The first control module 43 is further configured to control the baseline consumers to obtain the baseline messages from at least one baseline partition allocated to the baseline consumers.

[0180] Optionally, the second control module 44 includes a third acquisition unit 441 and a control unit 442. The third acquisition unit 441 is configured to take the remainder of the number of the baseline partitions according to the eigenvalue of the baseline message to obtain a target baseline partition number corresponding to the baseline message. The control unit 442 controls the message producer to send the baseline message to the target baseline partition corresponding to the target baseline partition number.

[0181] Figure 4 The message processing device in the illustrated embodiment can be used to execute the technical solutions in the above method embodiments. The implementation principles and technical effects are similar and will not be elaborated here.

[0182] Figure 5 FIG. 500 is a schematic structural diagram of a vehicle provided by an embodiment of the present disclosure. Exemplarily, as Figure 5 shown, the vehicle includes a memory 501 and a processor 502. Among them, an executable program code 5011 is stored in the memory 501, and the processor 502 is configured to call and execute the executable program code 5011 to execute a message processing method.

[0183] In this embodiment, the vehicle can be divided into functional modules according to the above method examples. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0184] In the case of dividing each functional module corresponding to each function, the vehicle can include an acquisition module, an allocation module, a first control module, etc.

[0185] The vehicle provided in this embodiment is used to execute the above-mentioned message processing method, and thus can achieve the same effect as the above implementation method.

[0186] In the case of adopting an integrated unit, the vehicle can include a processing module and a storage module. Among them, the processing module can be used to control and manage the actions of the vehicle. The storage module can be used to support the vehicle to execute mutual program codes and data, etc.

[0187] Among them, the processing module can be a processor or a controller, which can implement or execute various exemplary logic blocks, modules and circuits described in combination with the disclosure of the present application. The processor can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory.

[0188] Figure 6 This is a schematic structural diagram of the electronic device provided by the embodiments of the present disclosure. The electronic device provided by the embodiments of the present disclosure can execute the processing flow provided by the embodiments of the message processing method, such as Figure 6 As shown, the electronic device 60 includes: a memory 61, a processor 62, a computer program, and a communication interface 63; wherein, the computer program is stored in the memory 61 and is configured to be executed by the processor 62 to perform the message processing method as described above.

[0189] The memory 61, as a non-transitory readable storage medium, can be used to store software programs, vehicle executable instructions, and modules, such as the program instructions / modules corresponding to the message processing method in the embodiments of the present disclosure. The processor 62 executes various functional applications and data processing of the server by running the software programs, instructions, and modules stored in the memory 61, that is, implements the message processing method of the above method embodiments.

[0190] The memory 61 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the vehicle, etc. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 61 may optionally include a memory remotely provided with respect to the processor 62, and these remote memories can be connected to the terminal device through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0191] This embodiment also provides a computer-readable storage medium. Computer program code is stored in the computer-readable storage medium (including, but not limited to, a disk memory, a CD-ROM, an optical memory, etc.). When the computer program code runs on a computer, the computer is caused to execute the above-related method steps to implement a message processing method provided by the above embodiments.

[0192] Specifically, when the computer program code runs on a computer, the computer is caused to execute: obtaining partition configuration information of partitions in a message middleware, where the partitions in the message middleware include at least one gray partition; based on the partition configuration information, allocating the gray partition to a gray consumer; controlling a message producer to send a gray message to the gray partition in the message middleware, where the gray message is a message including a gray label; and controlling the gray consumer to obtain the gray message from the gray partition.

[0193] In some embodiments, the partitions in the message middleware further include a plurality of baseline partitions. When the computer program code runs on a computer, the computer is caused to perform: based on the partition configuration information, allocate the plurality of baseline partitions to a plurality of baseline consumers; if there is no gray consumer, allocate the gray partition to the first baseline consumer.

[0194] In some embodiments, when the computer program code runs on a computer, the computer is caused to perform: obtain the number of baseline partitions and the number of baseline consumers; calculate the average number of partitions per baseline consumer and the number of baseline consumers with additional partitions according to the number of baseline partitions and the number of baseline consumers; determine the start and end numbers of the baseline partitions for each baseline consumer according to the average number of partitions per baseline consumer and the number of baseline consumers with additional partitions; and allocate the plurality of baseline partitions to the plurality of baseline consumers based on the start and end numbers of the baseline partitions.

[0195] In some embodiments, the start and end numbers of the baseline partitions include a start number of the baseline partition. When the computer program code runs on a computer, the computer is caused to perform: for each baseline consumer, if the baseline consumer belongs to the baseline consumers with additional partitions, determine the start number of the baseline partition of the baseline consumer according to the product of the average number of partitions and the serial number of the baseline consumer plus the serial number of the baseline consumer; or, if the baseline consumer does not belong to the baseline consumers with additional partitions, determine the start number of the baseline partition of the baseline consumer according to the product of the average number of partitions and the serial number of the baseline consumer plus the number of baseline consumers with additional partitions.

[0196] In some embodiments, when the computer program code runs on a computer, the computer is caused to perform: control the message producer to send baseline messages to the baseline partitions in the message middleware, where the baseline messages are messages that do not contain gray labels; control the baseline consumers to obtain the baseline messages from at least one baseline partition assigned to the baseline consumers.

[0197] In some embodiments, when the computer program code runs on a computer, the computer is caused to perform: take the remainder of the feature value of the baseline message with respect to the number of baseline partitions to obtain the target baseline partition number corresponding to the baseline message; control the message producer to send the baseline message to the target baseline partition corresponding to the target baseline partition number.

[0198] This embodiment also provides a computer program product. When the computer program product runs on a computer, the computer is caused to perform the above-related steps to implement a message processing method provided in the above embodiment.

[0199] Through the description of the above embodiments, those skilled in the art can understand that, for the convenience and conciseness of description, only the division of the above function modules is used as an example. In actual applications, the above functions can be allocated to different function modules as needed, that is, the internal structure of the device can be divided into different function modules to complete all or part of the functions described above.

[0200] In the embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0201] In the description of the present disclosure, it should be understood that if terms such as "upper", "lower", "front", "rear", "left" and "right" are used to indicate the orientation or positional relationship, it is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the indicated position or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of the present disclosure.

[0202] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0203] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0204] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims.

[0205] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0206] The units involved in the embodiments described in the present disclosure can be implemented in software or in hardware. Among them, the name of the unit does not constitute a limitation to the unit itself in some cases.

[0207] It should be noted that, in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. It should also be noted that the term "comprises", "comprising", or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the statement "comprising a..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0208] The above are only embodiments of the present disclosure and are not used to limit the present disclosure. For those skilled in the art, the present disclosure may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure should be included within the scope of the claims of the present disclosure.

Claims

1. A message processing method, characterized in that: The method comprises: Obtaining partition configuration information of a partition in a message middleware, wherein the partition in the message middleware includes at least one grayscale partition; Based on the partition configuration information, allocating the grayscale partition to a grayscale consumer; Control the grayscale consumer to obtain grayscale messages from the grayscale partition.

2. The method according to claim 1, characterized in that The allocating the grayscale partition to the grayscale consumer based on the partition configuration information includes: According to the grayscale identifier of the consumer, identify the grayscale consumer in the consumer list; Based on the partition configuration information, obtaining a grayscale partition number of the grayscale partition; Bind the grayscale partition number to the grayscale consumer.

3. The method according to claim 2, characterized in that The controlling the grayscale consumer to obtain the grayscale message from the grayscale partition includes: Authenticate the current consumer according to the binding relationship between the grayscale partition number and the grayscale consumer; When the identity information of the current consumer meets the binding relationship with the grayscale partition number, determining that the current consumer is a grayscale consumer bound to the grayscale partition; The grayscale consumer is allowed to obtain grayscale messages from the grayscale partition.

4. The method according to claim 1, characterized in that: The partitions in the message middleware further include a plurality of baseline partitions. After obtaining the partition configuration information of the partitions in the message middleware, the method further includes: Based on the partition configuration information, assigning the plurality of baseline partitions to a plurality of baseline consumers; If the grayscale consumer does not exist, the grayscale partition is assigned to the first baseline consumer.

5. The method according to claim 4, characterized in that The allocating the multiple baseline partitions to multiple baseline consumers based on the partition configuration information includes: Get the number of baseline partitions and the number of baseline consumers; Calculating an average number of partitions for each baseline consumer and a number of baseline consumers having additional partitions based on the number of the baseline partitions and the number of the baseline consumers; Determine the baseline partition start and end numbers of each baseline consumer according to the average number of partitions of each baseline consumer and the number of baseline consumers with additional partitions; Based on the baseline partition start and end numbers, the multiple baseline partitions are allocated to multiple baseline consumers.

6. The method according to claim 5, characterized in that The baseline partition start and end numbers include a baseline partition start number, and determining the baseline partition start and end numbers of each baseline consumer according to the average number of partitions of each baseline consumer and the number of baseline consumers with additional partitions includes: For each baseline consumer, if the baseline consumer is a baseline consumer with additional partitions, the baseline partition start number of the baseline consumer is determined according to the product of the average number of partitions, the sequence number of the baseline consumer and the sum of the sequence number of the baseline consumer; or, If the baseline consumer does not belong to a baseline consumer with additional partitions, the baseline partition start number of the baseline consumer is determined according to the product of the average number of partitions, the sequence number of the baseline consumer and the sum of the number of baseline consumers with additional partitions.

7. The method according to claim 1, characterized in that The method further comprises: Controlling the message producer to send a baseline message to the baseline partition in the message middleware, wherein the baseline message is a message that does not contain a grayscale label; The baseline consumer is controlled to obtain the baseline message from at least one baseline partition to which the baseline consumer is assigned.

8. The method according to claim 7, characterized in that The control message producer sends a baseline message to the baseline partition in the message middleware, where the baseline message is a message that does not contain a grayscale label, including: Taking the modulus of the number of the baseline partitions according to the characteristic value of the baseline message, to obtain the target baseline partition number corresponding to the baseline message; The control message producer sends the baseline message to the target baseline partition corresponding to the target baseline partition number.

9. A message processing device, characterized in that: include: An acquisition module, used to acquire partition configuration information of a partition in a message middleware, wherein the partition in the message middleware includes at least one grayscale partition; An allocation module, configured to allocate the grayscale partition to a grayscale consumer based on the partition configuration information; The first control module is used to control the grayscale consumer to obtain the grayscale message from the grayscale partition.

10. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 8.

Images