Message processing method and device, computer readable storage medium and electronic equipment

By adding backup consumers to the consumer cluster, transferring messages in the message queue to the database and processing them by the business service cluster, the message backlog problem is solved and message middleware downtime is avoided.

CN118939451APending Publication Date: 2024-11-12UBTECH ROBOTICS CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410980719.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

When using message queues to process services, the dramatic increase in producer traffic or consumer performance problems lead to the message consumption speed not keeping up with the producer speed, resulting in a message backlog, which may lead to a downtime of message middleware.

Method used

When the preset message backlog processing conditions are met, alternate consumers are added, through which these alternate consumers read messages from the message queue and write them to the database, and use the business service cluster to process messages in the database to alleviate the backlog.

Benefits of technology

By adding backup consumers to perform message transfer operations, specific business logic processing is avoided, message backlog is efficiently alleviated, and message middleware downtime is prevented.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939451A_ABST
    Figure CN118939451A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of message processing, and particularly relates to a message processing method and device, a computer readable storage medium and electronic equipment. The method comprises the following steps: when a preset message backlog processing condition is met, adding a preset standby consumer in a consumer cluster for message processing; reading the message in the message queue through the standby consumer, and writing the read message into a preset database; and processing the message in the database through a preset business service cluster. According to the message processing method and device, due to the fact that the newly-added standby consumer only needs to carry out the message transfer operation and does not need to carry out specific business logic processing, the messages in the message queue can be efficiently transferred to the database and processed through the business service cluster, and message backlog can be effectively relieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the technical field of message processing, and in particular, relates to a message processing method, device, computer-readable storage medium and electronic device. Background Art

[0002] In software development, message queues are usually used to handle business. Message queues can achieve peak load shaving, asynchronous communication, decoupling and other effects. However, in the process of using message queues, sometimes due to design reasons, sudden surges in producer traffic or performance problems of consumers, the consumer's message consumption speed cannot keep up with the producer's message production speed, and message backlogs will occur. If they are not handled in a timely manner, the message backlog will become more and more serious, and may even cause the message middleware to shut down directly. Summary of the invention

[0003] In view of this, embodiments of the present application provide a message processing method, device, computer-readable storage medium, and electronic device to solve the problem of message backlog that is prone to occur in existing message processing methods.

[0004] A first aspect of an embodiment of the present application provides a message processing method, which may include:

[0005] When the preset message backlog processing conditions are met, a preset standby consumer is added to the consumer cluster that is processing the message;

[0006] Read the messages in the message queue through the standby consumer, and write the read messages into a preset database;

[0007] The messages in the database are processed through a preset business service cluster.

[0008] In a specific implementation of the first aspect, the message backlog processing condition may be that the number of messages in the message queue is greater than a preset message number threshold;

[0009] When the preset message backlog processing condition is met, adding a preset standby consumer to the consumer cluster performing message processing may include:

[0010] Get the number of messages in the message queue;

[0011] When the number of messages in the message queue is greater than the message number threshold, the standby consumer is added to the consumer cluster that processes the messages.

[0012] In a specific implementation of the first aspect, the message backlog processing condition may be receiving a preset manual trigger instruction;

[0013] When the preset message backlog processing condition is met, adding a preset standby consumer to the consumer cluster performing message processing may include:

[0014] When the manual trigger instruction is received, the standby consumer is added to the consumer cluster that processes the message.

[0015] In a specific implementation of the first aspect, the processing of the message in the database by using a preset business service cluster may include:

[0016] Get the number of messages in the database;

[0017] Determining a first number of business services for message processing according to the number of messages in the database; wherein the first number is positively correlated with the number of messages in the database;

[0018] The first number of business services are started in the business service cluster to process the messages in the database.

[0019] In a specific implementation of the first aspect, the processing of the message in the database by using a preset business service cluster may include:

[0020] Obtaining the message writing speed of the database;

[0021] Determining a second number of business services for message processing according to a message writing speed of the database; wherein the second number is positively correlated with the message writing speed of the database;

[0022] The second number of business services are started in the business service cluster to process the messages in the database.

[0023] In a specific implementation of the first aspect, after adding a preset standby consumer to a consumer cluster that processes messages, the method may further include:

[0024] Rebalance the partitions in the message queue between consumers in the consumer cluster.

[0025] In a specific implementation manner of the first aspect, the number of consumers in the consumer cluster is less than or equal to the number of partitions in the message queue.

[0026] A second aspect of an embodiment of the present application provides a message processing device, which may include:

[0027] The consumer adding module is used to add a preset standby consumer to the consumer cluster that is processing messages when the preset message backlog processing conditions are met;

[0028] A message transfer module, used to read messages in the message queue through the standby consumer and write the read messages into a preset database;

[0029] The message processing module is used to process the messages in the database through a preset business service cluster.

[0030] In a specific implementation of the second aspect, the message backlog processing condition may be that the number of messages in the message queue is greater than a preset message number threshold; the consumer addition module may be specifically used to: obtain the number of messages in the message queue; when the number of messages in the message queue is greater than the message number threshold, add the backup consumer to the consumer cluster performing message processing.

[0031] In a specific implementation of the second aspect, the message backlog processing condition may be receiving a preset manual trigger instruction; the consumer adding module may be specifically used to: when the manual trigger instruction is received, add the backup consumer to the consumer cluster performing message processing.

[0032] In a specific implementation of the second aspect, the message processing module can be specifically used to: obtain the number of messages in the database; determine a first number of business services for message processing based on the number of messages in the database; wherein the first number is positively correlated with the number of messages in the database; and start the first number of business services in the business service cluster to process the messages in the database.

[0033] In a specific implementation of the second aspect, the message processing module can be specifically used to: obtain the message writing speed of the database; determine a second number of business services for message processing based on the message writing speed of the database; wherein the second number is positively correlated with the message writing speed of the database; and start the second number of business services in the business service cluster to process the messages in the database.

[0034] In a specific implementation of the second aspect, the message processing device may further include:

[0035] The rebalancing module is used to rebalance the partitions in the message queue among the consumers in the consumer cluster.

[0036] In a specific implementation manner of the second aspect, the number of consumers in the consumer cluster is less than or equal to the number of partitions in the message queue.

[0037] A third aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned message processing methods are implemented.

[0038] The fourth aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned message processing methods when executing the computer program.

[0039] A fifth aspect of an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the steps of any one of the above-mentioned message processing methods.

[0040] Compared with the prior art, the embodiments of the present application have the following beneficial effects: when the preset message backlog processing conditions are met, the embodiments of the present application add a preset standby consumer to the consumer cluster that processes messages; the messages in the message queue are read by the standby consumer, and the read messages are written to a preset database; the messages in the database are processed by the preset business service cluster. In the embodiments of the present application, since the newly added standby consumer only needs to perform the message transfer operation without performing specific business logic processing, the messages in the message queue can be efficiently transferred to the database, and processed by the business service cluster, thereby effectively alleviating the message backlog. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 This is a schematic diagram of using message queues for business processing;

[0043] Figure 2 This is a flow chart of an embodiment of a message processing method in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of a specific example of a message processing method in an embodiment of the present application;

[0045] Figure 4 This is a structural diagram of an embodiment of a message processing device in an embodiment of the present application;

[0046] Figure 5This is a schematic block diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0049] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0050] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0051] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0052] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0053] In software development, message queues are usually used to handle business. Message queues can achieve peak load shaving, asynchronous communication, decoupling and other effects. However, in the process of using message queues, sometimes due to design reasons, sudden surges in producer traffic or performance problems of consumers, the consumer's message consumption speed cannot keep up with the producer's message production speed. At this time, message backlogs will occur. If they are not handled in time, the message backlog will become more and more serious, and may even cause the message middleware to shut down directly.

[0054] In view of this, embodiments of the present application provide a message processing method, device, computer-readable storage medium, and electronic device to solve the problem of message backlog that is prone to occur in existing message processing methods.

[0055] In an embodiment of the present application, when the preset message backlog processing conditions are met, a preset backup consumer can be added to the consumer cluster performing message processing; the messages in the message queue are read by the backup consumer, and the read messages are written to a preset database. Since the newly added backup consumer only needs to perform message transfer operations without performing specific business logic processing, the messages in the message queue can be efficiently transferred to the database and processed by the business service cluster, thereby effectively alleviating the message backlog.

[0056] The execution subject of the embodiments of the present application may be an electronic device, which may include but is not limited to computing devices such as mobile phones, tablet computers, desktop computers, notebooks, PDAs, robots, and servers.

[0057] Figure 1 The figure shows a schematic diagram of using a message queue for business processing, wherein, in the embodiment of the present application, any message queue in the prior art can be used according to the actual situation, for example, it can include but is not limited to Kafka, RabbitMQ, RocketMQ, ActiveMQ, ZeroMQ and MetaMq, etc., which are not specifically limited in the embodiment of the present application. For the convenience of description, unless otherwise specified, Kafka is used as an example in the embodiment of the present application, and the processing methods of other message queues are similar to them, which will not be repeated here.

[0058] In the embodiment of the present application, there may be one or more producers (Producer) for message production and one or more consumers (Consumer) for message processing. Producers can send messages to topics (Topic), and consumers can get messages from topics. For performance considerations, a topic can include more than one partition (Partition). In fact, the producer sends messages to the partition of a specific topic, and the consumer also gets messages from the partition of a specific topic. It should be noted that in RocketMQ, the concept similar to partition is queue (Queue). For the sake of ease of description, the embodiment of the present application uniformly describes it as partition.

[0059] exist Figure 1 In the example shown, there are three producers, namely producer 1, producer 2 and producer 3, and three consumers, namely consumer 1, consumer 2 and consumer 3, which form a consumer cluster (Consumer Group). There are four partitions in the topic, namely partition 1, partition 2, partition 3 and partition 4. Among them, producer 1 sends messages to partition 1 and partition 4, producer 2 sends messages to partition 2, and producer 3 sends messages to partition 3. Consumer 1 obtains messages from partition 1 and partition 4 and performs business logic processing on the obtained messages. Consumer 2 obtains messages from partition 2 and performs business logic processing on the obtained messages. Consumer 3 obtains messages from partition 3 and performs business logic processing on the obtained messages.

[0060] In an embodiment of the present application, when a preset message backlog processing condition is met, a corresponding message backlog processing flow may be executed, wherein the message backlog processing condition includes but is not limited to the number of messages in the message queue being greater than a preset message number threshold, and / or receiving a preset manual trigger instruction.

[0061] Taking the message backlog processing condition that the number of messages in the message queue is greater than the message number threshold as an example, the number of messages in the message queue can be obtained and compared with the message number threshold. When the number of messages in the message queue is less than or equal to the message number threshold, the electronic device can continue to process the messages normally. When the number of messages in the message queue is greater than the message number threshold, it indicates that the message backlog processing condition has been met, and the electronic device can execute the corresponding message backlog processing process. Among them, the specific value of the message number threshold can be set according to the actual situation, and the embodiment of the present application does not specifically limit it.

[0062] Taking the message backlog processing condition of receiving a manual trigger instruction as an example, relevant personnel can actively issue instructions for the corresponding message backlog processing process through physical buttons, virtual buttons, voice control or other instruction issuance methods according to actual conditions, that is, manual trigger instructions. When the electronic device does not receive a manual trigger instruction, it can continue to process the message normally. When the electronic device receives a manual trigger instruction, it indicates that the message backlog processing condition has been met and the corresponding message backlog processing process can be executed.

[0063] In a specific implementation of the embodiment of the present application, when the message backlog processing condition is met, the electronic device may execute the following Figure 2 The message backlog processing flow is shown:

[0064] Step S201: Add a preset standby consumer to the consumer cluster that processes messages.

[0065] In a message queue, a partition can only be consumed by one consumer, so when a message backlog occurs, blindly adding consumers may not necessarily solve the problem, because when the number of consumers is greater than the number of partitions, these redundant consumers are actually idle and cannot obtain any messages, which does not help solve the problem of message backlog. However, it is not possible to deliberately add a large number of partitions for this situation, which will actually affect performance. In a specific implementation method of an embodiment of the present application, an evaluation can be performed based on the required throughput of the message queue to determine the number benchmark of partitions, and then a preset ratio can be increased based on this number benchmark to obtain the required number of partitions. Among them, the specific value of the preset ratio can be set according to the actual situation. For example, it can be set to 10%, 20%, 30% or other values, and the embodiment of the present application does not specifically limit it.

[0066] In a specific implementation of the embodiment of the present application, the number of consumers in the consumer cluster is less than or equal to the number of partitions in the message queue. The number of consumers is the sum of the number of original consumers and the number of standby consumers, and the original consumers refer to consumers that perform normal message processing, that is, consumers that have been performing message processing before executing the message backlog processing process. As an example, the number of consumers can be 1 to 2 less than the number of partitions, so as to avoid the situation where the standby consumers cannot be allocated partitions to consume when adding standby consumers.

[0067] Adding a backup consumer can trigger rebalancing of the message queue, that is, rebalancing the partitions in the message queue among the consumers in the consumer cluster (including the original consumers and the newly added backup consumers), and some partitions can be allocated to the backup consumers for consumption.

[0068] Step S202: Read the messages in the message queue through the standby consumer, and write the read messages into a preset database.

[0069] Considering the limitation of the number of partitions on the number of consumers, the number of newly added standby consumers is limited. If standby consumers are used to perform specific business logic processing, message backlog may still occur. Therefore, in the embodiment of the present application, the newly added standby consumers do not need to perform specific business logic processing, but only need to perform message transfer operations, that is, transfer the messages in the message queue to the database. Among them, the specific type of database can be set according to the actual situation, for example, it can include but not limited to MySQL, Redis, SQL Server, Oracle and Doris, etc., which are not specifically limited in the embodiment of the present application.

[0070] Step S203: Process the message in the database through the preset business service cluster.

[0071] In a specific implementation of the embodiment of the present application, the number of messages in the database can be obtained, and according to the number of messages in the database, the first number of business services (Service) for message processing can be determined, and then the first number of business services can be started in the business service cluster to process the messages in the database. Among them, the first number is positively correlated with the number of messages in the database, that is, the more the number of messages in the database, the more the first number of business services for message processing, and vice versa, the fewer the number of messages in the database, the fewer the first number of business services for message processing.

[0072] In another specific implementation of the embodiment of the present application, the message writing speed of the database can be obtained, and according to the message writing speed of the database, the second number of business services for message processing can be determined, and then the second number of business services for message processing can be started in the business service cluster to process the messages in the database. Among them, the second number is positively correlated with the message writing speed of the database, that is, the faster the message writing speed of the database, the more the second number of business services for message processing, and conversely, the slower the message writing speed of the database, the fewer the second number of business services for message processing.

[0073] In this way, sufficient business services can be dynamically and flexibly started to perform specific business logic processing based on the number of messages in the database or the speed at which messages are written to the database, thereby effectively alleviating message backlogs.

[0074] Figure 3The figure shows a schematic diagram of a specific example of a message processing method in an embodiment of the present application. As shown in the figure, when performing normal message processing, there are three producers, namely producer 1, producer 2 and producer 3, three consumers, namely consumer 1, consumer 2 and consumer 3, and six partitions in the topic, namely partition 1, partition 2, partition 3, partition 4, partition 5 and partition 6. Among them, producer 1 sends messages to partition 1 and partition 4, producer 2 sends messages to partition 2 and partition 5, producer 3 sends messages to partition 3 and partition 6, consumer 1 obtains messages from partition 1 and partition 4, and performs business logic processing on the obtained messages, consumer 2 obtains messages from partition 2 and partition 5, and performs business logic processing on the obtained messages, consumer 3 obtains messages from partition 3 and partition 6, and performs business logic processing on the obtained messages. When the message backlog processing conditions are met, backup consumer 1 and backup consumer 2 are added, triggering rebalancing of the message queue. Backup consumer 1 obtains messages from partition 5 and writes them to the database. Backup consumer 2 obtains messages from partition 6 and writes them to the database. Business service 1, business service 2, business service 3, etc. in the business service cluster process the messages in the database respectively.

[0075] In summary, in the embodiment of the present application, when the preset message backlog processing conditions are met, a preset standby consumer is added to the consumer cluster that processes the message; the message in the message queue is read by the standby consumer, and the read message is written into a preset database; the message in the database is processed by the preset business service cluster. In the embodiment of the present application, since the newly added standby consumer only needs to perform the message transfer operation without performing specific business logic processing, the messages in the message queue can be efficiently transferred to the database, and processed by the business service cluster, thereby effectively alleviating the message backlog.

[0076] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0077] Corresponding to a message processing method described in the above embodiment, Figure 4 A structural diagram of an embodiment of a message processing device provided in an embodiment of the present application is shown.

[0078] In this embodiment, a message processing device may include:

[0079] The consumer adding module 401 is used to add a preset standby consumer to the consumer cluster performing message processing when a preset message backlog processing condition is met;

[0080] The message transfer module 402 is used to read the messages in the message queue through the standby consumer and write the read messages into a preset database;

[0081] The message processing module 403 is used to process the messages in the database through a preset business service cluster.

[0082] In a specific implementation method of an embodiment of the present application, the message backlog processing condition may be that the number of messages in the message queue is greater than a preset message number threshold; the consumer addition module may be specifically used to: obtain the number of messages in the message queue; when the number of messages in the message queue is greater than the message number threshold, add the backup consumer to the consumer cluster performing message processing.

[0083] In a specific implementation of an embodiment of the present application, the message backlog processing condition may be receiving a preset manual trigger instruction; the consumer adding module may be specifically used to: when the manual trigger instruction is received, add the backup consumer to the consumer cluster performing message processing.

[0084] In a specific implementation method of an embodiment of the present application, the message processing module can be specifically used to: obtain the number of messages in the database; determine a first number of business services for message processing based on the number of messages in the database; wherein the first number is positively correlated with the number of messages in the database; and start the first number of business services in the business service cluster to process the messages in the database.

[0085] In a specific implementation method of an embodiment of the present application, the message processing module can be specifically used to: obtain the message writing speed of the database; determine a second number of business services for message processing based on the message writing speed of the database; wherein the second number is positively correlated with the message writing speed of the database; and start the second number of business services in the business service cluster to process the messages in the database.

[0086] In a specific implementation of the embodiment of the present application, the message processing device may further include:

[0087] The rebalancing module is used to rebalance the partitions in the message queue among the consumers in the consumer cluster.

[0088] In a specific implementation manner of the embodiment of the present application, the number of consumers in the consumer cluster is less than or equal to the number of partitions in the message queue.

[0089] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0090] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0091] Figure 5 A schematic block diagram of an electronic device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0092] like Figure 5 As shown, the electronic device 5 of this embodiment includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, the steps in the above-mentioned message processing method embodiments are implemented, for example Figure 2 Alternatively, when the processor 50 executes the computer program 52, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 4 The functions of modules 401 to 403 are shown.

[0093] Exemplarily, the computer program 52 may be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 52 in the electronic device 5.

[0094] The electronic device 5 may include but is not limited to computing devices such as mobile phones, tablet computers, desktop computers, notebooks, PDAs, robots, and servers. Those skilled in the art will appreciate that Figure 5 It is only an example of the electronic device 5 and does not constitute a limitation of the electronic device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 5 may also include input and output devices, network access devices, buses, etc.

[0095] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0096] The memory 51 may be an internal storage unit of the electronic device 5, such as a hard disk or memory of the electronic device 5. The memory 51 may also be an external storage device of the electronic device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5. Further, the memory 51 may also include both an internal storage unit of the electronic device 5 and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the electronic device 5. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0097] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0098] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0099] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0100] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0101] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0102] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0103] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0104] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A message processing method, It is characterized in that include: When the preset message backlog processing conditions are met, a preset standby consumer is added to the consumer cluster that is processing the message; Read the messages in the message queue through the standby consumer, and write the read messages into a preset database; The messages in the database are processed through a preset business service cluster.

2. The message processing method according to claim 1, It is characterized in that The message backlog processing condition is that the number of messages in the message queue is greater than a preset message number threshold; When the preset message backlog processing condition is met, adding a preset standby consumer to the consumer cluster performing message processing includes: Get the number of messages in the message queue; When the number of messages in the message queue is greater than the message number threshold, the standby consumer is added to the consumer cluster that processes the messages.

3. The message processing method according to claim 1, It is characterized in that The message backlog processing condition is receiving a preset manual trigger instruction; When the preset message backlog processing condition is met, adding a preset standby consumer to the consumer cluster performing message processing includes: When the manual trigger instruction is received, the standby consumer is added to the consumer cluster that processes the message.

4. The message processing method according to claim 1, It is characterized in that The processing of the message in the database by using a preset business service cluster includes: Get the number of messages in the database; Determining a first number of business services for message processing according to the number of messages in the database; wherein the first number is positively correlated with the number of messages in the database; The first number of business services are started in the business service cluster to process the messages in the database.

5. The message processing method according to claim 1, It is characterized in that The processing of the message in the database by a preset business service cluster includes: Obtaining the message writing speed of the database; Determining a second number of business services for message processing according to a message writing speed of the database; wherein the second number is positively correlated with the message writing speed of the database; The second number of business services are started in the business service cluster to process the messages in the database.

6. The message processing method according to claim 1, It is characterized in that After adding a preset standby consumer to the consumer cluster for message processing, it also includes: Rebalance the partitions in the message queue between consumers in the consumer cluster.

7. The message processing method according to any one of claims 1 to 6, It is characterized in that The number of consumers in the consumer cluster is less than or equal to the number of partitions in the message queue.

8. A message processing device, It is characterized in that include: The consumer adding module is used to add a preset standby consumer to the consumer cluster that is processing messages when the preset message backlog processing conditions are met; A message transfer module, used to read messages in the message queue through the standby consumer and write the read messages into a preset database; The message processing module is used to process the messages in the database through a preset business service cluster.

9. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the steps of the message processing method according to any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the message processing method according to any one of claims 1 to 7 are implemented.