Methods, devices, electronic equipment and storage media for controlling messages

Abstract

This invention relates to a message control method, apparatus, electronic device, and storage medium. The method includes: configuring a message traffic threshold for a target cluster, the message traffic threshold representing the upper limit of the message traffic processing capacity of the target cluster; monitoring the target traffic of a target message queue corresponding to the target cluster; determining whether to execute a capacity transformation strategy based on the target traffic and the message traffic threshold; and controlling the capacity of the target cluster according to the capacity transformation strategy when it is determined to execute the capacity transformation strategy. This enables dynamic adjustment of the number of clusters and traffic scaling control based on traffic flow, transforming manual processing into automatic processing, reducing labor costs and the drawbacks of relying on experience-based adjustments, solving the problem of dynamic processing capacity for large-volume messages, maximizing the utilization of machine resources, improving service message throughput, and reducing operation and maintenance costs.

Description

Technical Field

[0001] The present invention relates to the technical field of controlling message processing capabilities, and more particularly to a message control method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the existing architecture, the upper limit of the message cluster's consumption capacity is affected by the number of message queue shards; when the message volume exceeds the consumption capacity, message backlog will occur.

[0003] The current approach involves manually dividing a topic into multiple topics to increase the number of shards, but this is very costly. In addition, the service needs to be configured after adding a new topic, which greatly increases the input of manual costs and the drawbacks of relying on experience for adjustment. At the same time, the adjustment time during the manual adjustment process is longer than the time to increase the number of shards, which leads to processing anomalies and affects the overall message processing capacity. Summary of the Invention

[0004] In view of this, in order to solve the above-mentioned technical problem of manually controlling message processing capabilities, embodiments of the present invention provide a message control method, apparatus, electronic device, and storage medium.

[0005] In a first aspect, embodiments of the present invention provide a message control method, comprising:

[0006] Configure the message traffic threshold for the target cluster, whereby the message traffic threshold represents the upper limit of the message traffic processing capacity of the target cluster;

[0007] Monitor the target traffic of the target message queue corresponding to the target cluster;

[0008] Based on the target traffic and the message traffic threshold, determine whether to execute a capacity transformation strategy;

[0009] When determining to implement the capacity transformation strategy, the capacity of the target cluster is controlled according to the capacity transformation strategy.

[0010] In one possible implementation, determining whether to execute a capacity transformation strategy based on the target traffic and the message traffic threshold includes:

[0011] When the target traffic exceeds the message traffic threshold within the first preset time period, the expansion strategy will be executed.

[0012] When the target traffic is lower than the message traffic threshold within a second preset time period, a scaling-down strategy is determined to be executed, wherein the first preset time period is shorter than the second preset time period.

[0013] In one possible implementation, controlling the capacity of the target cluster according to the capacity transformation strategy includes:

[0014] The expansion amount is determined according to the expansion strategy, where the expansion amount is the number of clusters to be added;

[0015] Based on the stated expansion quantity, control is applied to the target cluster to increase the cluster size.

[0016] In one possible implementation, controlling the capacity of the target cluster according to the capacity transformation strategy includes:

[0017] The number of clusters to be reduced is determined according to the reduction strategy, where the number of clusters to be reduced is the number of clusters to be recycled.

[0018] Based on the stated reduction quantity, control is performed to reclaim the target cluster.

[0019] In one possible implementation, determining the expansion amount according to the expansion strategy includes:

[0020] Get the average traffic threshold for a single cluster;

[0021] The difference between the target traffic and the message traffic threshold is taken as the first traffic difference;

[0022] The first flow difference is used to divide the average flow threshold to obtain a first quotient value, which is then rounded up to determine the expansion quantity.

[0023] In one possible implementation, controlling the addition of clusters to the target cluster based on the expansion amount includes:

[0024] When the expansion quantity is one, an expansion message queue is created through the first message interface, and the first configuration data of the expansion message queue is stored in the target database;

[0025] The first service interface is invoked to create an expansion cluster of the target cluster, and the expansion cluster shares the same data center and the same regional resources as the expansion message queue;

[0026] The first configuration data stored in the target database is obtained through the expanded cluster.

[0027] In one possible implementation, determining the reduction quantity according to the reduction strategy includes:

[0028] Get the average traffic threshold for a single cluster;

[0029] The difference between the target traffic and the message traffic threshold is taken as the second traffic difference;

[0030] The average flow threshold is divided by the second flow difference to obtain a second quotient, which is then rounded down to determine the reduction quantity.

[0031] In one possible implementation, controlling the reclamation of the target cluster based on the reduction amount includes:

[0032] When the number of reductions is one, control message traffic is prohibited from being distributed to the reduction message queue;

[0033] The second service interface is invoked to reclaim one of the scaled-down clusters in the target cluster. The scaled-down cluster shares the same data center and regional resources as the scaled-down message queue.

[0034] Delete the shrinking message queue and clear the second configuration data corresponding to the shrinking message queue stored in the target database.

[0035] In one possible implementation, the method further includes:

[0036] When there are multiple expansion quantities, multiple expansion message queues are created through the first message interface, and the configuration data of the multiple expansion message queues are stored in the target database respectively.

[0037] The first service interface is invoked to create multiple expansion clusters of the target cluster, and the step of obtaining the first configuration data stored in the target database through the expansion cluster is executed.

[0038] In one possible implementation, the method further includes:

[0039] When there are multiple scaling-up numbers, control message traffic is prohibited from being distributed to multiple scaling-up message queues;

[0040] The second service interface is invoked to reclaim multiple scaled-down clusters in the target cluster, and the step of deleting the scaled-down message queue is executed.

[0041] In a second aspect, embodiments of the present invention provide a message control device, comprising:

[0042] The configuration module is used to configure the message traffic threshold of the target cluster, wherein the message traffic threshold represents the upper limit of the message traffic processing capacity of the target cluster;

[0043] The detection module is used to monitor the target traffic of the target message queue corresponding to the target cluster;

[0044] The determination module is used to determine whether to execute a capacity transformation strategy based on the target traffic and the message traffic threshold.

[0045] The control module is used to control the capacity of the target cluster according to the capacity transformation strategy when it is determined to execute the capacity transformation strategy.

[0046] Thirdly, embodiments of the present invention provide an electronic device, including: a processor and a memory, wherein the processor is configured to execute a control processing program for messages stored in the memory to implement the message control method described in any of the first aspects.

[0047] Fourthly, embodiments of the present invention provide a storage medium storing one or more programs, which can be executed by one or more processors to implement the message control method described in any of the first aspects.

[0048] The message control scheme provided in this invention configures a message traffic threshold for a target cluster, where the message traffic threshold represents the upper limit of the message traffic processing capacity of the target cluster; monitors the target traffic of the target message queue corresponding to the target cluster; determines whether to execute a capacity transformation strategy based on the target traffic and the message traffic threshold; and controls the capacity of the target cluster according to the capacity transformation strategy when it is determined to execute the capacity transformation strategy. By setting a message traffic threshold and comparing the detected target traffic with the message traffic threshold, it is determined whether a capacity transformation strategy needs to be executed; and the capacity transformation strategy is used to add or remove clusters in the target cluster. This scheme can realize the dynamic change of the number of clusters and the scaling control of traffic based on traffic, turning manual processing into automatic processing, reducing the input of manual costs and the drawbacks of relying on experience-based control, solving the problem of dynamic processing capacity for large-volume messages, and achieving the technical effects of maximizing the utilization of machine resources, improving service message throughput, and reducing operation and maintenance costs. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0052] Figure 1 A flowchart illustrating a message control method provided in an embodiment of the present invention;

[0053] Figure 2 A flowchart illustrating another message control method provided in an embodiment of the present invention;

[0054] Figure 3 A flowchart illustrating a message expansion strategy provided in an embodiment of the present invention;

[0055] Figure 4 A flowchart illustrating another message control method provided in an embodiment of the present invention;

[0056] Figure 5 A flowchart illustrating a message scaling-down strategy provided in an embodiment of the present invention;

[0057] Figure 6 A schematic diagram of a message control device provided in an embodiment of the present invention;

[0058] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] In this invention, the terms "comprising" and "having" are used to indicate an open-ended inclusion, meaning that additional elements / components / etc. may exist besides those listed. The terms "first" and "second," etc., are used only as labels and are not intended to limit the number of objects. Furthermore, the different elements and areas in the drawings are only schematic, and therefore the invention is not limited to the dimensions or distances shown in the drawings.

[0061] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0062] Figure 1 This is a flowchart illustrating a message control method provided in an embodiment of the present invention. The executing entity of the present invention is the system in which message control is performed. Figure 1The provided diagrams and message control methods specifically include:

[0063] S101. Configure the message traffic threshold for the target cluster. The message traffic threshold represents the upper limit of the message traffic processing capacity of the target cluster.

[0064] This invention is applied to a message service system for regulating message traffic. By setting a message traffic threshold, the detected target traffic is compared with the threshold to determine whether a capacity adjustment strategy needs to be implemented. The capacity adjustment strategy is then used to add or remove clusters from the target cluster. This allows for dynamic adjustment of the number of clusters and control of traffic scaling based on traffic volume, transforming manual processing into automated processing. This reduces labor costs and the drawbacks of relying on experience for control, solving the problem of dynamic processing of large-volume messages. It maximizes the utilization of machine resources, improves service message throughput, and reduces operational costs.

[0065] The target cluster mentioned here can be understood as a collection used to allocate message processing by the message service system. This target cluster may contain one or more clusters. The message traffic threshold mentioned here can be understood as the maximum message traffic value that each cluster can handle when it is in operation.

[0066] Furthermore, by detecting the upper limit of the message traffic processing capacity of the target cluster, which includes all clusters, in the operation of the message service system, a message traffic threshold for the target cluster is obtained, which is used to monitor whether the message traffic of the message service system exceeds the upper limit of the message traffic processing capacity in the next step.

[0067] S102. Monitor the target traffic of the target message queue corresponding to the target cluster.

[0068] The target message queue mentioned here can be understood as a list that stores retrieved messages. Each cluster is associated with a corresponding message queue through a database. The target traffic mentioned here can be understood as the actual traffic value retrieved in the current target message queue.

[0069] Furthermore, the monitoring module periodically monitors the actual traffic volume of the target message queue to determine whether the current traffic volume of the target message queue exceeds the traffic threshold.

[0070] S103. Based on the target traffic and message traffic threshold, determine whether to execute the capacity transformation strategy.

[0071] The capacity adjustment strategy mentioned here can be understood as adjusting the number of clusters in the target cluster and the number of target message queues corresponding to the target cluster, thereby adjusting the message processing capacity.

[0072] Furthermore, when the upper limit of the traffic processing capacity, namely the message traffic threshold and the actual traffic value, is obtained, the relationship between the two is used to determine whether the current working state of the target cluster exceeds the message traffic capacity. If it exceeds the current message traffic processing capacity, a new cluster needs to be added to meet the message processing requirements. If the target traffic is much lower than the message traffic threshold, the number of clusters needs to be reduced to save memory space.

[0073] S104. When determining the execution of the capacity transformation strategy, control the capacity of the target cluster according to the capacity transformation strategy.

[0074] The control mentioned here can be understood as the operation of increasing or decreasing the number of clusters in the target cluster and the number of message queues that store the corresponding data, so as to dynamically adjust the target cluster and message queues.

[0075] Furthermore, under the condition of adjusting the target cluster and target message queue, when the current message traffic exceeds the processing range, control is implemented to add a new cluster and a new message queue to the target cluster; when the target traffic is far below the message traffic threshold, in order to save memory space, the number of clusters and the number of target message queues need to be reduced to save space. This achieves dynamic adjustment of the number of clusters and traffic scaling control based on traffic, transforming manual processing into automated processing, reducing labor costs and the drawbacks of relying on experience-based adjustments, solving the dynamic processing capability of large-volume messages, maximizing the utilization of machine resources, improving service message throughput, and reducing operational costs.

[0076] The message control method provided in this invention configures a message traffic threshold for a target cluster, which represents the upper limit of the message traffic processing capacity of the target cluster; monitors the target traffic of the target message queue corresponding to the target cluster; determines whether to execute a capacity transformation strategy based on the target traffic and the message traffic threshold; and controls the capacity of the target cluster according to the capacity transformation strategy when it is determined to execute the capacity transformation strategy. By setting a message traffic threshold and comparing the detected target traffic with the message traffic threshold, it is determined whether a capacity transformation strategy needs to be executed; and the capacity transformation strategy is used to add or remove clusters in the target cluster. This solution can dynamically change the number of clusters and control the scaling up and down of traffic based on traffic, turning manual processing into automatic processing, reducing the input of manual costs and the drawbacks of relying on experience-based control, solving the problem of dynamic processing capacity for large-volume messages, and achieving the technical effects of maximizing the utilization of machine resources, improving service message throughput, and reducing operation and maintenance costs.

[0077] Figure 2This is a flowchart illustrating another message control method provided in an embodiment of the present invention. The executing entity of the present invention is the system in which the message control is performed. Figure 2 This is based on the previous embodiment. Figure 2 The provided diagrams and message control methods specifically include:

[0078] S201. Configure the message traffic threshold for the target cluster. The message traffic threshold represents the upper limit of the message traffic processing capacity of the target cluster.

[0079] This invention is applied to a message service system for regulating message traffic. By setting a message traffic threshold, the detected target traffic is compared with the threshold to determine whether a capacity adjustment strategy needs to be implemented. The capacity adjustment strategy is then used to add or remove clusters from the target cluster. This allows for dynamic adjustment of the number of clusters and control of traffic scaling based on traffic volume, transforming manual processing into automated processing. This reduces labor costs and the drawbacks of relying on experience for control, solving the problem of dynamic processing of large-volume messages. It maximizes the utilization of machine resources, improves service message throughput, and reduces operational costs.

[0080] The target cluster mentioned here can be understood as a set of clusters used to allocate message processing by the message service system. This target cluster may contain one or more clusters. For example, when the target cluster contains only one cluster, the message processing capacity can only be increased by increasing the number of target clusters. When the target cluster contains multiple clusters, not only can the message processing capacity be increased by increasing the number of target clusters, but system space can also be saved by reducing the number of clusters within the target cluster. The message traffic threshold mentioned here can be understood as the maximum message traffic value that each cluster can handle when it is in a working state.

[0081] Furthermore, by detecting the upper limit of the message traffic processing capacity of the target cluster, which includes all clusters, in the operation of the message service system, a message traffic threshold for the target cluster is obtained, which is used to monitor whether the message traffic of the message service system exceeds the upper limit of the message traffic processing capacity in the next step.

[0082] S202. Monitor the target traffic of the target message queue corresponding to the target cluster.

[0083] The target message queue mentioned here can be understood as a list that stores retrieved messages. Each cluster is associated with a corresponding message queue through a database. The target traffic mentioned here can be understood as the actual traffic value retrieved in the current target message queue.

[0084] Furthermore, the monitoring module periodically monitors the actual traffic volume of the target message queue to determine whether the current traffic volume of the target message queue exceeds the traffic threshold.

[0085] S203. When the target traffic exceeds the message traffic threshold within the first preset time period, determine to execute the expansion strategy.

[0086] The first preset time period mentioned here can be understood as the time cycle for detecting target traffic. For example, setting a detection cycle of one second achieves the purpose of real-time detection of target traffic. The scaling strategy mentioned here can be understood as a strategy to increase the number of clusters in the target cluster, which can involve adding one or more clusters.

[0087] Furthermore, the target traffic detected in real time within the first preset time period is compared with the set message traffic threshold to determine whether the target traffic exceeds the message traffic threshold. If the target traffic exceeds the message traffic threshold within the set time period, it is determined that the message processing capacity of the current target cluster cannot meet the demand. At this time, an expansion strategy is executed to increase the message processing capacity by increasing the number of clusters in the target cluster and the number of target message queues.

[0088] S204. When the target traffic is lower than the message traffic threshold within the second preset time period, determine to execute the scaling down strategy, where the first preset time period is shorter than the second preset time period.

[0089] The second preset time period mentioned here can be understood as another time period for detecting target traffic. For example, setting a detection period of one hour achieves the purpose of periodically detecting target traffic. The first preset time period is shorter than the second preset time period, reflecting the message service system's principle of dynamically adjusting the processing of the target cluster. This addresses the issue of message traffic overload to a limited extent and improves message traffic processing capacity. The scaling-down strategy mentioned here can be understood as a strategy to reduce the number of clusters in the target cluster, which can involve reducing one or more clusters.

[0090] Furthermore, the target traffic obtained by periodic detection corresponding to the second preset time period is compared with the set message traffic threshold to determine whether the target traffic is far below the message traffic threshold. When the target traffic is consistently lower than the message traffic threshold within the set time period, it is determined that the message processing capacity of the current target cluster far exceeds the demand, resulting in resource waste. At this time, a scaling-down strategy is executed to reduce the message processing capacity by reducing the number of clusters and the number of target message queues in the target cluster.

[0091] S205. Determine the expansion quantity according to the expansion strategy. The expansion quantity is the number of clusters to be added.

[0092] The expansion mentioned here can be understood as increasing the number of clusters, which can be one or more.

[0093] Specifically, the steps for determining the expansion quantity are implemented in the following manner:

[0094] Step 1: Obtain the average traffic threshold for a single cluster.

[0095] Step 2: Use the difference between the target traffic and the message traffic threshold as the first traffic difference.

[0096] Step 3: Use the first flow difference to perform a division operation on the average flow threshold to obtain the first quotient value, and round it up to determine the expansion quantity.

[0097] The first traffic difference mentioned here can be understood as the portion of the current message traffic that exceeds the processing capacity of the message service system.

[0098] The average traffic threshold is obtained by calculating the message traffic processing capacity of a cluster. The current target traffic is then subtracted from the set message traffic threshold. The difference is taken as the first traffic difference, which is the portion exceeding the processing capacity of the message service system. The first traffic difference is divided by the average traffic threshold of a cluster to determine the number of clusters that can be added to account for the excess traffic. If the first traffic difference is not divisible by the average traffic threshold, the first quotient is rounded up (i.e., the first quotient is incremented by one) to obtain the number of clusters to be expanded.

[0099] In one possible scenario, assume the average traffic threshold for a single cluster is 1000; the message traffic threshold is set to 5000; when the target message in the message service system is detected to be 7500, first obtain the first difference, i.e., 7500-5000=2500, then divide the obtained first difference 2500 by the average traffic threshold to get 2500 / 1000=2.5. Since the first quotient is not an integer, round 2.5 up to get 3, i.e., the expansion quantity is 3.

[0100] Alternatively, the expansion amount can be obtained in other ways. For example, it can be obtained using the modulo division method.

[0101] Assume a message queue topic has m shards, the consumer cluster takes t seconds to process a message, and can process a maximum of n messages per second, where n = (1000ms / t)*m. The maximum processing capacity of a single consumer container is x messages. Therefore, n messages per second require y containers, meaning a cluster contains y containers; y = n / x = (1000ms / t)*m / x. If m = 100, t = 20ms, and x = 1000, then n = (1000ms / t)*m = 5000, and y = n / x = (1000ms / t)*m / x = 5. Based on these assumptions, when the message volume fluctuates between 1 and 5000, the corresponding number of consumer cluster containers fluctuates between 1 and 5. When the message volume exceeds 5000, it will exceed the processing capacity of the cluster, triggering a new topic and dynamic scaling of consumer containers. Let z be the total number of topics and the total number of consumer clusters, and a be the total number of messages. When a%n = 0, z = a / n; that is, the number of clusters that need to be added is zm. When a%n! = 0, z = (round down)(a / n) + 1, and the number of clusters that need to be added is still zm.

[0102] S206. Based on the expansion quantity, perform control over adding clusters to the target cluster.

[0103] S207. When the expansion quantity is one, create an expansion message queue through the first message interface and store the first configuration data of the expansion message queue in the target database.

[0104] S208. Call the first service interface to create an expansion cluster of the target cluster. The expansion cluster and the expansion message queue share the same data center and the same regional resources.

[0105] S209. Obtain the first configuration data stored in the target database by expanding the cluster.

[0106] Furthermore, the component monitors the target traffic of the target message queue. When the target traffic exceeds the message traffic threshold, it dynamically expands the message queue according to a detailed strategy and stores the expansion message queue configuration information in the database. It then calls the message cluster interface to create an expansion cluster. The expansion cluster reads the configuration from the database and consumes messages in the expansion message queue, thus diverting a portion of the traffic to the expansion message queue.

[0107] S210. When there are multiple expansion quantities, multiple expansion message queues are created through the first message interface, and the configuration data of the multiple expansion message queues are stored in the target database respectively.

[0108] S211. Call the first service interface to create multiple expansion clusters of the target cluster, and execute the step of obtaining the first configuration data stored in the target database through the expansion clusters.

[0109] The scaling calculation is implemented at the component level. When the message traffic threshold is exceeded, the excess traffic is allocated to the newly added scaling message queue. The target cluster remains unchanged. When the total message traffic threshold of the target cluster and the scaling cluster is exceeded, the excess traffic is allocated to the newly added second scaling message queue. By expanding the second scaling cluster again, the message processing capacity is dynamically increased.

[0110] In one possible scenario Figure 3 This is a flowchart illustrating a message expansion strategy provided in an embodiment of the present invention. (Reference) Figure 3 The provided diagram illustrates that a scaling strategy is triggered when the target traffic exceeds the message traffic threshold. When scaling up to one queue, message queue B is created based on cluster A and message queue A through the message component interface. After successful creation, message queue B's information is stored in the database. This information includes message queue B's name, connection information, consumer group, etc. This information is written to a fixed MySQL table, and then a unique ID of the stored information is passed to message cluster B, which dynamically creates message cluster B by calling the consumer service interface. Message cluster B and message queue B share the same data center and regional resources to maximize performance. The number of containers in message cluster B is dynamically calculated based on traffic. For example, if one machine is needed for every 1000 queries per second (QPS) of traffic, the number of containers will dynamically adjust according to the message volume. When the message service system processes traffic exceeding the message traffic threshold, it switches to a new topic to retrieve messages. When there are multiple expansions, if the message QPS value in the target message queue exceeds the total traffic processing capacity of message clusters A and B, it will trigger the dynamic expansion of message queue C and message cluster C. The expansion process is similar to that of message cluster B and will not be described in detail. Theoretically, it can be expanded horizontally indefinitely.

[0111] Another message control method provided in this embodiment of the invention sets a message traffic threshold and detects target traffic. By comparing the magnitudes of the two, it determines whether to trigger the capacity transformation strategy of the target cluster. When the target traffic exceeds the message traffic threshold, a capacity expansion strategy is triggered. By counting the number of expansions, expansion message queues and expansion clusters are created to achieve the purpose of dynamically expanding message processing capabilities. This realizes the dynamic change of the number of clusters and the control of traffic scaling based on traffic, transforming manual processing into automatic processing. It reduces the input of manual costs and the drawbacks of relying on experience-based regulation, solves the problem of dynamic processing capabilities for large-volume messages, and achieves the technical effects of maximizing the utilization of machine resources, improving service message throughput, and reducing operation and maintenance costs.

[0112] Figure 4This is a flowchart illustrating another message control method provided in an embodiment of the present invention. The executing entity of the present invention is the system in which the message control is performed. Figure 4 This is based on the first embodiment. Figure 4 The provided diagrams and message control methods specifically include:

[0113] S401. Configure the message traffic threshold for the target cluster. The message traffic threshold represents the upper limit of the message traffic processing capacity of the target cluster.

[0114] The target cluster mentioned here can be understood as a set of clusters used to allocate message processing by the message service system. This target cluster may contain one or more clusters. For example, when the target cluster contains only one cluster, the message processing capacity can only be increased by increasing the number of target clusters. When the target cluster contains multiple clusters, not only can the message processing capacity be increased by increasing the number of target clusters, but system space can also be saved by reducing the number of clusters within the target cluster. The message traffic threshold mentioned here can be understood as the maximum message traffic value that each cluster can handle when it is in a working state.

[0115] Furthermore, by detecting the upper limit of the message traffic processing capacity of the target cluster, which includes all clusters, in the operation of the message service system, a message traffic threshold for the target cluster is obtained, which is used to monitor whether the message traffic of the message service system exceeds the upper limit of the message traffic processing capacity in the next step.

[0116] S402. Monitor the target traffic of the target message queue corresponding to the target cluster.

[0117] The target message queue mentioned here can be understood as a list that stores retrieved messages. Each cluster is associated with a corresponding message queue through a database. The target traffic mentioned here can be understood as the actual traffic value retrieved in the current target message queue.

[0118] Furthermore, the monitoring module periodically monitors the actual traffic volume of the target message queue to determine whether the current traffic volume of the target message queue exceeds the traffic threshold.

[0119] S403. When the target traffic exceeds the message traffic threshold within the first preset time period, determine to execute the expansion strategy.

[0120] The first preset time period mentioned here can be understood as the time cycle for detecting target traffic. For example, setting a detection cycle of one second achieves the purpose of real-time detection of target traffic. The scaling strategy mentioned here can be understood as a strategy to increase the number of clusters in the target cluster, which can involve adding one or more clusters.

[0121] Furthermore, the target traffic detected in real time within the first preset time period is compared with the set message traffic threshold to determine whether the target traffic exceeds the message traffic threshold. If the target traffic exceeds the message traffic threshold within the set time period, it is determined that the message processing capacity of the current target cluster cannot meet the demand. At this time, an expansion strategy is executed to increase the message processing capacity by increasing the number of clusters in the target cluster and the number of target message queues.

[0122] S404. When the target traffic is lower than the message traffic threshold within the second preset time period, determine to execute the scaling down strategy, where the first preset time period is shorter than the second preset time period.

[0123] The second preset time period mentioned here can be understood as another time period for detecting target traffic. For example, setting a detection period of one hour achieves the purpose of periodically detecting target traffic. The first preset time period is shorter than the second preset time period, reflecting the message service system's principle of dynamically adjusting the processing of the target cluster. This addresses the issue of message traffic overload to a limited extent and improves message traffic processing capacity. The scaling-down strategy mentioned here can be understood as a strategy to reduce the number of clusters in the target cluster, which can involve reducing one or more clusters.

[0124] Furthermore, the target traffic obtained by periodic detection corresponding to the second preset time period is compared with the set message traffic threshold to determine whether the target traffic is far below the message traffic threshold. When the target traffic is consistently lower than the message traffic threshold within the set time period, it is determined that the message processing capacity of the current target cluster far exceeds the demand, resulting in resource waste. At this time, a scaling-down strategy is executed to reduce the message processing capacity by reducing the number of clusters and the number of target message queues in the target cluster.

[0125] S405. Determine the number of clusters to be reduced based on the reduction strategy. The number of clusters to be reduced is the number of clusters to be recycled.

[0126] The reduction in scale mentioned here can be understood as reducing the number of clusters. The reduction in scale can be one or more.

[0127] Specifically, the steps for determining the reduction quantity are implemented in the following manner:

[0128] Step 1: Obtain the average traffic threshold for a single cluster.

[0129] Step 2: Use the difference between the target traffic and the message traffic threshold as the second traffic difference.

[0130] Step 3: Use the second flow difference to divide the average flow threshold to obtain the second quotient value, and round it down to determine the reduction quantity.

[0131] The second traffic difference mentioned here can be understood as the portion of the current message traffic that is below the processing capacity of the message service system.

[0132] The average traffic threshold is obtained by calculating the message traffic processing capacity of a cluster. The current target traffic is then subtracted from the set message traffic threshold, and the difference is taken as the second traffic difference. The portion below the message traffic threshold is obtained by dividing the second traffic difference by the average traffic threshold of a cluster. The number of clusters that can be reduced below the message traffic threshold is then calculated. If the second traffic difference is not divisible by the average traffic threshold, the second quotient is rounded down (i.e., the second quotient is subtracted by one) to obtain the number of clusters to be reduced.

[0133] In one possible scenario, assume the average traffic threshold for a single cluster is 1000; set the message traffic threshold to 10000; when the target message in the message service system is detected to be 7500, first obtain the second difference, i.e., 10000-7500=2500, then divide the obtained second difference 2500 by the average traffic threshold to get 2500 / 1000=2.5. Since the second quotient is not an integer, round down to 2, i.e., the number of units to be reduced is 2.

[0134] Alternatively, the reduction quantity can be obtained in other ways. For example, it can be obtained using the modulo method. The method is the same as the expansion method described above, and will not be repeated here.

[0135] S406. Based on the number of clusters to be reduced, control the reclaiming of the target cluster.

[0136] S407. When the number of reductions is one, control message traffic is prohibited from being distributed to the reduction message queue.

[0137] S408. Call the second service interface to reclaim a scaled-down cluster in the target cluster. The scaled-down cluster and the scaled-down message queue share the same data center and the same regional resources.

[0138] S409. Delete the shrinking message queue and clear the second configuration data corresponding to the shrinking message queue stored in the target database.

[0139] The component monitors the target traffic in the target message queue. When the target traffic falls below the target cluster's message traffic threshold for a certain period of time (e.g., one hour), a scaling-down strategy is triggered. Messages are no longer distributed to the target message queue. At the same time, the consumer service interface is called to dynamically reclaim the scaled-down cluster of the target cluster, delete the scaled-down message queue, and clean up the configuration data corresponding to the message queue in the database, thereby saving system runtime space.

[0140] S410. When there are multiple reduction quantities, control message traffic to prevent distribution to multiple reduction message queues.

[0141] S411. Call the second service interface to reclaim multiple scaled-down clusters in the target cluster and execute the step of deleting the scaled-down message queue.

[0142] When the number of shrinking message queues and shrinking clusters that need to be recycled exceeds two, the recycling process is similar to the process of recycling a single shrinking cluster, and will not be repeated here.

[0143] The calculation method for the number of reduced capacity and the corresponding number of topics is similar to that for expansion, and will not be repeated here.

[0144] In one possible scenario Figure 5 This is a flowchart illustrating a message scaling-down strategy provided in an embodiment of the present invention. (Reference) Figure 5 The provided diagram illustrates that when the target traffic falls below the message traffic threshold, a scaling-down strategy is triggered. When the scaling-down operation involves only one item, message clusters A, B, and N are created based on message queues A, B, and N. Traffic is redirected to the original topic, the corresponding message cluster is taken offline, and finally, the topic is recycled. Configuration information in the database is then cleaned up, completing the scaling-down process and saving system space.

[0145] This invention provides another message control method that sets a message traffic threshold and detects target traffic. By comparing the magnitudes of the two, it determines whether to trigger a capacity transformation strategy for the target cluster. When the target traffic is lower than the message traffic threshold, a scaling-down strategy is triggered. By counting the number of scaling-down events, the scaling-down message queue and the scaling-down cluster are restored, achieving the goal of dynamically recycling message processing. This realizes the dynamic adjustment of the cluster size and traffic scaling-up / scaling control based on traffic, transforming manual processing into automatic processing. This reduces the input of manual costs and the drawbacks of relying on experience-based adjustments, solves the problem of dynamic processing capabilities for large-volume messages, and achieves the technical effects of maximizing the utilization of machine resources, improving service message throughput, and reducing operation and maintenance costs.

[0146] Figure 6 This is a schematic diagram of a message control device provided in an embodiment of the present invention. Figure 6 The provided diagram shows that the message control device specifically includes:

[0147] Configuration module 61 is used to configure the message traffic threshold of the target cluster. The message traffic threshold represents the upper limit of the message traffic processing capacity of the target cluster.

[0148] Detection module 62 is used to monitor the target traffic of the target message queue corresponding to the target cluster;

[0149] The determination module 63 is used to determine whether to execute a capacity transformation strategy based on the target traffic and message traffic threshold.

[0150] The control module 64 is used to control the capacity of the target cluster according to the capacity transformation strategy when determining the execution of the capacity transformation strategy.

[0151] The message control device provided in this embodiment may be as follows: Figure 6 The control device for the message shown can perform actions such as Figure 1-5 All steps of the message control method in the middle, thereby realizing Figure 1-5 For details on the technical effects of the message control method shown, please refer to [link / reference]. Figure 1-5 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0152] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 7 The illustrated electronic device 700 includes at least one processor 701, a memory 702, at least one network interface 704, and other user interfaces 703. The various components in the electronic device 700 are coupled together via a bus system 705. It is understood that the bus system 705 is used to implement communication between these components. In addition to a data bus, the bus system 705 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 7 The general labeled all buses as Bus System 705.

[0153] The user interface 703 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0154] It is understood that the memory 702 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 702 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0155] In some implementations, memory 702 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 7021 and application program 7022.

[0156] The operating system 7021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 7022 includes various applications, such as a media player and a browser, used to implement various application functions. The program implementing the method of this embodiment can be included in the application program 7022.

[0157] In this embodiment of the invention, by calling the program or instructions stored in the memory 702, specifically the program or instructions stored in the application program 7022, the processor 701 executes the method steps provided in each method embodiment, including, for example:

[0158] Configure the message traffic threshold for the target cluster, which represents the upper limit of the message traffic processing capacity of the target cluster; monitor the target traffic of the target message queue corresponding to the target cluster; determine whether to execute a capacity transformation strategy based on the target traffic and the message traffic threshold; and control the capacity of the target cluster according to the capacity transformation strategy when it is determined to execute the capacity transformation strategy.

[0159] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 701. Processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 701 or by instructions in software form. The processor 701 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 702. Processor 701 reads the information in memory 702 and, in conjunction with its hardware, completes the steps of the above method.

[0160] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0161] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0162] The electronic device provided in this embodiment may be as follows: Figure 7 The electronic device shown can perform the following: Figure 1-5 All steps of the message control method in the middle, thereby realizing Figure 1-5 For details on the technical effects of the message control method shown, please refer to [link / reference]. Figure 1-5 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0163] This invention also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; the memory may also include combinations of the above types of memory.

[0164] When one or more programs in the storage medium can be executed by one or more processors to implement the message control method described above, which is executed on the message control device side.

[0165] The processor is used to execute a control program for a message stored in memory to implement the following steps of a message control method executed on the message control device side:

[0166] Configure the message traffic threshold for the target cluster, which represents the upper limit of the message traffic processing capacity of the target cluster; monitor the target traffic of the target message queue corresponding to the target cluster; determine whether to execute a capacity transformation strategy based on the target traffic and the message traffic threshold; and control the capacity of the target cluster according to the capacity transformation strategy when it is determined to execute the capacity transformation strategy.

[0167] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0168] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0169] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A message control method, characterized in that, include: Configure the message traffic threshold for the target cluster, whereby the message traffic threshold represents the upper limit of the message traffic processing capacity of the target cluster; Monitor the target traffic of the target message queue corresponding to the target cluster; Based on the target traffic and the message traffic threshold, determine whether to execute a capacity transformation strategy, including: when the target traffic exceeds the message traffic threshold within a first preset time period, determine to execute a capacity expansion strategy; when the target traffic is lower than the message traffic threshold within a second preset time period, determine to execute a capacity reduction strategy, wherein the first preset time period is shorter than the second preset time period. When determining to execute the capacity transformation strategy, the capacity of the target cluster is controlled according to the capacity transformation strategy; The process of controlling the addition of clusters to the target cluster based on the expansion quantity includes: when the expansion quantity is one, creating an expansion message queue through a first message interface and storing the first configuration data of the expansion message queue in the target database; calling a first service interface to create an expansion cluster for the target cluster, wherein the expansion cluster and the expansion message queue share the same data center and the same regional resources; and obtaining the first configuration data stored in the target database through the expansion cluster; when the expansion quantity is multiple, creating multiple expansion message queues through the first message interface and storing the configuration data of the multiple expansion message queues in the target database respectively; calling the first service interface to create multiple expansion clusters for the target cluster and performing the step of obtaining the first configuration data stored in the target database through the expansion cluster. Based on the number of downsizing operations, control is performed on the target cluster to reclaim the cluster, including: when the number of downsizing operations is one, controlling message traffic to prevent distribution to the downsizing message queue; calling the second service interface to reclaim one downsizing cluster in the target cluster, the downsizing cluster sharing the same data center and regional resources as the downsizing message queue; deleting the downsizing message queue and clearing the second configuration data corresponding to the downsizing message queue stored in the target database; when the number of downsizing operations is multiple, controlling message traffic to prevent distribution to multiple downsizing message queues; calling the second service interface to reclaim multiple downsizing clusters in the target cluster, and performing the step of deleting the downsizing message queue.

2. The method according to claim 1, characterized in that, The step of controlling the capacity of the target cluster according to the capacity transformation strategy includes: The expansion amount is determined according to the expansion strategy, where the expansion amount is the number of clusters to be added; Based on the stated expansion quantity, control is applied to the target cluster to increase the cluster size.

3. The method according to claim 1, characterized in that, The step of controlling the capacity of the target cluster according to the capacity transformation strategy includes: The number of clusters to be reduced is determined according to the reduction strategy, where the number of clusters to be reduced is the number of clusters to be recycled. Based on the stated reduction quantity, control is performed to reclaim the target cluster.

4. The method according to claim 2, characterized in that, Determining the expansion quantity according to the expansion strategy includes: Get the average traffic threshold for a single cluster; The difference between the target traffic and the message traffic threshold is taken as the first traffic difference; The first flow difference is used to divide the average flow threshold to obtain a first quotient value, which is then rounded up to determine the expansion quantity.

5. The method according to claim 3, characterized in that, Determining the reduction quantity according to the reduction strategy includes: Get the average traffic threshold for a single cluster; The difference between the target traffic and the message traffic threshold is taken as the second traffic difference; The average flow threshold is divided by the second flow difference to obtain a second quotient, which is then rounded down to determine the reduction quantity.

6. A message control device, characterized in that, include: The configuration module is used to configure the message traffic threshold of the target cluster, wherein the message traffic threshold represents the upper limit of the message traffic processing capacity of the target cluster; The detection module is used to monitor the target traffic of the target message queue corresponding to the target cluster; The determination module is used to determine whether to execute a capacity transformation strategy based on the target traffic and the message traffic threshold. Specifically, it is used to determine to execute a capacity expansion strategy when the target traffic exceeds the message traffic threshold within a first preset time period. When the target traffic is lower than the message traffic threshold within a second preset time period, a scaling-down strategy is determined to be executed, wherein the first preset time period is shorter than the second preset time period. A control module is used to control the capacity of the target cluster according to the capacity transformation strategy when it is determined to execute the capacity transformation strategy; The control module is specifically used for: when the expansion quantity is one, creating an expansion message queue through a first message interface, and storing the first configuration data of the expansion message queue in the target database; calling a first service interface to create an expansion cluster of the target cluster, wherein the expansion cluster and the expansion message queue share the same data center and the same regional resources; The first configuration data stored in the target database is obtained through the expansion cluster; when there are multiple expansions, multiple expansion message queues are created through the first message interface, and the configuration data of the multiple expansion message queues are stored in the target database respectively. Call the first service interface to create multiple expansion clusters of the target cluster, and execute the step of obtaining the first configuration data stored in the target database through the expansion clusters; The control module is specifically configured to: when the number of scaling down is one, control message traffic to prevent distribution to the scaling down message queue; call the second service interface to reclaim one of the scaling down clusters in the target cluster, wherein the scaling down cluster and the scaling down message queue share the same data center and the same regional resources; delete the scaling down message queue and clear the second configuration data corresponding to the scaling down message queue stored in the target database; when the number of scaling down is multiple, control message traffic to prevent distribution to multiple scaling down message queues. The second service interface is invoked to reclaim multiple scaled-down clusters in the target cluster, and the step of deleting the scaled-down message queue is executed.

7. An electronic device, characterized in that, include: A processor and a memory, the processor being configured to execute a control processing program for messages stored in the memory, to implement the message control method according to any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the message control method according to any one of claims 1 to 5.

Images