A message pushing method and device based on a large language model and electronic equipment

Abstract

The application discloses a message pushing method and device based on a large language model and electronic equipment, and relates to the technical field of artificial intelligence. In response to a pushing request initiated by a target tenant, the application searches for an optimized control flow strategy corresponding to the target tenant in a specified cache; according to the search result of the optimized control flow strategy and the existence of a check identifier, a target control flow strategy corresponding to the target tenant is determined; if the current task number of the target tenant reaches a task threshold, the to-be-pushed message of the target tenant is stored in a waiting queue; the large language model is used to analyze the pushing channel state, the length of the waiting queue and the system resource state, so as to push the to-be-pushed message in the waiting queue; if the current task number of the target tenant does not reach the task threshold, the to-be-pushed message is pushed at a target pushing rate, thereby improving the stability and continuity of multi-tenant message pushing.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a message push method, apparatus, and electronic device based on a large language model. Background Technology

[0002] Push notification systems are typically used to send messages generated by business systems to target users, target terminals, or third-party message receiving systems. In multi-tenant push notification scenarios, the same push notification system can provide push notification services to multiple tenants simultaneously. A tenant can be understood as a business entity, client, or application that accesses the push notification system; different tenants may differ in terms of business scale, service level, push frequency, and resource guarantee requirements.

[0003] During message push, the system typically controls the message push process for different tenants based on pre-configured strategies. While these strategies can provide some rate limiting and protection in general scenarios, in actual operation, the dynamic changes in the message push process's state can negatively impact the stability and continuity of multi-tenant message push if the strategies fail to adapt promptly. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a message push method, apparatus, and electronic device based on a large language model, which can improve the stability and continuity of multi-tenant message push.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On the one hand, this invention provides a message push method based on a large language model, the method comprising: In response to a push request initiated by a target tenant, the optimization flow control strategy corresponding to the target tenant is searched in a specified cache. The optimization flow control strategy is determined through a large language model strategy analysis task. Based on the search results of the optimized flow control strategy and the existence of the check identifier, the target flow control strategy corresponding to the target tenant is determined. The check identifier is generated according to a preset period and is cleared when the large language model strategy analysis task is triggered. The target flow control strategy includes the target push rate and the task threshold. If the current number of tasks of the target tenant reaches the task threshold, the messages to be pushed to the target tenant are stored in the waiting queue. The push channel status, waiting queue length, and system resource status are analyzed using a large language model to perform reasoning processing on the messages to be pushed in the waiting queue. If the current number of tasks for the target tenant has not reached the task threshold, the message to be pushed shall be pushed according to the target push rate.

[0006] On the other hand, the present invention also provides a message push device based on a large language model, the device comprising: The lookup module is used to respond to a push request initiated by a target tenant and look up the optimization flow control strategy corresponding to the target tenant in a specified cache. The optimization flow control strategy is determined through a large language model strategy analysis task. The target determination module is used to determine the target flow control strategy corresponding to the target tenant based on the search results of the optimized flow control strategy and the existence of the check identifier. The check identifier is generated according to a preset period and is cleared when the large language model strategy analysis task is triggered. The target flow control strategy includes the target push rate and the task threshold. The waiting module is used to store the target tenant's pending push messages into a waiting queue if the current number of tasks of the target tenant reaches the task threshold. The wake-up module is used to analyze the status of the push channel, the length of the waiting queue, and the status of system resources through a large language model, so as to push the messages to be pushed in the waiting queue. The push module is used to push messages to be pushed at the target push rate if the current number of tasks of the target tenant has not reached the task threshold.

[0007] On the other hand, the present invention also provides an electronic device, including a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to execute steps in any of the message push methods based on a large language model provided by the present invention.

[0008] On the other hand, the present invention also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute steps in any of the message push methods based on a large language model provided by the present invention.

[0009] On the other hand, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in any of the message push methods based on a large language model provided by the present invention.

[0010] The beneficial effects of the technical solution provided by this invention include at least the following: In this embodiment of the invention, when a target tenant initiates a push request, the optimized flow control strategy determined by the large language model strategy analysis task is retrieved from a specified cache. The existence of a check flag is then used to determine the currently executable target flow control strategy, avoiding frequent triggering of the large language model strategy analysis task and thus preventing blockage of the main push process. Simultaneously, the push process of the target tenant is controlled by the target push rate and task threshold. When the current number of tasks reaches the task threshold, the messages to be pushed are stored in a waiting queue. The push process is then resumed based on the push channel status, the waiting queue length, and the system resource status. This allows the message push process to adapt to changes in system resources and push channel status, reducing push task overload and secondary congestion caused by concentrated release of the waiting queue, thereby improving the stability and continuity of multi-tenant message push. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram illustrating an application scenario of the message push method based on a large language model provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the message push method based on a large language model provided in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the process of determining the target flow control strategy provided in an embodiment of the present invention; Figure 4 A schematic diagram of the structure of the message push device based on a large language model provided in this embodiment of the invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] It is understood that in specific embodiments of the present invention, data involving user information and related data requires user permission or consent, and the collection, use and processing of such data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0015] See also Figure 1 This diagram illustrates an application scenario for a message push method based on a large language model. The application scenario can include a terminal 101 and a server 102, which can exchange data via a network. The terminal 101 can be a mobile phone, tablet, smart Bluetooth device, computer, large screen, robot, etc.; the server 102 can be a single server or a server cluster consisting of multiple servers.

[0016] Users can send push requests to server 102 via terminal 101. The server considers the tenant initiating the push request as the target tenant and searches for the corresponding optimized flow control policy in a specified cache. The optimized flow control policy is determined through a large language model policy analysis task. Based on the search results of the optimized flow control policy and the existence of a check identifier, the target flow control policy corresponding to the target tenant is determined. The check identifier is generated according to a preset period and is cleared when the large language model policy analysis task is triggered. The target flow control policy includes a target push rate and a task threshold. If the current number of tasks reaches the task threshold, the messages to be pushed by the target tenant are stored in a waiting queue. The large language model is used to analyze the push channel status, the length of the waiting queue, and the system resource status to push the messages to be pushed in the waiting queue. If the current number of tasks does not reach the task threshold, the messages to be pushed can be directly processed according to the target push rate.

[0017] After the message is pushed, server 102 can send a notification to terminal 101.

[0018] In this embodiment, a message push method based on a large language model is provided, such as... Figure 2 As shown, the specific process of this message push method based on a large language model can be as follows: S110. In response to the push request initiated by the target tenant, search for the optimized flow control policy corresponding to the target tenant in the specified cache.

[0019] A tenant refers to an independent entity that accesses the system. The system isolates resources based on the tenant identifier, allocates independent message queues and dedicated delivery thread pools to each tenant, and isolates data and flow control resources between different tenants. Each tenant can independently configure flow control policies.

[0020] The tenant that initiates the push request is referred to as the target tenant. After receiving the push request, the system can look up the target tenant's optimized flow control strategy in the specified cache. The optimized flow control strategy is determined through the large language model strategy analysis task.

[0021] It should be noted that each tenant has its default rate control policy, which is usually related to the tenant's tier. The default rate control policy for each tenant tier can be preset. Due to system resource limitations, the default rate control policy may not be suitable for all situations. A large language model can be used to analyze system resources to determine the rate control policy that is most suitable for the tenant under the existing resources; this is called optimizing the rate control policy.

[0022] The generated optimized flow control strategy can be written to a designated cache. When a push request is received, the optimized flow control strategy can be retrieved from the designated cache. The specific method for generating the optimized flow control strategy will be described in detail in subsequent embodiments.

[0023] The flow control strategy can include the following core configuration items: number of threads, task threshold, push rate, and priority coefficient. The number of threads represents the core number of threads in the tenant's thread pool; the task threshold represents the maximum number of tasks that the thread pool can accommodate; the push rate represents the upper limit of the number of messages that the tenant can push per second; and the priority coefficient represents the relative priority of the tenant when multiple tenants compete for resources.

[0024] S120. Based on the search results of the optimized flow control strategy and the existence of the check identifier, determine the target flow control strategy corresponding to the target tenant.

[0025] The check flag is used to control the frequency of generating optimized flow control strategies. This check flag can be automatically generated according to a preset period. The preset period can be set according to actual needs; for example, if the preset period is 10 minutes, a check flag will be generated every 10 minutes. The check flag can be cleared when the large language model strategy analysis task is triggered. By determining the existence of the check flag, the update time of the optimized flow control strategy can be determined. For example, if both the check flag and the optimized flow control strategy exist, it indicates that the optimized flow control strategy was generated at least one preset period ago.

[0026] Generating optimized flow control strategies consumes model resources. To conserve resources, a large language model strategy analysis task can be performed under certain conditions. When searching for optimized flow control strategies in a specified cache, two results may occur: found or not found. Based on the search results and the existence of identifiers, the target flow control strategy that the target tenant should currently use can be determined. (See also...) Figure 3 The diagram illustrates the process of determining the target flow control strategy.

[0027] As one implementation method, when determining the target flow control policy, if the optimized flow control policy corresponding to the target tenant is not found in the specified cache, the target flow control policy is determined based on the tenant level corresponding to the target tenant; if the optimized flow control policy corresponding to the target tenant is found in the specified cache, the target flow control policy is determined based on the existence of the check identifier.

[0028] If no optimized flow control policy corresponding to the target tenant is found in the specified cache, it indicates that there is currently no optimized flow control policy available for the target tenant. To avoid congestion, the target flow control policy can be determined directly based on the tenant's tenant level. Optionally, multiple preset levels can be pre-set, and a corresponding preset flow control policy can be set for each preset level to form a default flow control policy. The preset levels can be divided according to the tenant's payment amount; the higher the payment amount, the higher the tenant level. For each preset level, a preset flow control policy can be preset. For example, in this embodiment of the invention, the tenant level can include 5 levels from 1 to 5, with larger numbers indicating higher levels. For example, the default flow control policy could be: { "level5": {"coreThreads": 20, "taskThreshold": 500, "pushRate": 500, "priorityCoefficient": 1.0}, "level4": {"coreThreads": 15, "taskThreshold": 300, "pushRate": 300, "priorityCoefficient": 0.8}, "level3": {"coreThreads": 10, "taskThreshold": 200, "pushRate": 200, "priorityCoefficient": 0.6}, "level2": {"coreThreads": 5, "taskThreshold": 100, "pushRate": 100, "priorityCoefficient": 0.4}, "level1": {"coreThreads": 2, "taskThreshold": 50, "pushRate": 50, "priorityCoefficient": 0.2} } If no optimized flow control policy is found for the target tenant, the tenant level of the target tenant can be obtained directly. This tenant level is one of the preset levels. The preset flow control policy corresponding to the tenant level is found and determined as the target flow control policy for the target tenant.

[0029] If the optimized flow control policy corresponding to the target tenant is found in the specified cache, the timeliness of the optimized flow control policy can be further determined based on the existence of the check identifier, so as to determine whether the optimized flow control policy can be directly used as the target flow control policy.

[0030] Optionally, when determining the target flow control strategy based on the existence of the check identifier, the check identifier of the target tenant can be queried; if the check identifier is not found, the optimized flow control strategy is determined as the target flow control strategy; if the check identifier is found, the check identifier is cleared, and the large language model strategy analysis task is asynchronously triggered to determine a new optimized flow control strategy; if the large language model strategy analysis task is successfully triggered, the optimized flow control strategy is determined as the target flow control strategy.

[0031] The check flag can be generated according to a preset cycle and stored in key-value pairs, where the key is check:flag:{tenantId} and the value is the current cycle ID. The tenantId is the tenant identifier, which can be used to uniquely identify a specific tenant, and the cycle ID can be used to uniquely identify a specific preset cycle.

[0032] When querying the check identifier of the target tenant, the tenant identifier corresponding to the target tenant can be used for querying. If no check identifier is found, it indicates that a large language model strategy analysis task has been triggered within the current preset period, and the check identifier was cleared when the task was triggered, and the next preset period for regenerating the check identifier has not yet arrived. Therefore, it can be determined that the optimized flow control strategy stored in the specified cache is the most recently generated optimized flow control strategy within the current preset period, and thus the optimized flow control strategy can be directly determined as the target flow control strategy.

[0033] If the check identifier is found, it indicates that the check identifier has been generated within the current latest preset period, and the large language model strategy analysis task corresponding to the current preset period has not yet been triggered, meaning the latest optimized flow control strategy is not yet stored in the specified cache. In this case, the check identifier can be cleared, and the large language model strategy analysis task can be triggered to generate the latest optimized flow control strategy. To avoid the large language model strategy analysis process blocking the current push process, the large language model strategy analysis task can be triggered asynchronously.

[0034] After confirming the successful triggering of the large language model strategy analysis task, the optimized flow control strategy obtained from the designated cache can be determined as the target flow control strategy, and the current push request can be executed based on the target flow control strategy. Simultaneously, after the large language model strategy analysis task completes execution and generates the latest optimized flow control strategy, the latest optimized flow control strategy is updated to the designated cache for subsequent push requests to query and use. It should be noted that here it is only necessary to ensure the successful triggering of the large language model strategy analysis task, without waiting for its successful execution.

[0035] When an inspection flag is found, the inspection flag is cleared and the large language model policy analysis task is triggered asynchronously. The inspection flag can be used to control the number of times the large language model policy analysis task is triggered, avoiding repeated triggering of the policy analysis task for the same tenant in a short period of time, thereby reducing the number of calls to the large language model and the consumption of computing resources.

[0036] Meanwhile, since the large language model strategy analysis task is executed asynchronously, the current push request will not be blocked during the generation of the latest optimized flow control strategy. The system can continue to determine the optimized flow control strategy already stored in the specified cache as the target flow control strategy for the current push request, so as to ensure that the current push process can be executed in a timely and continuous manner.

[0037] Specifically, the optimized flow control strategy is generated through a large language model strategy analysis task. This task involves obtaining flow control impact parameters across multiple dimensions, including the target tenant dimension and system resource dimension; analyzing the flow control objective and the safe range of flow control parameters under specified constraints using a large language model, where the specified constraints are related to the flow control impact parameters across the multiple dimensions; generating multiple candidate flow control strategies that match the flow control objective under the constraints of the safe range; and determining the optimized flow control strategy based on the degree of change between the candidate flow control strategies and the flow control strategies currently being used by the target tenant.

[0038] The system can periodically collect flow control impact parameters from multiple dimensions, including target tenant dimensions and system resource dimensions. Target tenant dimensions may include tenant level, active time period, and waiting queue length, while system resource dimensions may include CPU utilization, memory utilization, network bandwidth utilization, and push channel status.

[0039] The active time period can be the distribution of active time based on the analysis of the target tenant's historical push records. Specifically, it can include the daily active time period distribution, weekly active patterns, and push time period patterns for different types of messages. Specifically, the daily active time period distribution can include the activity probability of each hour within a day, and the weekly active patterns can include the difference in activity between weekdays and weekends. The system can combine different hours, weekdays / rest days, message types, and historical response data to divide the time period into active, normal, or inactive periods.

[0040] The push channel status can include the following parameters: consumption rate, rate limiting threshold, queue depth, and channel status index. The consumption rate refers to the number of messages consumed per second; the rate limiting threshold refers to the maximum processing capacity per second; the queue depth refers to the total number of messages waiting to be consumed; the channel status index characterizes the congestion level of the push channel, with a value ranging from 0 to 100. This index is determined comprehensively based on the channel's queue depth, current consumption rate, rate limiting threshold, and the consumption rate trend. A higher channel status index indicates a more congested push channel.

[0041] The obtained flow control impact parameters are input into the large language model for analysis and reasoning, so that the large language model can summarize the specified constraints from the flow control impact parameters, and output the flow control target and the safe range of the flow control parameters based on the specified constraints. The flow control parameters are the parameters in the optimized flow control strategy.

[0042] The flow control objective refers to the goal that the optimized flow control strategy generated this time aims to achieve. For example, if the current flow control impact parameters show that memory usage is too high and there is a high risk of system crash, the flow control objective could be to reduce memory usage and maintain normal system operation. Another example is if the current flow control impact parameters show that messages are piling up in the waiting queue of a high-level tenant; in this case, the flow control objective could be to push the messages in the tenant's waiting queue out as quickly as possible.

[0043] The safe range of flow control parameters ensures that the parameters in the flow control strategy will not exceed the boundaries and cause security issues. For example, when CPU utilization is high, the push rate needs to be reduced, but if users are active at this time, the push rate needs to be increased. If the push rate is increased at this time, although it meets the needs of users during active periods, it will cause CPU utilization to continue to rise, which may lead to a crash. Therefore, large language models need to determine the safe range of push rate by comprehensively considering various flow control parameters.

[0044] Within the constraints of a security range, the large language model can generate multiple candidate flow control strategies that match the flow control objective. All candidate flow control strategies can achieve the flow control objective, differing only in their specific flow control parameters. These candidate flow control strategies are then compared with the flow control strategy currently being used by the target tenant to determine the degree of policy variation, and the candidate flow control strategy with the smallest degree of variation is selected as the optimized flow control strategy.

[0045] Optionally, when generating an optimized flow control strategy through a large language model, a prompt word template can be pre-set. This prompt word template may include at least one of the following: strategy generation process, analysis requirements, generation requirements, and output format requirements. By filling the prompt word template with the obtained flow control influence parameters from multiple dimensions, the target prompt information can be obtained. The target prompt information is then input into the large language model to guide the large language model in generating the optimized flow control strategy.

[0046] The above approach allows us to first determine the flow control objectives and safe ranges for flow control parameters in the current operational scenario based on multiple dimensions of flow control impact parameters. Then, within these safe ranges, we generate multiple candidate flow control strategies that match the flow control objectives. This enables us to select the strategy with the least change from the flow control strategy currently being used by the target tenant as the optimized flow control strategy from among the available options that meet the flow control objectives. This approach not only adapts the optimized flow control strategy to the current operational scenario but also reduces push fluctuations caused by excessive adjustments to flow control parameters, thereby improving the stability of the message push flow control process.

[0047] Since the optimized flow control strategy is generated by the large language model based on multi-dimensional operating states such as target tenant dimension, system resource dimension, push channel dimension and waiting queue dimension, the ability of the large language model can be fully utilized to analyze the flow control parameters such as target push rate and task threshold that are adapted to the current operating state, avoiding the inaccurate push control caused by long-term reliance on fixed flow control configuration.

[0048] S130. If the current number of tasks of the target tenant reaches the task threshold, the message to be pushed to the target tenant is stored in the waiting queue.

[0049] The target flow control strategy can include a task threshold, which represents the maximum number of tasks that the thread pool can accommodate. It can obtain the current number of tasks for the target tenant and determine whether the current number of tasks has reached the task threshold. If the current number of tasks reaches the task threshold, circuit breaking can be triggered; that is, the target tenant's pending push messages can be stored in a waiting queue.

[0050] S140. Analyze the push channel status, waiting queue length, and system resource status using a large language model to push the messages to be pushed in the waiting queue.

[0051] Messages in the waiting queue still need to be pushed at the appropriate time. A large language model can be used to analyze the push channel, the length of the waiting queue, and the status of system resources to determine whether it is necessary to push the messages waiting in the waiting queue.

[0052] The system can be configured with a timer module to retrieve the push channel status, waiting queue length, and system resource status from the cache. The system resource status refers to the usage of CPU, memory, and network bandwidth; the push channel status can be found in the explanation of the corresponding embodiments described above; and the waiting queue length refers to the length of the waiting queue for each tenant within the system.

[0053] By analyzing these data points, we can avoid blindly releasing messages waiting in the queue when the push channel has not yet recovered or system resources are still strained. This reduces the risk of the push channel becoming congested again or system resources becoming overloaded again due to queue release. Specifically, we can determine whether the queue wake-up conditions are met based on the push channel status, the length of the waiting queues for all tenants, and the system resource status. If the queue wake-up conditions are met, a wake-up strategy is determined based on the push channel status, the length of the waiting queues for all tenants, and system resources. Messages in the waiting queue are then pushed according to the wake-up strategy.

[0054] The queue wake-up conditions can be set according to actual needs. In this embodiment of the invention, the queue wake-up conditions can be determined when the push channel status is better, or when the clearing rate of the waiting queue of the specified tenant is high, or when the system resources are sufficient.

[0055] Among them, the push channel status is better, which means that the channel status index drops from greater than the first value to less than the second value and shows a downward trend. The first value is greater than the second value. The specific value can be set according to actual needs. For example, the first value is 90 and the second value is 70.

[0056] The designated tenant is a tenant with a higher tenant level. As described in the previous embodiments, the tenant level can include five levels from 1 to 5, with higher numbers indicating higher levels. The designated tenant can refer to a tenant with a tenant level of 5. A higher clearing rate for the waiting queue of a designated tenant can mean that the length of the waiting queue for all designated tenants is less than a preset value. The preset value can be set according to actual needs and is not specifically limited here.

[0057] Sufficient system resources mean that the utilization rates of CPU and memory are both less than the specified thresholds and remain below the specified duration. Both the specified thresholds and the specified durations can be set according to actual needs. For example, the specified threshold can be set to 60% and the specified duration to 5 minutes.

[0058] Meeting any one of the above three conditions is considered sufficient to satisfy the queue wake-up criteria. If the queue wake-up criteria are satisfied, a wake-up strategy can be further determined. The wake-up strategy can include the scope of tenants to be woken up, the number of tenants to be woken up, and the wake-up rate. The scope of tenants to be woken up specifies which tenants' waiting queues to be woken up. The number of tenants to be woken up refers to the number of messages to be pushed during this wake-up. The wake-up rate is the upper limit on the number of messages retrieved from the waiting queue and attempted to be pushed per second. By controlling the message recovery and push process in the waiting queue through the three aspects of the scope of tenants to be woken up, the number of tenants to be woken up, and the wake-up rate, the messages in the waiting queue can be released in a tiered, limited, and rate-limited manner according to the current system capacity, reducing the risk of concentrated release causing congestion in the push channel or renewed strain on system resources.

[0059] Optionally, when determining the wake-up strategy based on the push channel status, the length of the waiting queues of all tenants, and the system resource status, the wake-up capability index can be calculated based on the push channel status, the length of the waiting queues of all tenants, and the system resource status; the preset tenant level corresponding to the wake-up capability index can be determined as the wake-up tenant range; the number of wake-ups can be determined using the wake-up capability index, the system baseline capacity, and historical wake-up data; and the wake-up rate can be determined based on the tenant level corresponding to the tenant to which the waiting queue belongs. The wake-up tenant range, the number of wake-ups, and the wake-up rate can then be combined to obtain the wake-up strategy.

[0060] The push channel status, the length of all tenants' waiting queues, and the system resource status can be input into a large language model, which then comprehensively analyzes and calculates a wake-up capability index. A higher wake-up capability index indicates stronger message push capabilities.

[0061] The range of tenants to be woken up is determined based on the index range in which the wake-up capability index falls. A mapping relationship between the index range and tenant levels can be pre-set, with the tenant level corresponding to the index range in which the wake-up capability index falls being used as the range of tenants to be woken up.

[0062] For example, if the tenant level corresponding to the wake-up capability index is 5 or 4, then all tenants with tenant levels of 5 or 4 are within the wake-up tenant range.

[0063] After determining the scope of wake-up tenants, the wake-up count and wake-up rate can be calculated to ensure that messages in the waiting queue can be pushed smoothly and to avoid channel congestion. The system baseline capacity refers to the number of messages the system can safely release within one wake-up cycle. This number can be obtained by comprehensively analyzing the system's historical push capabilities and push channel rate limiting capabilities using a neural network model.

[0064] When determining the number of wake-ups, the system baseline capacity can be scaled based on the wake-up capability index to obtain the basic number of wake-ups; the basic number of wake-ups can be corrected based on historical wake-up data to obtain a first number; the smaller of the first number and the second number can be determined as the number of wake-ups, where the second number is the total number of messages in the waiting queues of all tenants within the wake-up tenant range.

[0065] The higher the wake-up capability index, the more suitable the current push channel and system resources are for releasing the waiting queue, and therefore the larger the basic wake-up count; the lower the wake-up capability index, the weaker the current release capability, and therefore the smaller the basic wake-up count.

[0066] Furthermore, the initial number of wake-ups can be obtained by adjusting the base number of wake-ups based on historical wake-up data. Specifically, if the success rate of push notifications is high after historical wake-ups, and the push channel status and system resource status remain stable, the historical correction coefficient can be set to a value greater than 1 to appropriately increase the number of wake-ups this time. If the push channel status index, system resource utilization rate, or push failure rate increases after historical wake-ups, the historical correction coefficient can be set to a value less than 1 to reduce the number of wake-ups this time. If no usable historical wake-up data exists, the historical correction coefficient can be set to 1.

[0067] In this way, the number of wake-ups increases with the increase of the wake-up capability index, and is jointly constrained by the system baseline capacity, historical wake-up effect and the current actual number of waiting messages, thereby avoiding the release of too many waiting messages at once when the push channel or system resources have just been restored.

[0068] Specifically, the formula for calculating the number of wake-ups can be: N = min{L, floor(B × W / 100 × H)}; Where N represents the number of wake-ups; L represents the total number of messages in the waiting queues of all tenants within the wake-up tenant range; B represents the system baseline capacity; W represents the wake-up capability index; H represents the historical correction coefficient; and floor represents rounding down.

[0069] For wake-up scenarios, a preset rate can be pre-set for each tenant level. The wait queue then finds the preset rate corresponding to the tenant level and determines it as the wake-up rate.

[0070] By combining the scope of tenants to be woken up, the number of tenants to be woken up, and the wake-up rate, a wake-up strategy can be obtained, and messages in the waiting queue can be pushed and processed according to the wake-up strategy.

[0071] During the wake-up process, the status of the push channel and system resources can be continuously monitored. If the push channel status is detected to be congested again, or the system resources are detected to be under high load again, the current wake-up round is paused, and the unreleased messages to be pushed are kept in the waiting queue. If the push channel status and system resource status remain stable for several consecutive wake-up cycles, the scope of wake-up tenants can be gradually expanded, the number of wake-ups can be increased, or the wake-up rate can be increased in subsequent wake-up cycles to achieve a gradual recovery of the push of messages to be pushed in the waiting queue.

[0072] Specifically, when pushing messages in the waiting queue according to the wake-up strategy, the process may involve obtaining the waiting queues of tenants within the wake-up tenant range; retrieving a number of wake-up messages from the waiting queue and storing them in the release queue according to the tenant level and tenant activity status; and pushing messages in the release queue according to the wake-up rate.

[0073] Based on the wake-up tenant range in the wake-up policy, the waiting queue of each tenant within the wake-up tenant range can be obtained; the waiting queues are sorted according to the tenant level and the tenant activity status so that a number of wake-up messages can be retrieved from multiple waiting queues and stored in the release queue.

[0074] Optionally, a tenant's activity status can include both active and inactive states, which can be determined based on the current time and the tenant's active period. For example, check if the current time falls within the tenant's active period; if so, determine that the tenant is active; if not, determine that the tenant is inactive.

[0075] Furthermore, the active status can be divided into "active" and "about to become inactive," and the inactive status can be divided into "long-term inactive" and "about to become active." If the current time is within the tenant's active period, the duration between the current time and the end time of the active period can be calculated; if the duration is greater than the first duration, it is determined to be "active"; otherwise, it is determined to be "about to become inactive."

[0076] Similarly, if the current time is not within the tenant's active period, calculate the duration between the current time and the start time of the most recent active period; if the duration is greater than the second duration, determine that the tenant is inactive for a long time; if the duration is less than the second duration, determine that the tenant is about to become inactive.

[0077] Both the first and second durations can be set according to actual needs, and no specific limitations are made here.

[0078] A score can be pre-set for each active status. The activity score of the waiting queue is determined based on the identified active status; it's understood that the activity score for the same tenant's waiting queue is the same. Tenant levels can be converted into level scores according to a preset mapping relationship, which may include the level score corresponding to each level. The level scores are weighted using level weights to obtain a weighted level score; the activity scores are weighted using activity weights to obtain a weighted activity score; the sum of the weighted level scores and the weighted activity scores yields the target score. The waiting queues are then sorted in descending order of the target score to obtain the queue sequence.

[0079] According to the order of the waiting queue in the queue sequence, a certain number of wake-up messages are taken out from them in turn and stored in the release queue. The messages in the release queue are pushed according to the wake-up rate.

[0080] S150. If the current number of tasks of the target tenant has not reached the task threshold, the message to be pushed is pushed according to the target push rate.

[0081] If the current number of tasks for the target tenant has not reached the task threshold, push processing can be performed directly according to the target flow control policy. Specifically, messages to be pushed can be pulled from the message queue according to the target push rate in the target flow control policy, submitted to the thread pool for push execution, and then an acknowledgment code can be sent back to the message queue.

[0082] The solution provided in this invention can be applied to various scenarios requiring message distribution and push control. For example, it can be applied to robot cloud management platforms, robot cluster scheduling platforms, and edge-cloud collaborative control platforms in embodied intelligence scenarios. In these scenarios, the cloud platform may need to push task instructions, control commands, path update information, status synchronization messages, and anomaly alarm messages to devices corresponding to different customers, venues, or robot clusters. Different customers or robot clusters can act as different tenants, and the number of devices, task frequency, service level, and communication resource requirements of different tenants may vary.

[0083] Taking a robot cluster scheduling scenario as an example, when the robot cluster corresponding to a target tenant initiates or triggers a task message push request, the optimized flow control strategy corresponding to the target tenant can be searched in the specified cache, and the existence of the check flag can be used to determine the currently executable target flow control strategy. If the current number of push tasks for the target tenant reaches the task threshold, the task instructions or status messages to be pushed can be stored in the waiting queue; if the current number of push tasks does not reach the task threshold, the corresponding messages can be pushed to robots, edge nodes, or operation and maintenance terminals according to the target push rate. For messages in the waiting queue, the push channel status, waiting queue length, and system resource status can be analyzed, and the push of messages in the waiting queue can be resumed based on the analysis results.

[0084] The solution provided by the embodiments of the present invention can dynamically control the push process of task instructions, control commands and status messages based on the push task status, communication channel status and platform system resource status of the corresponding tenant of the robot cluster in the embodied intelligence scenario. This reduces message backlog, channel congestion or increased system load caused by the centralized distribution of push tasks, thereby improving the stability and continuity of message push of the robot cloud management platform in multi-tenant and multi-device concurrent scenarios.

[0085] As can be seen from the above, the embodiments of the present invention can leverage the comprehensive analysis capabilities of a large language model to comprehensively analyze multiple dimensions of flow control impact parameters, thereby generating an optimized flow control strategy that matches the current system state. Compared to rule-based judgment, this approach is more flexible, has stronger applicability, and lower maintenance costs. The large language model strategy analysis task checks the trigger frequency by checking the identifier and executes asynchronously, which avoids frequent task triggering to save computing power, and asynchronous execution also avoids blocking the push process. When the number of current tasks for the target tenant reaches the task threshold, the messages to be pushed are stored in a waiting queue, and a wake-up strategy is determined based on the push channel status, waiting queue length, and system resource status to perform graded, limited, and rate-limited pushes on the messages in the waiting queue. This reduces the problems of insufficient adaptation to fixed strategies, push task overload, and secondary congestion of the channel caused by concentrated release of the waiting queue, improving the stability and continuity of multi-tenant message push.

[0086] To better implement the above methods, this embodiment of the invention also provides a message push device based on a large language model. This device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, robot, or other similar device. The server can be a single server or a server cluster composed of multiple servers.

[0087] For example, in this embodiment, the method of the present invention will be described in detail by taking the message push device based on a large language model specifically integrated into the server as an example.

[0088] For example, such as Figure 4 As shown, the message push device 200 based on a large language model may include a search module 210, a target determination module 220, a waiting module 230, a wake-up module 240, and a push module 250.

[0089] The lookup module 210 is used to respond to a push request initiated by a target tenant and look up the optimization flow control strategy corresponding to the target tenant in a specified cache. The optimization flow control strategy is determined through a large language model strategy analysis task. The target determination module 220 is used to determine the target flow control strategy corresponding to the target tenant based on the search results of the optimized flow control strategy and the existence of the check identifier. The check identifier is generated according to a preset period and is cleared when the large language model strategy analysis task is triggered. The target flow control strategy includes the target push rate and the task threshold. The waiting module 230 is used to store the target tenant's pending push messages into a waiting queue if the current number of tasks of the target tenant reaches the task threshold. The wake-up module 240 is used to analyze the status of the push channel, the length of the waiting queue, and the status of system resources through a large language model, so as to push the messages to be pushed in the waiting queue. The push module 250 is used to push the message to be pushed according to the target push rate if the current number of tasks of the target tenant has not reached the task threshold.

[0090] In some embodiments, the target determination module 220 is specifically used for: If the optimized flow control policy corresponding to the target tenant is not found in the specified cache, the target flow control policy is determined based on the tenant level corresponding to the target tenant. If the optimized flow control policy corresponding to the target tenant is found in the specified cache, the target flow control policy is determined based on the existence of the check identifier.

[0091] In some embodiments, the target determination module 220 is specifically used for: Query the inspection identifier of the target tenant; If the inspection identifier is not found, the optimized flow control strategy will be determined as the target flow control strategy. If the check flag is found, clear the check flag and asynchronously trigger the large language model strategy analysis task to determine a new optimized flow control strategy; If the large language model strategy analysis task is successfully triggered, the optimized flow control strategy will be determined as the target flow control strategy.

[0092] In some embodiments, the message push device based on a large language model further includes an analysis module, which is used to generate an optimized flow control strategy, specifically for: Obtain flow control impact parameters from multiple dimensions, including target tenant dimension and system resource dimension; Using a large language model, the flow control objectives and the safe range of flow control parameters under specified constraints are analyzed. These specified constraints are related to the flow control influence parameters of the multiple dimensions. Under the constraints of the stated security range, multiple candidate flow control strategies that match the flow control objective are generated; Based on the degree of change between the candidate flow control strategies and the flow control strategies currently being used by the target tenant, the optimal flow control strategy is determined.

[0093] In some embodiments, the wake-up module 240 is specifically used for: Based on the push channel status, the length of all tenants' waiting queues, and the system resource status, determine whether the queue wake-up conditions are met. If the same conditions for queue wake-up are met, the wake-up strategy is determined based on the push channel status, the length of the waiting queues of all tenants, and the system resource status. The messages in the waiting queue are pushed according to the wake-up strategy.

[0094] In some embodiments, the wake-up module 240 is specifically used for: The wake-up capability index is calculated using the push channel status, the length of the waiting queue for all tenants, and the system resource status. The preset tenant level corresponding to the wake-up capability index is determined as the wake-up tenant range; The number of wake-ups is determined using the wake-up capability index, the system baseline capacity, and historical wake-up data. The wake-up rate is determined based on the tenant level corresponding to the tenant to which the waiting queue belongs; By combining the range of tenants to be woken up, the number of tenants to be woken up, and the wake-up rate, a wake-up strategy is obtained.

[0095] In some embodiments, the wake-up module 240 is specifically used for: The basic number of wake-ups is obtained by scaling the system baseline capacity based on the wake-up capability index. The base wake-up count is corrected based on historical wake-up data to obtain a first count; The smaller of the first quantity and the second quantity is determined as the wake-up quantity, where the second quantity is the total number of messages in the waiting queues of all tenants within the wake-up tenant range.

[0096] In some embodiments, the wake-up module 240 is specifically used for: Obtain the waiting queue of tenants within the scope of the awakened tenant; Based on the tenant level and tenant activity status, retrieve the required number of wake-up messages from the waiting queue and store them in the release queue; The messages in the release queue are pushed according to the wake-up rate.

[0097] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0098] As can be seen from the above, the message push device based on the large language model in this embodiment can retrieve the optimized flow control strategy determined by the large language model strategy analysis task from the specified cache when the target tenant initiates a push request, and determine the currently executable target flow control strategy by checking the existence of the check flag, thus avoiding frequent triggering of the large language model strategy analysis task and blocking the main push process. At the same time, the push process of the target tenant is controlled by the target push rate and task threshold. When the current number of tasks reaches the task threshold, the messages to be pushed are stored in the waiting queue, and the messages to be pushed in the waiting queue are restored for push based on the push channel status, the waiting queue length, and the system resource status. Therefore, the message push process can adapt to changes in system resources and push channel status, reduce push task overload and secondary congestion of the channel caused by concentrated release of the waiting queue, and improve the stability and continuity of multi-tenant message push.

[0099] This invention also provides an electronic device, which can be a terminal, a server, or other similar device. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, robot, etc.; the server can be a single server or a server cluster composed of multiple servers, and so on.

[0100] In some embodiments, the message push device based on a large language model can also be integrated into multiple electronic devices. For example, the message push device based on a large language model can be integrated into multiple servers, and multiple servers can implement the message push method based on a large language model of the present invention.

[0101] In this embodiment, a server will be used as an example for detailed description. For example, ... Figure 5As shown, it illustrates a structural schematic diagram of the electronic device involved in an embodiment of the present invention, specifically: The electronic device may include components such as a processor 310 with one or more processing cores, a memory 320 with one or more computer-readable storage media, a power supply 330, an input module 340, and a communication module 350. Those skilled in the art will understand that... Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 310 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes various functions and processes data by running or executing software programs and / or modules stored in the memory 320, and by calling data stored in the memory 320. In some embodiments, the processor 310 may include one or more processing cores; in some embodiments, the processor 310 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 310.

[0102] The memory 320 can be used to store software programs and modules. The processor 310 executes various functional applications and data processing by running the software programs and modules stored in the memory 320. The memory 320 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 320 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 320 may also include a memory controller to provide the processor 310 with access to the memory 320.

[0103] The electronic device also includes a power supply 330 that supplies power to the various components. In some embodiments, the power supply 330 can be logically connected to the processor 310 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 330 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0104] The electronic device may also include an input module 340, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0105] The electronic device may also include a communication module 350. In some embodiments, the communication module 350 may include a wireless module, through which the electronic device can perform short-range wireless transmission, thereby providing users with wireless broadband internet access. For example, the communication module 350 can be used to help users send and receive, browse web pages, and access streaming media.

[0106] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 310 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 320 according to the following instructions, and the processor 310 runs the applications stored in the memory 320, thereby implementing the steps in the methods of the various embodiments of the present invention.

[0107] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0108] As can be seen from the above, the electronic device provided in this embodiment of the invention can retrieve the optimized flow control strategy determined by the large language model strategy analysis task from the specified cache when the target tenant initiates a push request, and determine the currently executable target flow control strategy by checking the existence of the check flag, thus avoiding frequent triggering of the large language model strategy analysis task and blocking the main push process. Simultaneously, the push process of the target tenant is controlled by the target push rate and task threshold. When the current number of tasks reaches the task threshold, the messages to be pushed are stored in the waiting queue, and the messages to be pushed in the waiting queue are restored by combining the push channel status, the waiting queue length, and the system resource status. Therefore, the message push process can adapt to changes in system resources and push channel status, reduce push task overload and secondary congestion of the channel caused by concentrated release of the waiting queue, and improve the stability and continuity of multi-tenant message push.

[0109] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0110] To this end, embodiments of the present invention provide a computer-readable storage medium storing a plurality of instructions which can be loaded by a processor to execute steps in any of the message push methods based on a large language model provided in embodiments of the present invention.

[0111] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0112] According to one aspect of the present invention, a computer program product or computer program is provided, comprising a computer program / instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instructions from the computer-readable storage medium and executes the computer program / instructions, causing the electronic device to perform the methods provided in the various optional implementations of the above embodiments.

[0113] Since the instructions stored in the storage medium can execute the steps in any of the message push methods based on a large language model provided in the embodiments of the present invention, the beneficial effects that any of the message push methods based on a large language model provided in the embodiments of the present invention can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0114] The foregoing has provided a detailed description of a message push method, apparatus, and electronic device based on a large language model provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A message push method based on a large language model, characterized in that, The method includes: In response to a push request initiated by a target tenant, the optimized flow control strategy corresponding to the target tenant is searched in a specified cache. The optimized flow control strategy is determined through a large language model strategy analysis task. Based on the search results of the optimized flow control strategy and the existence of the check identifier, the target flow control strategy corresponding to the target tenant is determined. The check identifier is generated according to a preset period and is cleared when the large language model strategy analysis task is triggered. The target flow control strategy includes the target push rate and the task threshold. If the current number of tasks of the target tenant reaches the task threshold, the messages to be pushed to the target tenant are stored in the waiting queue. The push channel status, waiting queue length, and system resource status are analyzed using a large language model to push the messages to be pushed in the waiting queue. If the current number of tasks for the target tenant has not reached the task threshold, the message to be pushed shall be pushed according to the target push rate.

2. The method according to claim 1, characterized in that, The step of determining the target flow control policy corresponding to the target tenant based on the search results of the optimized flow control policy and the existence of the check identifier includes: If the optimized flow control policy corresponding to the target tenant is not found in the specified cache, the target flow control policy is determined based on the tenant level corresponding to the target tenant. If the optimized flow control policy corresponding to the target tenant is found in the specified cache, the target flow control policy is determined based on the existence of the check identifier.

3. The method according to claim 2, characterized in that, The step of determining the target flow control strategy based on the existence of the check identifier includes: Query the inspection identifier of the target tenant; If the inspection identifier is not found, the optimized flow control strategy will be determined as the target flow control strategy. If the check flag is found, clear the check flag and asynchronously trigger the large language model strategy analysis task to determine a new optimized flow control strategy; If the large language model strategy analysis task is successfully triggered, the optimized flow control strategy will be determined as the target flow control strategy.

4. The method according to any one of claims 1-3, characterized in that, The optimized flow control strategy is generated in the following manner: Obtain flow control impact parameters from multiple dimensions, including target tenant dimension and system resource dimension; Using a large language model, the flow control objectives and the safe range of flow control parameters under specified constraints are analyzed. These specified constraints are related to the flow control influence parameters of the multiple dimensions. Under the constraints of the stated security range, multiple candidate flow control strategies that match the flow control objective are generated; Based on the degree of change between the candidate flow control strategies and the flow control strategies currently being used by the target tenant, the optimal flow control strategy is determined.

5. The method according to claim 1, characterized in that, The process of analyzing the push channel status, waiting queue length, and system resource status using a large language model to push messages in the waiting queue includes: Based on the push channel status, the length of all tenants' waiting queues, and the system resource status, determine whether the queue wake-up conditions are met. If the queue wake-up conditions are met, the wake-up strategy is determined based on the push channel status, the length of the waiting queues of all tenants, and the system resource status. The messages in the waiting queue are pushed according to the wake-up strategy.

6. The method according to claim 5, characterized in that, The step of determining the wake-up strategy based on the push channel status, the length of all tenants' waiting queues, and the system resource status includes: The wake-up capability index is calculated using the push channel status, the length of the waiting queue for all tenants, and the system resource status. The preset tenant level corresponding to the wake-up capability index is determined as the wake-up tenant range; The number of wake-ups is determined using the wake-up capability index, the system baseline capacity, and historical wake-up data. The wake-up rate is determined based on the tenant level corresponding to the tenant to which the waiting queue belongs; By combining the range of tenants to be woken up, the number of tenants to be woken up, and the wake-up rate, a wake-up strategy is obtained.

7. The method according to claim 6, characterized in that, The determination of the wake-up count based on the wake-up capability index, the system baseline capacity, and historical wake-up data includes: The basic number of wake-ups is obtained by scaling the system baseline capacity based on the wake-up capability index. The base wake-up count is corrected based on historical wake-up data to obtain a first count; The smaller of the first quantity and the second quantity is determined as the wake-up quantity, where the second quantity is the total number of messages in the waiting queues of all tenants within the wake-up tenant range.

8. The method according to claim 6, characterized in that, The step of pushing messages in the waiting queue according to the wake-up strategy includes: Obtain the waiting queue of tenants within the scope of the awakened tenant; Based on the tenant level and tenant activity status, retrieve the required number of wake-up messages from the waiting queue and store them in the release queue; The messages in the release queue are pushed according to the wake-up rate.

9. A message push device based on a large language model, used to implement the method of any one of claims 1-8, characterized in that, The device includes: The lookup module is used to respond to a push request initiated by a target tenant and look up the optimization flow control strategy corresponding to the target tenant in a specified cache. The optimization flow control strategy is determined through a large language model strategy analysis task. The target determination module is used to determine the target flow control strategy corresponding to the target tenant based on the search results of the optimized flow control strategy and the existence of the check identifier. The check identifier is generated according to a preset period and is cleared when the large language model strategy analysis task is triggered. The target flow control strategy includes the target push rate and the task threshold. The waiting module is used to store the target tenant's pending push messages into a waiting queue if the current number of tasks of the target tenant reaches the task threshold. The wake-up module is used to analyze the status of the push channel, the length of the waiting queue, and the status of system resources through a large language model, so as to push the messages to be pushed in the waiting queue. The push module is used to push messages to be pushed at the target push rate if the current number of tasks of the target tenant has not reached the task threshold.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to execute the steps in the message push method based on a large language model as described in any one of claims 1 to 8.

Images