A method and system for dynamically allocating AI server resources

By analyzing the tag words and resource consumption history in user conversations, establishing association relationships, and realizing dynamic scheduling of AI server resources, the problem of inflexible resource allocation in traditional methods is solved, and resource utilization and service quality are improved.

CN118260081BActive Publication Date: 2024-11-15HEBEI KAISHA CENTURY INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410397817.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-03
Publication Date
2024-11-15
Estimated Expiration
2044-04-03

AI Technical Summary

Technical Problem

Traditional AI server resource allocation methods are difficult to accurately predict the real-time demand of AI services, resulting in inflexible and accurate resource allocation, which often leads to oversupply or insufficient resources during certain periods.

Method used

By analyzing the marker words and resource consumption history in each conversation of the user, establishing resource volume correlation, dynamically schedule resource allocation to match user needs, and avoiding idle and insufficient resources.

Benefits of technology

It improves the response matching of server resources, effectively utilizes resources, avoids idle and insufficient resources, and improves the performance and efficiency of AI services.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118260081B_ABST
    Figure CN118260081B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for dynamically allocating resources of an AI server, and relates to the technical field of computing power scheduling. The present invention includes a scheduling execution unit and a scheduling execution unit, wherein the historical records of the tag words contained in the instructions and prompts of each user in each conversation and the resource amount of each resource item consumed in each conversation are calculated to obtain the association relationship between the tag words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item; according to the tag words contained in the instructions of the user in the current conversation and the continuously updated prompts, and the tag words contained in the instructions and prompts of the user in the conversation and the resource amount of each resource item, the resource amount is continuously scheduled and allocated for each resource item of the user in the current conversation to obtain the scheduling resource amount of each resource item of the user in the current conversation. The present invention effectively avoids the idleness and insufficiency of server resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computing power scheduling, and in particular, to a method and system for dynamically allocating resources of an AI server. Background Art

[0002] In the field of modern computing, the application of artificial intelligence (AI) has become widespread, and its demand for computing resources has also increased. As an important infrastructure for providing AI services, the resource allocation and management of AI servers directly affect the performance and cost of AI services. Due to the diversity and complexity of AI processing tasks, AI servers need to be able to dynamically adjust the allocation of computing resources according to the needs of different tasks to improve resource utilization and service quality.

[0003] Traditional resource allocation methods have difficulty accurately predicting the real-time resource demand of AI services, resulting in inflexible and inaccurate resource allocation. They are also unable to adapt to the dynamic changes in task loads, often resulting in excess resources in some periods and insufficient resources in other periods. Summary of the invention

[0004] The purpose of the present invention is to provide an AI server resource dynamic allocation method and system, which realizes the prediction of the current conversation and dynamic resource scheduling by analyzing the marked words of each user in each conversation and the amount of resources consumed, thereby improving the response matching degree of server resources and effectively avoiding idleness and shortage of server resources.

[0005] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0006] The present invention provides a method for dynamically allocating resources of an AI server, comprising:

[0007] Get the history of token words contained in commands and prompts of each user in each conversation;

[0008] Obtain a historical record of the amount of resources consumed by each user for each resource item in each of the conversations;

[0009] Calculate the historical records of the marker words contained in the instructions and prompts of each user in each conversation and the resource amount of each resource item consumed in each conversation to obtain the association relationship between the marker words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item;

[0010] Obtain the instructions of the user in the current conversation and continue to obtain the prompts;

[0011] Based on the instructions of the user in the current conversation, the marker words contained in the continuously updated prompts, and the association between the instructions of the user in the conversation and the marker words contained in the prompts and the resource quantities of each resource item, the resource quantities are continuously scheduled and allocated for each resource item of the user in the current conversation to obtain the scheduled resource quantities for each resource item of the user in the current conversation.

[0012] The present invention also discloses a method for dynamically allocating resources of an AI server, comprising:

[0013] The scheduling execution unit is used to receive the scheduling resource quantity of each resource item in the current session from the user;

[0014] Server resources are allocated to users according to the scheduled resource amount of each resource item in the current session.

[0015] The present invention also discloses an AI server resource dynamic allocation system, comprising:

[0016] Allocating a computing unit for obtaining a history of the tag words contained in the instructions and prompts of each user in each conversation;

[0017] Obtain a historical record of the amount of resources consumed by each user for each resource item in each of the conversations;

[0018] Calculate the historical records of the marker words contained in the instructions and prompts of each user in each conversation and the resource amount of each resource item consumed in each conversation to obtain the association relationship between the marker words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item;

[0019] Obtain the instructions of the user in the current conversation and continue to obtain the prompts;

[0020] According to the instructions of the user in the current conversation and the marking words contained in the continuously updated prompts, and the association relationship between the instructions of the user in the conversation and the marking words contained in the prompts and the resource amount of each resource item, continuously scheduling and allocating the resource amount for each resource item of the user in the current conversation to obtain the scheduled resource amount of each resource item of the user in the current conversation;

[0021] The scheduling execution unit is used to receive the scheduling resource quantity of each resource item in the current session from the user;

[0022] Server resources are allocated to users according to the scheduled resource amount of each resource item in the current session.

[0023] The present invention uses an allocation calculation unit to perform a tag word analysis on the instructions and prompts of each user in each conversation, and establishes an association relationship with the resource amount of each resource item. During the user's AI service conversation, the allocation calculation unit receives each instruction submitted by the user and analyzes it, and combines the instructions and prompts in the conversation with the tag words contained in the resource amount of each resource item to obtain the scheduling resource amount of each resource item in the current conversation, thereby providing each user with server resources that meet the needs in a timely and effective manner, thereby improving the effective utilization rate of server resources.

[0024] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0026] Figure 1 A schematic diagram of an AI server resource dynamic allocation system according to an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of a step flow of allocating a computing unit according to an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of a step flow of a scheduling execution unit according to an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of the step flow of step S3 in one embodiment of the present invention;

[0030] Figure 5 A schematic diagram of a step flow chart of step S34 in an embodiment of the present invention;

[0031] Figure 6 This is a schematic diagram of the step flow of step S5 in one embodiment of the present invention;

[0032] Figure 7 A schematic diagram of a step flow chart of step S52 in an embodiment of the present invention;

[0033] Figure 8 A schematic diagram of a step flow chart of step S54 in an embodiment of the present invention;

[0034] Fig. 9 A schematic diagram of a step flow chart of step S55 of the present invention in an embodiment;

[0035] Fig.10 This is a schematic diagram of a step flow chart of step S553 in an embodiment of the present invention;

[0036] In the accompanying drawings, the components represented by the reference numerals are listed as follows:

[0037] 1- Allocate computing units, 2- Schedule execution units. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0039] It should be noted that the terms "first", "second", etc. in this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0040] AI is the abbreviation of Artificial Intelligence. It is a branch of computer science that studies how to enable computers to simulate human intelligent behavior. The goal of AI is to develop intelligent systems that can perceive, learn, reason and make decisions, so that computers can imitate and execute human cognitive abilities. AI services are usually carried out in the form of user dialogues. AI dialogue services refer to dialogue systems based on artificial intelligence technology, which provide intelligent question answering, task execution and information query services through natural language interaction with users. The core of AI dialogue services is Natural Language Processing (NLP) and Dialog Management (Dialog Management) technology. It can understand user input, parse user intent, and generate corresponding answers or perform corresponding tasks.

[0041] In the process of AI dialogue service, a user's conversation starts with a command, but a single command usually cannot meet the user's needs. The user needs to continue to send prompts after issuing the command to continue the conversation. In order to measure the complexity of commands and prompts, they can be segmented into words to obtain tokens. The number of tokens can also measure the resource requirements of AI services to a certain extent, but it is not linearly related. The computing resource requirements of AI services are also closely related to other factors.

[0042] Due to the complexity of AI services, different users have changing resource requirements for different resource items in different AI conversations, such as CPU, GPU, memory, hard disk I / O rate, network I / O rate, etc. If a fixed amount of resources is provided to users, the user's AI service computing needs will not be well responded to, and the server resources may also be idle. In order to avoid this problem, the present invention provides the following solution.

[0043] See also Figures 1 to 3 As shown, the present invention provides an AI server resource dynamic allocation system, which includes an allocation calculation unit 1 and a scheduling execution unit 2 in terms of functional modules. The allocation calculation unit 1 is used to predict the demand for each resource item and resource quantity in combination with the user's historical records and the current conversation status, and then perform actual scheduling through the scheduling execution unit 2. In this process, dynamic and targeted computing resource scheduling can be performed in combination with user needs, avoiding idle computing resources while meeting user needs, and improving the effective working efficiency of the AI ​​server.

[0044] In the specific working process of this system, first, the allocation calculation unit 1 executes steps S1 and S2 to obtain the historical records of the tag words contained in the instructions and prompts of each user in each conversation by accessing the log file of the AI ​​server, and obtains the historical records of the resource amount of each resource item consumed by each user in each conversation. Next, step S3 can be executed to calculate the historical records of the tag words contained in the instructions and prompts of each user in each conversation and the resource amount of each resource item consumed in each conversation to obtain the correlation relationship between the tag words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item. This is not a simple linear numerical relationship between the number of tag words and the resource amount, but fully considers the keywords in the conversation and the number of tag words in the instructions and prompts therein.

[0045] In the process of interactive services between users and AI servers, step S4 can be executed first to obtain the user's instructions in the current conversation and continuously obtain prompts. Then, step S5 can be executed to continuously schedule and allocate resources for each resource item of the user in the current conversation based on the user's instructions in the current conversation and the mark words contained in the continuously updated prompts, as well as the relationship between the user's instructions in the conversation and the mark words contained in the prompts and the resource quantity of each resource item, so as to obtain the scheduled resource quantity of each resource item of the user in the current conversation, thereby obtaining an AI server resource dynamic scheduling plan that meets user needs.

[0046] In the specific execution link, the scheduling execution unit 2 can execute step S6 to receive the scheduled resource amount of each resource item of the user in the current conversation, and then execute step S7 to allocate server resources to the user according to the scheduled resource amount of each resource item of the user in the current conversation.

[0047] Of course, the above is based on the premise that AI server resources can fully meet user needs. If the AI ​​server resources cannot meet the needs of all users at the same time, they can also be reduced proportionally or adjusted and allocated according to the importance of each user.

[0048] See also Figure 4 As shown, since the marker words in the instructions and prompts of the dialogue are not in a simple linear relationship with the amount of resources consumed by different resource items, it is necessary to combine the specific content of the marker words and the number of marker words to establish a nonlinear association relationship. Before establishing the association relationship, it can be considered that the AI ​​service of the same user is coherent, so the association relationship can be established for each user separately. Specifically, for each user, step S31 can be first performed to select several keywords from the marker words in all instructions and prompts of each dialogue. The number of keywords can be one or more, but it needs to be consistent, usually three in practice. The extraction of keywords can use a frequency-based method to define keywords as words or phrases with high frequency in the text. Common technologies include TF-IDF (term frequency-inverse document frequency) and TextRank algorithm. TF-IDF calculates the importance of words based on the frequency of words and the inverse document frequency of words in a text collection. TextRank is a graph-based sorting algorithm that takes words in the text as nodes, constructs a graph based on the co-occurrence relationship between them, and iterates to calculate the importance score of the node.

[0049] In the subsequent process of establishing the association relationship, step S32 can be first executed to obtain the number of marker words in the instructions and prompts of each conversation. Next, step S33 can be executed to extract the input features of each conversation based on the number of marker words in the instructions and prompts of each conversation. Next, step S34 can be executed to obtain several typical input features and the resource amount of each resource item consumed in the corresponding conversation based on the input features of each conversation. This is because the number of marker words in the instructions and prompts of the conversation is difficult to compare directly, and it is easier to compare and calculate by extracting input features such as arrays and vectors. Finally, step S35 can be executed to use the keywords of each conversation in the historical records and each typical input feature as well as the resource amount of each resource item consumed in the corresponding conversation as the association relationship between the marker words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item. The association relationship in this solution does not refer to a single relationship, as long as it can establish a connection between the conversation and the resource amount.

[0050] See also Figure 5 As shown in the figure, since the AI ​​server may provide rich AI dialogue services to a large number of users at the same time, the number of dialogues in the historical records is extremely large, making it difficult to calculate and compare them one by one. In order not to reduce the response speed and prediction accuracy of the subsequent prediction of the current dialogue resource demand, it is necessary to select some representative dialogues from the numerous dialogues.

[0051] The above content is implemented in this solution, that is, some of the input features of many conversations are selected as typical input features. Specifically, step S341 can be executed first. For each conversation, the number of marker words contained in the instructions and prompts of the conversation, the number of marker words contained in all prompts, and the number of marker words contained in the last prompt are arranged in the same order as the input feature vector of the conversation. Next, step S342 can be executed to select several of the conversations as anchor conversations from all the conversations. It can be selected arbitrarily here. The number of selected anchor conversations can be arbitrary or set by the administrator. Next, step S343 can be executed to calculate the vector difference modulus between the input feature vectors of the anchor conversation and other conversations to measure the quantitative relationship between the anchor conversation and other conversations. Next, step S344 can be executed to divide the anchor conversation with the smallest vector difference modulus between each other conversation and the input feature vector into the same conversation group, that is, to try to normalize similar conversations. Next, steps S345 to S347 can be executed to determine whether the dialogue group to be normalized has the same identity. The specific steps are to calculate and obtain the mean vector of the input feature vectors of the dialogues contained in each dialogue group, and take the dialogue corresponding to the input feature vector with the smallest vector difference modulus length with the corresponding mean vector in each dialogue group as the updated anchor dialogue, and determine whether the updated anchor dialogue has changed. If so, it is determined that there is no normalization, and then steps S343 to S347 can be executed to continuously update the dialogue group and the anchor dialogue. If it is not determined that there is normalization, then step S348 can be executed to take the input feature vector of the anchor dialogue as a typical input feature, and obtain the resource amount of each resource item consumed in the corresponding anchor dialogue, which represents the correlation between the anchor dialogue and the resource amount of each resource item consumed.

[0052] In order to supplement the implementation process of the above steps S341 to S348, the source code of some functional modules is provided, and the explanation is compared in the comment section. In order to avoid the leakage of data involving commercial secrets, some data that does not affect the implementation of the solution are desensitized, the same below.

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059] This code implements the search for typical input features in a set of conversation data. Each conversation has an input feature vector that contains the number of tagged words and the number of tagged words in the last prompt, as well as the amount of resources consumed in the conversation. The conversations are grouped and the most typical input feature vector in each group is found as the anchor conversation. Then the amount of resources consumed in these anchor conversations can be obtained. The code includes steps such as finding the nearest anchor conversation and updating the anchor conversation to the mean. Finally, the code outputs the input feature vector and resource consumption of each anchor conversation.

[0060] See also Figure 6 As shown, in the process of predicting the resource demand of the user interacting with the AI ​​server at the current moment, it is necessary to consider the marker words contained in the instructions and prompts of the current conversation, so firstly, step S51 can be executed to obtain the marker words contained in the user's instructions in the current conversation. When the user has just issued an instruction in this conversation, step S52 can be executed next to generate the estimated resource amount of each resource item consumed by the user in the current conversation according to the marker words contained in the user's instructions in the current conversation and the association relationship between the marker words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item.

[0061] When the user is not satisfied with the result of the AI ​​server response, then step S53 can be executed to continuously obtain the marker words contained in the prompt of the user in the current conversation, and then step S54 can be executed to supplement the estimated resource amount of each resource item consumed by the user in the current conversation according to the marker words contained in the prompt of the user in the current conversation, so as to obtain the supplementary estimated resource amount of each resource item consumed by the user in the current conversation. Finally, step S55 can be executed to continuously schedule and allocate each resource item of the user in the current conversation according to the supplementary estimated resource amount of each resource item consumed by the user in the current conversation, so as to obtain the scheduled resource amount of each resource item of the user in the current conversation, that is, the AI ​​server resources prepared for the user to call after comprehensive consideration.

[0062] See also Figure 7As shown, in a conversation, when the user has only issued one instruction and has not issued a subsequent prompt, the most important thing at this time is the number of marker words in the instruction and the keywords contained in the instruction. In view of this, for each user, during the implementation of the above-mentioned step S52, step S521 can be first executed to extract the keywords in the marker words contained in the user's instructions in the current conversation. Next, steps S522 to S523 can be executed to determine whether a conversation with the same number of marker words and the same keywords contained in the corresponding instruction can be retrieved from the user's historical records as the target conversation. If so, it means that the target conversation exists in the user's historical records. Otherwise, it means that it does not exist in the user's historical records, and the historical records of other users need to be considered.

[0063] Although the usage habits of the same user are consistent, if the number of marker words and keywords in the instruction are the same, it can be said that the corresponding dialogue can be used as the target dialogue. Therefore, step S524 can be executed to retrieve dialogues with the same number of marker words and keywords in the corresponding instructions from the historical records of other users as the target dialogue. Finally, step S525 can be executed to use the resource amount of each resource item consumed in the target dialogue as the estimated resource amount of each resource item consumed by the user in the current dialogue, that is, the number of marker words and keywords in the instruction are taken as the priority consideration factors.

[0064] See also Figure 8 As shown, in the same conversation, compared with instructions, the AI ​​server resource consumption of prompts is more related to the number of marker words. And it is not only related to the number of marker words in the currently submitted prompt, but also to the number of marker words in the previously submitted prompts, and more to the number of marker words in the previously submitted instructions. In order to fully consider the above three factors, the user can first execute step S541 during the same conversation. Every time the user submits the prompt once, the input feature vector of the current conversation is obtained, which is composed of the number of marker words contained in the prompt of the user in the current conversation, the number of marker words contained in all the prompts, and the number of marker words contained in the last prompt, so as to fully consider the above three factors.

[0065] In the process of comparison and selection, step S542 can be first executed to obtain the input feature vector of the typical input feature in the historical record. Next, step S543 can be executed to calculate and select the dialogue with the smallest vector difference modulus length from the input feature vector of the typical input feature in the historical record as the target dialogue. Finally, step S544 can be executed to use the resource amount of each resource item consumed in the target dialogue as the supplementary estimated resource amount of each resource item consumed by the user in the current dialogue. The differences between dialogues are quantified by calculating the vector difference modulus length, which facilitates calculation and comparison.

[0066] See also Figures 9 to 10 As shown in the figure, since the estimated resource volume and supplementary estimated resource volume of each resource item calculated in the above steps are theoretical calculation values, if the AI ​​server resources are directly allocated according to these values, there may be a systematic deviation from the actual situation. In order to correct this systematic deviation between reality and theory, we can first collect the difference between reality and theory each time, and then correct the resource allocation of the current submitted instruction based on the difference.

[0067] For the same conversation of the user, in the process of collecting the difference between reality and theory each time, step S551 can be first executed to obtain the actual amount of resources consumed for each resource item after the user submits a prompt. Next, step S552 can be executed to obtain the ratio of the actual amount of resources to the supplementary estimated amount of resources for each resource item after the user submits a prompt. In the process of correcting the resource allocation of the current submission instruction in combination with the difference, step S553 can be first executed to obtain the correction coefficient of each resource item according to the ratio of the actual amount of resources to the supplementary estimated amount of resources for each resource item after the user submits a prompt each time. Finally, step S554 can be executed to correct the supplementary estimated amount of resources for each resource item consumed by the user in the current conversation with the correction coefficient of each resource item after the user submits a prompt, and obtain the scheduled resource amount of each resource item of the user in the current conversation.

[0068] The correction coefficient of each resource item in the above step S553 will change each time the user submits a prompt to the AI ​​server. In order to meet the user's needs as much as possible, it is necessary to appropriately relax the user's resource requirements. Therefore, it is necessary to combine the changing state of the correction coefficient after each user submits a prompt to derive the range of relaxation adjustment, that is, the execution prompt tolerance. Specifically, step S5531 can be first executed to calculate the difference between the maximum and minimum values ​​of the ratio of the actual resource amount to the supplementary estimated resource amount of each resource item after each user submits a prompt as the execution dialogue tolerance. Next, step S5532 can be executed to use the ratio of the execution prompt tolerance to the number of times the prompt is submitted as the execution prompt tolerance.

[0069] Since it is necessary to prioritize the user's resource needs and adjust the user's AI server resources upward, step S5533 can be executed next to accumulate the ratio of the actual resource amount of each resource item to the supplementary estimated resource amount after the user submitted the prompt last time to execute the prompt tolerance to obtain the correction coefficient of the resource item. Finally, step S5534 can be executed to summarize and obtain the correction coefficient of each resource item.

[0070] In order to provide additional explanation for the implementation process of the above-mentioned steps S5531 to S5534, the source code of some functional modules is provided, and a comparative explanation is provided in the comment section.

[0071]

[0072]

[0073]

[0074] Brief feature summary:

[0075] The above code first defines several types for storing actual and estimated values, ratios, and correction factors of resource items. The getResourceRatios function calculates and returns the ratio of the actual value to the estimated value of all resource items. The calculateCorrectionFactors function calculates the correction factor for each resource item. It first calculates the execution prompt tolerance, and then adds this value to the historical ratio to get the correction factor. The main function example shows that the user submits prompts multiple times, and the code is updated each time to implement the functions described above. To calculate the correction factor for each resource item, the necessary data structures and function functions are first defined in the code to calculate the ratio of the actual resource quantity to the supplementary estimated resource quantity, then calculate the difference between these ratios, and finally calculate the correction factor for each resource item.

[0076] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, system, method and computer program product according to multiple embodiments of the present application. In this regard, each square frame in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the square frame can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square frames can actually be executed substantially in parallel, and they can also be executed in reverse order sometimes, depending on the functions involved.

[0077] It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented by hardware that performs the corresponding function or action, such as a circuit or ASIC (Application Specific Integrated Circuit), or can be implemented by a combination of hardware and software, such as firmware.

[0078] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0079] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those of ordinary skill in the art without departing from the scope of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for dynamically allocating AI server resources, characterized in that: include, Get the history of token words contained in commands and prompts of each user in each conversation; Obtain a historical record of the amount of resources consumed by each user for each resource item in each of the conversations; Calculate the historical records of the marker words contained in the instructions and prompts of each user in each conversation and the resource amount of each resource item consumed in each conversation to obtain the association relationship between the marker words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item; Obtain the instructions of the user in the current conversation and continue to obtain the prompts; According to the instructions of the user in the current conversation and the marking words contained in the continuously updated prompts, and the association relationship between the instructions of the user in the conversation and the marking words contained in the prompts and the resource amount of each resource item, continuously scheduling and allocating the resource amount for each resource item of the user in the current conversation to obtain the scheduled resource amount of each resource item of the user in the current conversation; The step of calculating the historical records of the marker words contained in the instructions and prompts of each user in each conversation and the resource amount of each resource item consumed in each conversation to obtain the association relationship between the marker words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item includes: For each user, Selecting a number of keywords from the tag words in all the instructions and prompts of each conversation, Obtain the number of the marked words in the instructions and prompts of each conversation, Extracting input features of each conversation according to the number of the marker words in the instructions and prompts of each conversation, Obtaining a plurality of typical input features and the resource amount of each resource item consumed in the corresponding conversation according to the input features of each conversation; The keywords and each typical input feature of each conversation in the historical records and the resource amount of each resource item consumed in the corresponding conversation are used as the association relationship between the marking words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item; The step of obtaining a plurality of typical input features and the resource amount of each resource item consumed in the corresponding conversation according to the input features of each conversation includes: For each of the conversations, the number of the marker words included in the instructions and prompts of the conversation, the number of the marker words included in all the prompts, and the number of the marker words included in the last prompt are arranged in the same order as the input feature vector of the conversation; Selecting a number of the conversations from among all the conversations as anchor conversations; Calculate and obtain the vector difference modulus between the input feature vectors of the anchor dialogue and the input feature vectors of other dialogues; Assigning the anchor dialogue having the smallest vector difference modulus between the input feature vectors of each other dialogue into the same dialogue group; Calculating and obtaining a mean vector of the input feature vectors of the dialogues included in each dialogue group; Taking the dialogue corresponding to the input feature vector having the smallest vector difference modulus length with the corresponding mean vector in each dialogue group as the updated anchor dialogue; Determine whether the updated anchored dialogue has changed; If yes, then continue to update the conversation group and the anchored conversation; If not, the input feature vector of the anchored dialogue is used as a typical input feature, and the resource amount of each resource item consumed in the corresponding anchored dialogue is obtained.

2. The method according to claim 1, characterized in that The step of continuously scheduling and allocating the resource amount for each resource item of the user in the current conversation according to the instruction of the user in the current conversation and the tag words contained in the continuously updated prompt and the association relationship between the instruction of the user in the conversation and the tag words contained in the prompt and the resource amount of each resource item to obtain the scheduled resource amount of each resource item of the user in the current conversation, include, Obtaining the marking word contained in the instruction of the user in the current conversation; Generate an estimated resource amount of each resource item consumed by the user in the current conversation according to the marking words contained in the instructions of the user in the current conversation and the association relationship between the marking words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item; Continuously obtaining the marking words contained in the prompts of the user in the current conversation; Supplementing the estimated resource amount of each resource item consumed by the user in the current conversation according to the continuously acquired marking words contained in the prompt of the user in the current conversation to obtain the supplementary estimated resource amount of each resource item consumed by the user in the current conversation; According to the supplementary estimated resource amount of each resource item consumed by the user in the current conversation, scheduling and allocating are continuously performed for each resource item of the user in the current conversation to obtain the scheduled resource amount of each resource item of the user in the current conversation.

3. The method according to claim 2, characterized in that The step of generating an estimated resource amount of each resource item consumed by the user in the current conversation based on the marker words contained in the instructions of the user in the current conversation and the association relationship between the marker words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item, include, For each user, Extracting keywords from the tag words contained in the instructions of the user in the current conversation, Determine whether the conversation containing the same number of the marking words and the same keywords as the corresponding instruction can be retrieved from the user's history records as the target conversation, If yes, then get the target dialogue, If not, then the conversations containing the same number of the marking words and the same keywords in the corresponding instructions are retrieved from the historical records of other users as the target conversations, The resource amount of each resource item consumed in the target conversation is used as the estimated resource amount of each resource item consumed by the user in the current conversation.

4. The method according to claim 2, characterized in that: The step of supplementing the estimated resource amount of each resource item consumed by the user in the current conversation according to the continuously acquired marker words contained in the prompt of the user in the current conversation to obtain the supplementary estimated resource amount of each resource item consumed by the user in the current conversation includes: During the same conversation, the user Each time the user submits the prompt, an input feature vector of the current dialog is obtained, which is composed of the number of the marker words included in the prompt of the user in the current dialog, the number of the marker words included in all the prompts, and the number of the marker words included in the last prompt. Get the input feature vector of typical input features in the history, The dialogue with the smallest vector difference modulus between the input feature vector of the typical input features in the historical records and the input feature vector of the current dialogue is selected as the target dialogue. The resource amount of each resource item consumed in the target conversation is used as the supplementary estimated resource amount of each resource item consumed by the user in the current conversation.

5. The method according to claim 2, characterized in that: The step of continuously scheduling and allocating each resource item in the current conversation according to the supplementary estimated resource amount of each resource item consumed by the user in the current conversation to obtain the scheduled resource amount of each resource item in the current conversation includes: During the same conversation, the user Each time the user submits the prompt, the actual amount of resources consumed by each resource item is obtained. Obtaining the ratio of the actual resource quantity to the supplementary estimated resource quantity of each resource item after the user submits the prompt once, The correction coefficient of each resource item is obtained according to the ratio of the actual resource amount of each resource item to the supplementary estimated resource amount after the user submits the prompt each time, Every time the user submits the prompt, the correction coefficient of each resource item is used to correct the supplementary estimated resource amount of each resource item consumed by the user in the current session to obtain the scheduled resource amount of each resource item of the user in the current session.

6. The method according to claim 5, characterized in that The step of obtaining the correction coefficient of each resource item according to the ratio of the actual resource amount of each resource item to the supplementary estimated resource amount after the user submits the prompt each time includes: For each of said resource items, The difference between the maximum and minimum values ​​of the ratio of the actual resource quantity to the supplementary estimated resource quantity of each resource item after the user submits the prompt each time is calculated as the execution dialogue tolerance, The ratio of the execution dialogue tolerance to the number of times the prompt is submitted is used as the execution prompt tolerance, The correction coefficient of the resource item is obtained by adding the execution prompt tolerance to the ratio of the actual resource amount to the supplementary estimated resource amount of each resource item after the user submitted the prompt last time; The correction coefficient of each resource item is obtained by summarizing.

7. A method for dynamically allocating AI server resources, characterized in that: include, A scheduling execution unit, configured to receive the scheduling resource amount of each resource item of the user in the current conversation in the method for dynamic allocation of AI server resources according to any one of claims 1 to 6; Server resources are allocated to users according to the scheduled resource amount of each resource item in the current session.

8. An AI server resource dynamic allocation system, characterized in that: include, Allocating a computing unit for obtaining a history of the tag words contained in the instructions and prompts of each user in each conversation; Obtain a historical record of the amount of resources consumed by each user for each resource item in each of the conversations; Calculate the historical records of the marker words contained in the instructions and prompts of each user in each conversation and the resource amount of each resource item consumed in each conversation to obtain the association relationship between the marker words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item; Obtain the instructions of the user in the current conversation and continue to obtain the prompts; According to the instructions of the user in the current conversation and the marking words contained in the continuously updated prompts, and the association relationship between the instructions of the user in the conversation and the marking words contained in the prompts and the resource amount of each resource item, continuously scheduling and allocating the resource amount for each resource item of the user in the current conversation to obtain the scheduled resource amount of each resource item of the user in the current conversation; The scheduling execution unit is used to receive the scheduling resource quantity of each resource item in the current session from the user; Allocate server resources to users according to the scheduled resource amount of each resource item in the current session; The step of calculating the historical records of the marker words contained in the instructions and prompts of each user in each conversation and the resource amount of each resource item consumed in each conversation to obtain the association relationship between the marker words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item includes: For each user, Selecting a number of keywords from the tag words in all the instructions and prompts of each conversation, Obtain the number of the marked words in the instructions and prompts of each conversation, Extracting input features of each conversation according to the number of the marker words in the instructions and prompts of each conversation, Obtaining a plurality of typical input features and the resource amount of each resource item consumed in the corresponding conversation according to the input features of each conversation; The keywords and each typical input feature of each conversation in the historical records and the resource amount of each resource item consumed in the corresponding conversation are used as the association relationship between the marking words contained in the instructions and prompts of each user in the conversation and the resource amount of each resource item; The step of obtaining a plurality of typical input features and the resource amount of each resource item consumed in the corresponding conversation according to the input features of each conversation includes: For each of the conversations, the number of the marker words included in the instructions and prompts of the conversation, the number of the marker words included in all the prompts, and the number of the marker words included in the last prompt are arranged in the same order as the input feature vector of the conversation; Selecting a number of the conversations from among all the conversations as anchor conversations; Calculate and obtain the vector difference modulus between the input feature vectors of the anchor dialogue and the input feature vectors of other dialogues; Assigning the anchor dialogue having the smallest vector difference modulus between the input feature vectors of each other dialogue into the same dialogue group; Calculating and obtaining a mean vector of the input feature vectors of the dialogues included in each dialogue group; Taking the dialogue corresponding to the input feature vector having the smallest vector difference modulus length with the corresponding mean vector in each dialogue group as the updated anchor dialogue; Determine whether the updated anchored dialogue has changed; If yes, then continue to update the conversation group and the anchored conversation; If not, the input feature vector of the anchored dialogue is used as a typical input feature, and the resource amount of each resource item consumed in the corresponding anchored dialogue is obtained.