A method and apparatus for saving computational power for large AI model inference
By introducing a front-end discrimination module and classification model, the system intelligently identifies the difficulty of tasks and processes them accordingly, solving the problem of unreasonable consumption of computing power in AI large language model inference tasks and achieving efficient and accurate task processing.
Patent Information
- Application Number
- CN202410330138.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-22
- Publication Date
- 2025-10-31
- Estimated Expiration
- 2044-03-22
AI Technical Summary
Existing AI large language models consume unreasonable amounts of computing power during inference tasks, especially since the computational power consumption is similar for simple and complex tasks, resulting in inefficient resource utilization and decreased accuracy.
A front-end discrimination module is introduced, which intelligently identifies the difficulty level of the input task through template matching and classification models. Simple tasks are processed by traditional NLU models, while complex tasks are processed by AI large language models. A classification model is built and deployed for further analysis.
It optimizes computing power allocation, improves inference efficiency and accuracy, avoids the accuracy decline caused by model compression in existing technologies, and has flexibility and scalability.
Smart Images
Figure CN118211660B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of natural language processing technology, and more particularly to the field of AI large models and human-computer interaction technology; specifically, it relates to a method and apparatus for saving the computational power of AI large model inference. Background Technology
[0002] In the field of natural language processing, significant progress has been made in the development of AI Large Language Models (LLM models), which are capable of processing and understanding complex natural language text.
[0003] However, these existing LLM models typically require a large amount of computational resources, especially during inference. Regardless of whether the task is simple or complex, or how complex the input task is, large language models often require similar computational power.
[0004] For example, there is no significant difference in the computational power consumption of the AI large language model between inputting a simple input like "hello" and inputting a complex mathematical problem, which is unreasonable in terms of resource utilization.
[0005] To reduce the inference cost and improve the inference efficiency of large AI language models, various methods have been tried at this stage. One common method is to reduce the model's parameter size or to compress and prune the model to reduce computing power consumption.
[0006] However, this existing approach leads to a decrease in the accuracy of large AI language models, failing to fully leverage the advantages of large models. Furthermore, this approach cannot address the issue of similar computational power consumption for simple and complex tasks. Summary of the Invention
[0007] Therefore, the purpose of this invention is to propose a method and apparatus for saving the inference computing power of large AI models. By introducing a front-end discrimination module, intelligent recognition of input features and task diversion are achieved, thereby reducing the inference computing power consumption of large AI models. The front-end discrimination module can automatically determine the difficulty level of the input features and submit texts of different difficulties to the appropriate processing modules, thereby reducing the inference cost of large AI language models, ensuring accurate processing of tasks of different difficulties, and improving the inference efficiency and accuracy of large AI language models.
[0008] This invention provides a method for saving computational power in large AI model inference, comprising the following steps:
[0009] S1. Receive user input text features, as well as context information and user status information; the input text features include questions, commands, etc.
[0010] S2. The front-end discrimination module performs preliminary analysis on the features of the input text to determine the difficulty level of the input task, and performs intelligent recognition and task splitting on the input task; the task splitting includes: simple task splitting and complex task splitting; template matching and rule matching technologies are used to achieve simple task splitting; if the input task is determined to be a simple task, the input task is handed over to the traditional Natural Language Understanding (NLU) processing; if the input task is determined to be a complex task, the input task is submitted to the AI large language model for deep reasoning;
[0011] The method for simple task offloading includes: designing and writing preset rules based on common simple patterns or features of specific scenarios; identifying and classifying simple tasks through the preset rules; and directly handing the input task over to a traditional natural language understanding (NLU) model for processing to avoid wasting the computing power of large language models.
[0012] S3. Construct and deploy a classification model to further analyze and judge input tasks that are difficult to handle by the preset rules;
[0013] The method for constructing and deploying the classification model includes the following steps:
[0014] S31. Collect a large amount of corpus data of varying difficulty;
[0015] S32. Use the collected corpus data to train the classification model so that the classification model can accurately identify the difficulty level of the task.
[0016] S33. Deploy the trained classification model to the front-end discrimination module to determine the difficulty of the input task in real time;
[0017] S4. Based on the judgment result of the front-end discrimination module, the input task is reasonably allocated to the traditional NLU model or large language model for processing.
[0018] Specifically, simple tasks are handled by traditional NLU models, while complex tasks are submitted to AI large language models. This task offloading approach optimizes and reduces the computational power required for large model inference.
[0019] Furthermore, the preset rules for simple task splitting in step S2 include any one or more combinations of the following:
[0020] Keyword rules: Used to identify simple questions containing specific keywords, such as "hello," "thank you," "goodbye," etc.
[0021] Formatting rules: These rules are used to match simple questions with specific formats, such as "Who is XXX?" or "What does XXX mean?". By using these formatting characteristics, you can quickly locate the answer.
[0022] Context rules: used to determine whether the current input task can be handled by a traditional NLU model based on the context of the dialogue;
[0023] Simple pattern rules: These are used to match simple questions with specific patterns, such as questions, requests, and commands. These specific patterns can be directly mapped to simple processing flows.
[0024] The preset rules can be regular expressions, keyword matching, etc., used to quickly determine whether the input belongs to a simple task.
[0025] Furthermore, the method for training the classification model in step S32 includes the following steps:
[0026] S321. Collect a large amount of corpus data of varying difficulty to train the classification model. This data covers different scenarios and domains to ensure the model's generalization ability;
[0027] S322. Label the collected data, labeling each sample as a simple task or a complex task, and use it as training data.
[0028] S323. Extract features from the labeled data. Features include: word vectors, sentence vectors, and contextual information.
[0029] S324. Select a suitable classification model, including: Naive Bayes, Support Vector Machine, Neural Network;
[0030] S325. Use labeled data and features to train the model, and optimize the loss function so that the model can accurately distinguish between simple and complex tasks.
[0031] S326. Use a validation set to validate the model, evaluate the model's accuracy and recall metrics, adjust model parameters, and prevent overfitting.
[0032] Through the above steps, a classification model that can accurately determine the difficulty of the input task is constructed, which is used to achieve task diversion, thereby improving inference efficiency and computing power utilization.
[0033] Furthermore, the template matching in step S2 involves matching the input task using predefined rules and patterns to determine whether the input task is a simple task. The template matching includes the following steps:
[0034] S21. Based on common simple problem patterns, predefine corresponding templates, which are written using JSGF syntax or ABNF syntax;
[0035] S22. Match the user's input task with the predefined template. If the match is successful, it is considered a simple task; if the match is unsuccessful, it is considered a non-simple task.
[0036] Furthermore, the difficulty level of the input task in step S2 includes the following levels:
[0037] It's very simple: it includes greetings, asking the time, and simple salutations such as "Hello," "What time is it?", "Thank you," etc.
[0038] Simple: This includes simple inquiries, requests, and commands, such as "How's the weather today?", "Recommend a movie?", "Turn on some music", etc.
[0039] Medium level: This includes inquiries that require inference based on context, slightly more complex requests, and commands, such as "What movie did you just mention?", "Can you help me check flight information?", "I want to go out to eat, can you recommend a restaurant?", etc.
[0040] Complex: This includes situations requiring deep reasoning, multi-turn dialogues, and complex problem-solving, such as "I want to buy a mobile phone, my budget is around 2000 yuan, can you recommend one?" or "I'm going to Shanghai on a business trip tomorrow, please check the weather and arrange my itinerary."
[0041] It is very complex: it requires expert-level knowledge and deep learning, such as "What is the backpropagation algorithm in machine learning?" or "I want to understand the principles of quantum computers."
[0042] The present invention also provides an apparatus for saving computational power for large AI model inference, and performs the method for saving computational power for large AI model inference as described above, including:
[0043] Input receiving module: used to receive the user's input text features, as well as context information and user state information;
[0044] The front-end discrimination module is used to perform preliminary analysis of the input text features, determine the difficulty level of the input task, and intelligently identify and split the input task. The task splitting includes simple task splitting and complex task splitting. Simple task splitting is achieved using template matching and rule matching technologies. If the input task is determined to be a simple task, it is handed over to traditional Natural Language Understanding (NLU) processing. If the input task is determined to be a complex task, it is submitted to an AI large language model for deep reasoning.
[0045] Classification model application module: used to build and deploy classification models, and to further analyze and judge input tasks that are difficult to handle by the preset rules;
[0046] Task splitting strategy module: Based on the judgment result of the front-end discrimination module, the input task is reasonably distributed to the traditional NLU model or large language model for processing.
[0047] Furthermore, the classification model application module includes:
[0048] Corpus Collection Submodule: Used to collect a large amount of corpus data of varying difficulty;
[0049] Model training submodule: Used to train the classification model using the collected corpus data, so that the classification model can accurately identify the difficulty level of the task;
[0050] Model deployment module: used to deploy the trained classification model to the front-end discrimination module to determine the difficulty of the input task in real time.
[0051] Furthermore, the model training submodule includes:
[0052] Data collection unit: Used to collect large amounts of corpus data of varying difficulty for training classification models. This data covers different scenarios and domains to ensure the model's generalization ability.
[0053] Data labeling unit: used to label the collected data, labeling each sample as a simple task or a complex task, and using it as training data;
[0054] Feature engineering unit: used to extract features from labeled data, including word vectors, sentence vectors, and contextual information;
[0055] Model selection unit: used to select a suitable classification model, including: Naive Bayes, Support Vector Machine, Neural Network;
[0056] Model training unit: Used to train the model using labeled data and features, and by optimizing the loss function, the model can accurately distinguish between simple and complex tasks;
[0057] Model Validation Unit: Used to validate the model using a validation set, evaluate the model's accuracy and recall metrics, adjust model parameters, and prevent overfitting.
[0058] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the method for saving computational power for large AI model inference as described above.
[0059] The method and apparatus of the present invention have the following advantages:
[0060] 1. Optimized computing power allocation: By introducing a front-end discrimination module, tasks are intelligently identified and distributed based on their difficulty. Simple tasks are handled by traditional NLU models, while complex tasks are submitted to large AI models. This effectively reduces the computational power consumption of large AI models, improves overall inference efficiency, and achieves reasonable allocation and optimization of computing power.
[0061] 2. Improved Processing Efficiency: By offloading tasks, large AI models only need to handle the truly complex tasks requiring in-depth processing, thus improving their overall processing efficiency. Meanwhile, traditional NLU models can quickly process simple tasks, further accelerating the overall processing speed.
[0062] 3. Maintaining accuracy: It avoids the accuracy drop issues caused by reducing model parameter size or compressing and pruning the model in existing technologies. Through task splitting, large language models can focus on processing complex tasks, maintaining high accuracy in task processing.
[0063] 4. Flexibility and scalability: The front-end discrimination module can integrate various technologies, such as template matching and classification models, exhibiting good flexibility and scalability. Through continuous optimization and updates, the front-end discrimination module can adapt to changes in different scenarios and needs, improving the accuracy and efficiency of task processing.
[0064] In summary, these advantages and positive effects make this invention a promising candidate for application in the field of natural language processing.
[0065] The present invention also provides a computer device, the computer device including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for saving computational power for large AI model inference as described above.
[0066] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0067] The method and apparatus for saving computational power for large AI model inference provided by this invention can optimize the allocation of computational power by introducing a front-end discrimination module and implementing a task splitting strategy, effectively improving processing efficiency, maintaining high accuracy in task processing, and having advantages and positive effects such as flexibility and scalability. It has broad application prospects in the field of natural language processing. Attached Figure Description
[0068] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.
[0069] In the attached diagram:
[0070] Figure 1 A processing system framework diagram for saving computational power for large AI model inference in this embodiment of the invention;
[0071] Figure 2 This is a flowchart of a method for saving computational power in large AI model inference according to the present invention;
[0072] Figure 3 This is a flowchart of the template matching method of the present invention;
[0073] Figure 4 This is a flowchart illustrating the method for constructing and deploying a classification model according to the present invention.
[0074] Figure 5 This is a flowchart of the method for training a classification model according to the present invention;
[0075] Figure 6 This is a schematic diagram of the configuration of a computer device according to an embodiment of the present invention. Detailed Implementation
[0076] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and products consistent with some aspects of this disclosure as detailed in the appended claims.
[0077] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
[0078] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."
[0079] The embodiments of the present invention will be described in further detail below.
[0080] This invention provides a method for saving computational power in large AI model inference, see [link to relevant documentation]. Figure 2 As shown, it includes the following steps:
[0081] S1. Receive user input text features, as well as context information and user status information; the input text features include questions, commands, etc.
[0082] S2. The front-end discrimination module performs preliminary analysis on the features of the input text to determine the difficulty level of the input task, and performs intelligent recognition and task splitting on the input task; the task splitting includes: simple task splitting and complex task splitting; template matching and rule matching technologies are used to achieve simple task splitting; if the input task is determined to be a simple task, the input task is handed over to the traditional Natural Language Understanding (NLU) processing; if the input task is determined to be a complex task, the input task is submitted to the AI large language model for deep reasoning;
[0083] The method for simple task offloading includes: designing and writing preset rules based on common simple patterns or features of specific scenarios; identifying and classifying simple tasks through the preset rules; and directly handing the input task over to a traditional natural language understanding (NLU) model for processing to avoid wasting the computing power of large language models.
[0084] The preset rules for simple task splitting include any one or more combinations of the following:
[0085] Keyword rules: Used to identify simple questions containing specific keywords, such as "hello," "thank you," "goodbye," etc.
[0086] Formatting rules: These rules are used to match simple questions with specific formats, such as "Who is XXX?" or "What does XXX mean?". By using these formatting characteristics, you can quickly locate the answer.
[0087] Context rules: used to determine whether the current input task can be handled by a traditional NLU model based on the context of the dialogue;
[0088] Simple pattern rules: These are used to match simple questions with specific patterns, such as questions, requests, and commands. These specific patterns can be directly mapped to simple processing flows.
[0089] The preset rules can be regular expressions, keyword matching, etc., used to quickly determine whether the input belongs to a simple task.
[0090] The template matching process uses predefined rules and patterns to match the input task and determine whether the input task is a simple task. The template matching process includes the following steps (see...). Figure 3 As shown):
[0091] S21. Based on common simple problem patterns, predefine corresponding templates, which are written using JSGF syntax or ABNF syntax;
[0092] S22. Match the user's input task with the predefined template. If the match is successful, it is considered a simple task; if the match is unsuccessful, it is considered a non-simple task.
[0093] The difficulty level of the input task includes the following levels:
[0094] It's very simple: it includes greetings, asking the time, and simple salutations such as "Hello," "What time is it?", "Thank you," etc.
[0095] Simple: This includes simple inquiries, requests, and commands, such as "How's the weather today?", "Recommend a movie?", "Turn on some music", etc.
[0096] Medium level: This includes inquiries that require inference based on context, slightly more complex requests, and commands, such as "What movie did you just mention?", "Can you help me check flight information?", "I want to go out to eat, can you recommend a restaurant?", etc.
[0097] Complex: This includes situations requiring deep reasoning, multi-turn dialogues, and complex problem-solving, such as "I want to buy a mobile phone, my budget is around 2000 yuan, can you recommend one?" or "I'm going to Shanghai on a business trip tomorrow, please check the weather and arrange my itinerary."
[0098] It is very complex: it requires expert-level knowledge and deep learning, such as "What is the backpropagation algorithm in machine learning?" or "I want to understand the principles of quantum computers."
[0099] S3. Construct and deploy a classification model to further analyze and judge input tasks that are difficult to handle by the preset rules;
[0100] The method for constructing and deploying the classification model includes the following steps (see...). Figure 4 As shown):
[0101] S31. Collect a large amount of corpus data of varying difficulty;
[0102] S32. Use the collected corpus data to train the classification model so that the classification model can accurately identify the difficulty level of the task.
[0103] The method for training the classification model includes the following steps (see...). Figure 5 As shown):
[0104] S321. Collect a large amount of corpus data of varying difficulty to train the classification model. This data covers different scenarios and domains to ensure the model's generalization ability;
[0105] S322. Label the collected data, labeling each sample as a simple task or a complex task, and use it as training data.
[0106] S323. Extract features from the labeled data. Features include: word vectors, sentence vectors, and contextual information.
[0107] S324. Select a suitable classification model, including: Naive Bayes, Support Vector Machine, Neural Network;
[0108] S325. Use labeled data and features to train the model, and optimize the loss function so that the model can accurately distinguish between simple and complex tasks.
[0109] S326. Use a validation set to validate the model, evaluate the model's accuracy and recall metrics, adjust model parameters, and prevent overfitting.
[0110] Through the above steps, a classification model that can accurately determine the difficulty of the input task is constructed, which is used to achieve task diversion, thereby improving inference efficiency and computing power utilization.
[0111] S33. Deploy the trained classification model to the front-end discrimination module to determine the difficulty of the input task in real time;
[0112] S4. Based on the judgment result of the front-end discrimination module, the input task is reasonably allocated to the traditional NLU model or large language model for processing.
[0113] Specifically, simple tasks are handled by traditional NLU models, while complex tasks are submitted to AI large language models. This task offloading approach optimizes and reduces the computational power required for large model inference.
[0114] Figure 1 This embodiment illustrates the processing system framework for saving computational power for large AI model inference.
[0115] This invention also provides an apparatus for saving computational power for large AI model inference, which performs the method for saving computational power for large AI model inference as described above, including:
[0116] Input receiving module: used to receive the user's input text features, as well as context information and user state information;
[0117] The front-end discrimination module is used to perform preliminary analysis of the input text features, determine the difficulty level of the input task, and intelligently identify and split the input task. The task splitting includes simple task splitting and complex task splitting. Simple task splitting is achieved using template matching and rule matching technologies. If the input task is determined to be a simple task, it is handed over to traditional Natural Language Understanding (NLU) processing. If the input task is determined to be a complex task, it is submitted to an AI large language model for deep reasoning.
[0118] Classification model application module: used to build and deploy classification models, and to further analyze and judge input tasks that are difficult to handle by the preset rules;
[0119] Task splitting strategy module: Based on the judgment result of the front-end discrimination module, the input task is reasonably distributed to the traditional NLU model or large language model for processing.
[0120] The classification model application module includes:
[0121] Corpus Collection Submodule: Used to collect a large amount of corpus data of varying difficulty;
[0122] Model training submodule: Used to train the classification model using the collected corpus data, so that the classification model can accurately identify the difficulty level of the task;
[0123] Model deployment module: used to deploy the trained classification model to the front-end discrimination module to determine the difficulty of the input task in real time.
[0124] The model training submodule includes:
[0125] Data collection unit: Used to collect large amounts of corpus data of varying difficulty for training classification models. This data covers different scenarios and domains to ensure the model's generalization ability.
[0126] Data labeling unit: used to label the collected data, labeling each sample as a simple task or a complex task, and using it as training data;
[0127] Feature engineering unit: used to extract features from labeled data, including word vectors, sentence vectors, and contextual information;
[0128] Model selection unit: used to select a suitable classification model, including: Naive Bayes, Support Vector Machine, Neural Network;
[0129] Model training unit: Used to train the model using labeled data and features, and by optimizing the loss function, the model can accurately distinguish between simple and complex tasks;
[0130] Model Validation Unit: Used to validate the model using a validation set, evaluate the model's accuracy and recall metrics, adjust model parameters, and prevent overfitting.
[0131] In this embodiment, the specific processing steps for intelligent identification and task allocation include:
[0132] (1) Rule matching: Use predefined rules, such as common simple question templates, to determine whether the features of the input text belong to a simple task; if the match is successful, it is handed over to the traditional natural language understanding NLU model for processing;
[0133] (2) Classification model judgment: If the rule matching fails, the classification model is used to further judge the difficulty of the input task; the classification model needs to be trained to accurately identify simple tasks and complex tasks.
[0134] (3) Task allocation: Based on the front-end judgment results, simple tasks are assigned to the traditional NLU model for fast processing, while complex tasks are submitted to the large language model for deep reasoning.
[0135] (4) Large model reasoning: Large models perform in-depth understanding of complex tasks to generate high-quality responses or execution commands.
[0136] (5) Result return: Return the processing results of the large language model or the traditional NLU model to the user.
[0137] This invention also provides a computer device. Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention; see the accompanying drawings. Figure 6 As shown, the computer device includes: an input device 23, an output device 24, a memory 22, and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the method for saving computational power for large AI model inference as provided in the above embodiments; wherein the input device 23, the output device 24, the memory 22, and the processor 21 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.
[0138] The memory 22, as a read / write storage medium for a computing device, can be used to store software programs and computer-executable programs, such as the program instructions corresponding to the method for saving AI large-model inference computing power as described in this embodiment of the invention. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device. Furthermore, the memory 22 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 22 may further include memory remotely located relative to the processor 21, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0139] Input device 23 can be used to receive input digital or character information, and generate key signal inputs related to user settings and function control of the device; output device 24 may include display devices such as a display screen.
[0140] The processor 21 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 22, thereby realizing the above-mentioned method of saving computing power for AI large model inference.
[0141] The computer equipment provided above can be used to execute the method for saving computational power for large AI model inference provided in the above embodiments, and has corresponding functions and beneficial effects.
[0142] This invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a method for saving computational power for large AI model inference as provided in the above embodiments. The storage medium can be any type of memory device or storage device, including: mounting media such as CD-ROM, floppy disk, or magnetic tape; computer system memory or random access memory such as DRAM, DDRRAM, SRAM, EDORAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements; the storage medium may also include other types of memory or combinations thereof; furthermore, the storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet); the second computer system can provide program instructions to the first computer for execution. The storage medium includes two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.
[0143] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the method for saving AI large model inference computing power as described in the above embodiments, but can also perform related operations in the method for saving AI large model inference computing power provided in any embodiment of the present invention.
[0144] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.
[0145] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for saving computational power in large AI model inference, characterized in that, Includes the following steps: S1. Receive the user's input text features, as well as context information and user state information; S2. Use the front-end discrimination module to perform preliminary analysis on the features of the input text, determine the difficulty of the input task, and perform intelligent recognition and task splitting of the input task; The task splitting includes: simple task splitting and complex task splitting; simple task splitting is achieved by using template matching and rule matching technologies; if the input task is determined to be a simple task, it is handed over to the traditional Natural Language Understanding (NLU) processing; if the input task is determined to be a complex task, it is submitted to the AI large language model for deep reasoning. S3. Build and deploy classification models to further analyze and judge input tasks that are difficult to handle by preset rules; The method for constructing and deploying the classification model includes the following steps: S31. Collect a large amount of corpus data of varying difficulty; S32. Use the collected corpus data to train the classification model so that the classification model can accurately identify the difficulty level of the task. S33. Deploy the trained classification model to the front-end discrimination module to determine the difficulty of the input task in real time; S4. Based on the judgment result of the front-end discrimination module, the input task is reasonably allocated to the traditional NLU model or large language model for processing; The preset rules for simple task splitting in step S2 include any one or more combinations of the following: Keyword rules: Used to identify simple questions that contain specific keywords; Formatting rules: Simple questions for matching specific formats; Context rules: used to determine whether the current input task can be handled by a traditional NLU model based on the context of the dialogue; Simple pattern rules: Simple questions used to match a specific pattern; The method for training the classification model in step S32 includes the following steps: S321. Collect a large amount of corpus data of varying difficulty to train the classification model. This data covers different scenarios and domains to ensure the generalization ability of the model. S322. Label the collected data, labeling each sample as a simple task or a complex task, and use it as training data. S323. Extract features from the labeled data. Features include: word vectors, sentence vectors, and contextual information. S324. Select a suitable classification model, including: Naive Bayes, Support Vector Machine, Neural Network; S325. Use labeled data and features to train the model, and optimize the loss function so that the model can accurately distinguish between simple and complex tasks. S326. Use a validation set to validate the model, evaluate the model's accuracy and recall metrics, adjust model parameters, and prevent overfitting. The template matching in step S2 involves matching the input task using predefined rules and patterns to determine whether the input task is a simple task. The template matching includes the following steps: S21. Predefine corresponding templates based on common simple problem patterns; S22. Match the user's input task with the predefined template. If the match is successful, it is considered a simple task; if the match is unsuccessful, it is considered a non-simple task. The difficulty level of the input task in step S2 includes the following levels: It's very simple: it includes greeting, asking the time, and a simple salutation; Simple: This includes simple inquiries, requests, and commands; Medium: This includes inquiries that require inference from the context, slightly complex requests, and commands; Complex: This includes tasks requiring deep reasoning, multi-turn dialogues, and complex problem-solving. It is extremely complex: it requires expert-level knowledge and deep learning.
2. An apparatus for saving computational power for large AI model inference, executing the method for saving computational power for large AI model inference as described in claim 1, characterized in that, include: Input receiving module: used to receive the user's input text features, as well as context information and user state information; Front-end discrimination module: used to perform preliminary analysis of the features of the input text, determine the difficulty of the input task, and perform intelligent recognition and task splitting of the input task; The task splitting includes: simple task splitting and complex task splitting; simple task splitting is achieved by using template matching and rule matching technologies; if the input task is determined to be a simple task, it is handed over to the traditional Natural Language Understanding (NLU) processing; if the input task is determined to be a complex task, it is submitted to the AI large language model for deep reasoning. Classification model application module: used to build and deploy classification models, and to further analyze and judge input tasks that are difficult to handle by the preset rules; Task splitting strategy module: Used to reasonably distribute input tasks to traditional NLU models or large language models for processing based on the judgment results of the front-end discrimination module.
3. The device for saving computational power for large AI model inference according to claim 2, characterized in that, The classification model application module includes: Corpus Collection Submodule: Used to collect a large amount of corpus data of varying difficulty; Model training submodule: Used to train the classification model using the collected corpus data, so that the classification model can accurately identify the difficulty level of the task; Model deployment module: used to deploy the trained classification model to the front-end discrimination module to determine the difficulty of the input task in real time.
4. The device for saving computational power for large AI model inference according to claim 3, characterized in that, The model training submodule includes: Data collection unit: used to collect a large amount of corpus data of varying difficulty for training classification models. This data covers different scenarios and domains to ensure the generalization ability of the model. Data labeling unit: used to label the collected data, labeling each sample as a simple task or a complex task, and using it as training data; Feature engineering unit: used to extract features from labeled data, including word vectors, sentence vectors, and contextual information; Model selection unit: used to select a suitable classification model, including: Naive Bayes, Support Vector Machine, Neural Network; Model training unit: Used to train the model using labeled data and features, and by optimizing the loss function, the model can accurately distinguish between simple and complex tasks; Model Validation Unit: Used to validate the model using a validation set, evaluate the model's accuracy and recall metrics, adjust model parameters, and prevent overfitting.
5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method for saving computational power for large AI model inference as described in claim 1.
6. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for saving computational power for large AI model inference as described in claim 1.
Citation Information
Patent Citations
Question and answer processing method and device
CN116450781A
Task planning-oriented model self-learning optimization method
CN117574776A