Information processing method and device based on local large model

Abstract

The invention discloses an information processing method and device based on a local large model, and belongs to the technical field of networks. According to the method, a task execution engine is constructed and deployed, so that a locally deployed large model can fully utilize the computing power of a graphics card, and efficient information processing is realized. Meanwhile, the video memory requirement is remarkably reduced through model quantification processing, and the reasoning speed is increased. A part of common system interaction related tasks are simplified by the task execution engine, so that a user can complete various tasks more conveniently, and the real-time response capability of the system is improved; based on a local large model, a user can perform intelligent dialogue and task execution even in a network-free environment, data privacy and security are guaranteed, and data privacy protection is realized; a user can initiate a task request without switching an interface, so that the operation efficiency and convenience are improved; the task execution engine can realize task execution deeply fused with the system, and the interaction between a user and the system is simplified; and more practical execution steps and methods are provided by disassembling complex tasks.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of network technology, and in particular to a method and device for processing information based on a local large model. Background Art

[0002] In the field of artificial intelligence, large models have become a key factor in promoting technological progress, especially in natural language processing (NLP). Intelligent dialogue systems are widely used in customer service, personal assistants, and voice assistants. These systems simulate human dialogue behavior to provide users with information query and problem answers, significantly improving user experience and business efficiency.

[0003] Currently, most intelligent dialogue systems rely on large cloud models for natural language processing and response generation. Although this model makes full use of cloud computing resources, it also brings problems such as data privacy and security, network dependence, data transmission delays, and insufficient customization. Traditional dialogue systems can usually only provide simple question-and-answer functions and cannot meet the complex needs of users. Task execution often requires manual intervention and has poor flexibility. In addition, the few systems that support both dialogue and task requests often require users to switch interfaces, affecting the smoothness of operation, and are limited to simple queries and basic command execution, lacking the ability to plan multi-step tasks and complex logical judgments.

[0004] Therefore, there is an urgent need for a method based on a local large model to integrate intelligent dialogue, task request and execution to provide safer, more stable, efficient, convenient and intelligent services. Summary of the invention

[0005] The embodiment of the present application provides a method and device for processing information based on a local large model, and the technical solution is as follows:

[0006] On the one hand, a method for processing information based on a local large model is provided, the method comprising:

[0007] Deploy local large models and task execution engines;

[0008] Obtain user input, where the user input supports text, voice and file input;

[0009] Determining user intention according to the user input;

[0010] In response to the user's intention indication being a task request, converting the task request into a processing instruction;

[0011] The processing instruction is fed back, and the processing instruction is in a form of text, voice and file.

[0012] Optionally, the deploying of the local large model and the task execution engine includes:

[0013] Building the task execution engine;

[0014] Quantifying the local large model and deploying it in the local environment and integrating it with the task execution engine;

[0015] Executing a dynamic on-demand loading strategy on the local large model;

[0016] Install the graphics card driver and software environment necessary for the local large model.

[0017] Optionally, obtaining user input includes:

[0018] In response to the user's voice input, the automatic speech recognition technology ASR is used to convert the voice input content into text input content in real time;

[0019] The text input content is submitted after confirmation by the user, wherein the content input by the user is processed by a preprocessing module to ensure that the format and content are applicable.

[0020] Optionally, determining the user intention according to the user input includes:

[0021] Based on the local large model, a prompt fine-tuning method is used to determine the user intention of the user input, wherein the user intention determination includes intention recognition, context enhancement, content decomposition, content simplification and normalization processing;

[0022] The user intention is determined through the constructed task vector database.

[0023] Optionally, in response to the user's intention indication being a task request, after converting the task request into a processing instruction, the method further includes:

[0024] In response to the user's intention indicating that the request is complex, the complex request is decomposed into subtasks through prompt fine-tuning;

[0025] The task execution engine parses the subtasks and executes corresponding instructions.

[0026] Optionally, the dynamic on-demand loading strategy includes:

[0027] Setting embedding supports on-demand loading of specific functional models;

[0028] Setting a dynamic loading and unloading mechanism, which supports not actively loading all large models that may be needed, and supports selectively loading the model into the video memory according to task requirements and priorities;

[0029] A task scheduling and resource management mechanism is set up, which supports predicting the models that will be needed soon based on real-time monitoring of task queues and video memory usage.

[0030] Optionally, the constructing the task execution engine includes:

[0031] Build instruction sets based on system operation application scenarios;

[0032] Implement the command interface through the API provided by the operating system, third-party libraries, application programming interfaces or scripts;

[0033] System commands or general script instructions are processed using the default call method.

[0034] On the other hand, a local large model information processing device is also provided, the device comprising:

[0035] Local deployment module, used to deploy local large models and task execution engines;

[0036] An input acquisition module is used to acquire user input, and the user input supports text, voice and file input;

[0037] An intention determination module, used to determine the user intention according to the user input;

[0038] a request conversion module, configured to convert the task request into a processing instruction in response to the user's intention indication being a task request;

[0039] The instruction feedback module is used to feedback the processing instruction, and the processing instruction includes text, voice and file.

[0040] On the other hand, a server is provided, comprising a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the local large model-based information processing method as described in the above aspect.

[0041] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the local large model-based information processing method as described in the above aspects.

[0042] On the other hand, a computer program product is also provided, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the local large model-based information processing method described in the above aspect.

[0043] In the embodiment of the present application, the method constructs and deploys a task execution engine, and the locally deployed large model can make full use of the computing power of the graphics card to achieve efficient information processing. At the same time, the model quantization processing significantly reduces the demand for video memory and improves the reasoning speed. The task execution engine simplifies some commonly used system interaction-related tasks, allowing users to complete various tasks more conveniently and improving the real-time response capability of the system; based on the local large model, users can conduct intelligent dialogues and task execution even in a network-free environment, ensuring data privacy and security, and realizing data privacy protection; users can initiate task requests without switching interfaces, improving operational efficiency and convenience; the task execution engine can realize task execution that is deeply integrated with the system, simplifying the interaction between users and the system; by disassembling complex tasks, more practical execution steps and methods are provided; based on the local large model, users can conduct intelligent dialogues and task execution even in a network-free environment, ensuring data privacy and security, and realizing data privacy protection; users can initiate task requests without switching interfaces, improving operational efficiency and convenience; the task execution engine can realize task execution that is deeply integrated with the system, simplifying the interaction between users and the system; by disassembling complex tasks, more practical execution steps and methods are provided.

[0044] In the other methods, more practical execution steps and methods are provided by decomposing complex tasks; in addition, ASR and TTS technologies are combined to achieve natural language interaction and improve user experience; interfaces are also opened to support third-party application calls and expand the application capabilities of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of a local large model information processing method provided by an exemplary embodiment of the present application is shown;

[0046] Figure 2 A schematic diagram of a dynamic on-demand loading model is shown;

[0047] Figure 3 A functional schematic diagram of a task execution engine is shown;

[0048] Figure 4 A schematic diagram of user intention determination is shown. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0050] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0051] Please refer to Figure 1 , which shows a flowchart of a local large model information processing method provided by an exemplary embodiment of the present application. The method is applicable to Figure 1 The live broadcast platform shown, the method includes:

[0052] Step 101, deploy the local large model and task execution engine.

[0053] In a possible implementation, step 101 includes the following content.

[0054] Content 1: Build a task execution engine.

[0055] In order to simplify the complexity of some tasks related to the system and improve the real-time responsiveness of task execution, such as Figure 3 As shown, this method requires building and deploying a task execution engine in advance.

[0056] First, build an instruction set and define instructions and parameters based on common system operation application scenarios, such as searching for files, taking screenshots, printing files, sending emails, system settings, etc.

[0057] Then the command interface is implemented through the API, third-party library, application program interface or script instruction provided by the operating system. For system commands or general script instructions, such as Shell, Python, etc., the default call method is used to process them to be compatible with general script instructions to achieve general task execution capabilities.

[0058] The task execution engine specifically includes the following functions: instruction parsing, which is responsible for parsing the instructions entered by the user and converting them into specific operations; task scheduling, which schedules corresponding operations based on the instruction parsing results; instruction execution: calling the operating system API, third-party library, application program interface or script to complete specific instructions; result feedback: feeding back the execution results to the user, such as "music playback has started" or "email has been sent."

[0059] Content 2: Quantify the local large model, deploy it in the local environment and integrate it with the task execution engine.

[0060] Content 3: Install the graphics card driver and software environment necessary for the local large model.

[0061] The purpose of this step is to deploy the local large model and task execution engine. The large model is quantified and deployed in the local environment, and the task execution engine is integrated to ensure that the system can efficiently handle various user requests. In addition, in order to accelerate the reasoning of the local large model, the necessary graphics card driver and software environment can also be installed.

[0062] Furthermore, in order to deploy a high-performance local large model, you can collect the hardware information of the graphics card, including the size of the video memory, the number of CUDA cores, the computing power of the graphics card, etc. Therefore, according to the performance of the graphics card, the most suitable large model parameters can be selected to ensure that the graphics card can maximize its computing power. For example, for a graphics card with a large video memory, you can choose a large model with more parameters and more complexity. Download and install the appropriate graphics card driver, CUDA, CUDNN library, and other necessary dependent environments. These drivers and libraries provide the key functions required to accelerate the operation of deep learning models.

[0063] In order to further reduce the video memory required for the model and improve the inference speed, this application uses AutoGPTQ to quantize the large model and convert the floating-point weights of the large model into INT8 format. Through quantization, the video memory required for the model is significantly reduced, so that the large model can run efficiently on consumer-grade graphics cards while maintaining the performance of the large model as much as possible.

[0064] In addition, considering that during each conversation, the large model may have the problem of gradually increasing video memory usage, which may eventually lead to insufficient video memory, thus affecting the next conversation reasoning or the reasoning ability of other models, such as voice wake-up, automatic speech recognition (ASR), text-to-speech (TTS) and other models, and even causing crashes. To solve this problem, this method introduces a strategy of dynamically loading models on demand, such as Figure 2 As shown, the use of video memory is optimized to improve the parameter capability and overall performance of large models. The specific implementation is as follows.

[0065] Modularity: settings are embedded to support on-demand loading of specific functional models, rather than all required models, to achieve on-demand loading of models.

[0066] The dynamic loading and unloading mechanism does not actively load all large models that may be needed. Only when a large model is actually needed, the model is selectively loaded into the video memory according to task requirements and priority. After use, the model is immediately unloaded and the video memory resources occupied by it are reclaimed. For large models, since they are still needed after the conversation, the video memory usage is reduced by reloading.

[0067] Task scheduling and resource management, real-time monitoring of task queues and video memory usage, can predict the models that will be needed soon, and load them in advance so as not to affect the user experience, but it will only be executed when the system has sufficient video memory to avoid video memory overload.

[0068] At this point, through the above dynamic on-demand loading strategy, the video memory resources are optimized and the stability and response speed of the session process are ensured.

[0069] Step 102: Obtain user input, where the user input supports text, voice and file input.

[0070] There is no limitation on the format of user input, which can be any of text, voice and file. For example, in response to the user inputting via voice, the automatic speech recognition technology ASR is further used to convert the voice input content into text input content in real time, and the text input content is submitted after the user confirms it. The content input by the user is ensured to be suitable in format and content through the preprocessing module, that is, to ensure that its format and content are suitable for the next step of analysis and processing.

[0071] For example, if the user inputs information through a file, the text input content can be extracted using text recognition technology, but this application does not limit this.

[0072] Therefore, there are three main ways to obtain user input in step 102, namely text, real-time voice, and file. Among them, text input is the most common way. The user enters the text content through the keyboard and then submits it through a button or shortcut key. Real-time voice input, the user starts recording through a button or shortcut key. This method uses the open source automatic speech recognition (ASR) technology whisper.cpp to convert the voice into text in real time and display it in the text input box. The user confirms that the recognized text is correct or submits it after correction. In addition, the user can also select various types of files such as documents, pictures, audio or video through the upload interface. These files will be parsed and processed according to the conversation scenario. For example, add a PDF file to the knowledge base, use mupdf to parse and process the PDF document, and obtain the formatted text information therein; extract the picture text, use OpenCV for image processing and OCR technology for image text recognition; set the desktop background, then only need to save the picture file path information as the corresponding instruction parameter. The above input methods can be selected or combined according to the specific application scenario and user needs.

[0073] This method also builds a server based on the Flask framework to process HTTP requests and responses, and opens the reasoning function of the large model to third-party applications. Interface call process: The third-party application first sends a user authentication request to the / auth interface for user authentication; after the authentication is passed, the third-party application carries the token and the required data to call the / api / function interface; the server verifies the validity of the token, and after passing, passes the request data to the large model reasoning function module for processing, and returns the processing result. For example, WPS office software can call the interface and send the message "Please help me summarize this content and output the summary. The content you want to summarize is [ ]", where [ ] contains the content to be summarized, so as to realize the full text summary function. Similarly, content polishing, official document drafting, etc. can also be realized. Third-party applications can call the interface according to functional needs to utilize the capabilities of the large model. Through the above method, the capabilities of the large model can be efficiently and safely opened to third-party applications, thereby expanding its application scope.

[0074] Step 103: Determine the user intention based on the user input.

[0075] Considering that there may be various task requests under different user intentions, step 103 includes the following contents.

[0076] Content 4. Based on the local large model, the prompt fine-tuning method is used to determine the user intention of the user input. The content of user intention determination includes intent recognition, context enhancement, content decomposition, content simplification and normalization.

[0077] Content 5. Determine user intention through the constructed task vector database.

[0078] In one possible implementation, the prompt fine-tuning method is used based on the local large model to perform a series of processing on the input content, such as intent recognition, context enhancement, content decomposition, content simplification and normalization. Finally, the request corresponding to the user's intention is judged through the constructed task vector database, such as querying whether there is a corresponding request, so as to further accurately judge whether the user's intention is an existing request and provide support.

[0079] In addition, in response to the user's intention indicating a complex request, the complex request is disassembled into subtasks through Prompt fine-tuning, and the subtasks are parsed and the corresponding instructions are executed through the task execution engine.

[0080] Therefore, in step 103, the user intention is determined. This method performs a series of processing on the user's input content based on the large model through the prompt fine-tuning method. The "[ ]" in the subsequent prompt word contains the text to be processed. These processing steps are as follows Figure 4 shown.

[0081] Intent recognition determines the intention of the user's input. The prompt words are such as "You are a language expert and can determine the intention based on the content. Intent can be classified into information seeking, problem solving, social interaction, emotional support, suggestions and recommendations, opinion expression, learning and growth, transactions and purchases, operation requests, task requests, etc. Content: [ ]".

[0082] Context enhancement: If the intent recognition is not an operation request or a task request, based on the previous method, the intent recognition is performed by combining the conversation history and background information to ensure accurate understanding of the current input.

[0083] If the user's intention is recognized as an operation request or a task request, continue with subsequent processing.

[0084] Content decomposition: The input may contain complex sentence structures or long paragraphs. Therefore, it is necessary to decompose the complex and long sentences or paragraphs into multiple simple sentences to make subsequent processing more efficient. The prompt words are such as "You are a language expert and are good at decomposing complex sentences or paragraphs into multiple simple sentences, one sentence per line. The content that needs to be decomposed is: [ ]";

[0085] Simplify the content and transform complex expressions into more understandable and concise forms. This includes simplifying vocabulary, phrases, and syntactic structures for more accurate analysis. Prompt words such as "You are a language expert and are good at simplifying sentences. Treat each line of the sentence separately, remove meaningless conjunctions and modifiers, and remove punctuation. The content that needs to be simplified is: [ ]";

[0086] Normalization processing, this step aims to standardize various input forms, effectively reduce data redundancy, improve data consistency, and thus improve data processing efficiency, such as removing interference words and invisible characters at the beginning and end;

[0087] Task vector database matching, after completing the above steps, the processed information will be matched with the constructed task vector database to check whether there is a corresponding task and generate a corresponding instruction sequence. The task vector database is a vector database containing all natural language instructions supported by the task execution engine, such as "turn on Bluetooth", "take a screenshot", "play music", etc. In this way, the system can more accurately determine whether the user's intention is a task request and determine whether the task execution engine can support it. The number of all natural language instructions supported by the task execution engine is huge. The use of vector database matching also solves the problem of the number of input tokens in the local large model, ensuring accurate identification of the intention of user operation requests or task requests.

[0088] Step 104 , in response to the user's intention indication being a task request, convert the task request into a processing instruction.

[0089] When the user's intention is a task request and the system can support it, the task request is processed and converted into corresponding instructions. For more complex tasks, the task is decomposed through the large model prompt fine-tuning method, and finally the built task execution engine parses the instructions and executes the corresponding task. When the user's intention is not a task request or is not supported, the large model directly answers the user's input question based on the context.

[0090] When the user's intention is expressed as a task request, most tasks can be converted into one or more instructions directly supported by the task execution engine through step 104. For tasks that cannot be directly supported, the task request is decomposed into multiple simple and executable steps based on the large model prompt fine-tuning method, and executable Python or Shell script code is generated. The prompt words are such as "You are an experienced Linux system programmer who can reasonably decompose tasks into simple steps and can use the most appropriate language in Shell or Python to implement them according to task requirements. Task: [Help me move the files in the home directory with the file name containing work to the desktop]". According to the results of the large model reasoning, the script code is extracted and used as a parameter of the bash or Python command to form an instruction.

[0091] Through the above method, most tasks will be converted into instruction sequences directly supported by the task execution engine, and then the instruction sequences will be saved to the input file of the task execution engine, and the task execution engine will be started. Each instruction is contained in curly brackets "{}" for easy processing and parsing. The task execution engine will read the content from the input file, parse these instructions step by step, and execute the corresponding tasks step by step. Instruction execution is implemented by calling operating system APIs, third-party libraries, application programming interfaces or scripts. For example, the instruction corresponding to "open WiFi network" is "open wifi". The open instruction supported by the task execution engine identifies the parameter wifi and calls the command "nmcli radio wifi on" to open the WIFI network. There are also dbus communication methods or other more complex methods to open the network. The task execution engine will choose the most appropriate method to implement the task function to improve the real-time response and user experience.

[0092] If the user's intention is not a task request, or the task execution engine does not support it, the big model will directly answer the user's input question based on the context. In this way, whether the user's needs are complex task requests or general questions, the system can provide effective responses and support.

[0093] Step 105: Feedback a processing instruction, in which the processing instruction may be in the form of text, voice, or file.

[0094] The feedback can be in the form of text, voice or file to ensure that users can clearly understand the results, and voice playback is achieved through text-to-speech technology (TTS).

[0095] Therefore, in step 105, in order to ensure that the user can clearly understand the feedback question or task execution result, the feedback form can be diversified, including text, voice or file. Text feedback uses intuitive text descriptions, which may contain codes and comments, so that users can easily read and understand. For non-task request input, the answer of the large model is directly fed back, otherwise the method and steps of task execution are returned, and the execution status will be returned after execution, such as "OK, Bluetooth is turned on", "Failed to burn the file, the CD does not support reading and writing", etc. Voice feedback, as a supplement to text feedback, relies on text-to-speech technology (TTS) to achieve so that users can perceive the results through hearing, which is suitable for situations with limited vision or requiring audio prompts. When the voice playback option is not turned off, when the user input is voice, the text content will be played at the same time as the text feedback. When the text content is too long, the text will be summarized based on the large model, and the summary content will be played by voice to further improve the user experience. When the user right-clicks the text, you can also choose to play the full text to retain the user's preferences. File feedback can include detailed content, charts, pictures or other supplementary materials to provide more comprehensive information support. For example, when a user requests to "find the word documents modified in the home directory in the last few days", a list of word files that meet the conditions will be fed back, and you can click to open the file. By combining the above feedback forms, the feedback needs of various inputs can be met more effectively.

[0096] While feeding back the results of questions or task execution, a local dynamic knowledge base will be built and maintained to save the self-evaluation of the big model and user feedback information to enhance the capabilities of the big model. The big model self-evaluates, and the big model evaluates the quality of its answers or executions for self-criticism, which includes analyzing the accuracy and relevance of the answers, finding possible errors and improvement points, and saving the above information to the knowledge base. When similar questions are asked in the future, the knowledge base will be associated with the conversation and provided to the big model as context. Errors will be avoided in the next answer, and improvements will be reflected in the next answer. User feedback, after each answer, user feedback options are provided, and users can score, evaluate and make suggestions on the answer or execution. The knowledge base records these user preferences, historical interactions, task methods and tools. For example, if the user is not satisfied with the answer and execution of "Move the files in the home directory with the file name containing work to the desktop", the knowledge base will be associated with the conversation and provided to the big model as context when similar questions are asked in the future. The big model will avoid the last unsatisfactory answer, and directly obtain the last execution instruction when satisfied, avoiding the steps of continuing intention judgment, thereby accelerating the answer and task response. Through the above methods, more accurate and personalized answers can be provided for future conversations, as well as more accurate and faster task responses.

[0097] In particular, users can directly add new knowledge to the knowledge base to ensure that the knowledge base is constantly updated and expanded. Knowledge can be added in the form of a dialogue, such as entering a paragraph of content and being prompted to add the above content to the knowledge base at the end; it can also be in the form of a file, dragging the file to the input box or button to add the file, and then entering "Add the above file to the knowledge base"; you can also directly operate through the knowledge base interface to add, delete or modify the content or files of the knowledge base. Knowledge retrieval enhancement can help the model provide more accurate and comprehensive information when answering questions.

[0098] In the embodiment of the present application, the method constructs and deploys a task execution engine, and the locally deployed large model can make full use of the computing power of the graphics card to achieve efficient information processing. At the same time, the model quantization processing significantly reduces the demand for video memory and improves the reasoning speed. The task execution engine simplifies some commonly used system interaction-related tasks, allowing users to complete various tasks more conveniently and improving the real-time response capability of the system; based on the local large model, users can conduct intelligent dialogues and task execution even in a network-free environment, ensuring data privacy and security, and realizing data privacy protection; users can initiate task requests without switching interfaces, improving operational efficiency and convenience; the task execution engine can realize task execution that is deeply integrated with the system, simplifying the interaction between users and the system; by disassembling complex tasks, more practical execution steps and methods are provided; based on the local large model, users can conduct intelligent dialogues and task execution even in a network-free environment, ensuring data privacy and security, and realizing data privacy protection; users can initiate task requests without switching interfaces, improving operational efficiency and convenience; the task execution engine can realize task execution that is deeply integrated with the system, simplifying the interaction between users and the system; by disassembling complex tasks, more practical execution steps and methods are provided.

[0099] In the other methods, more practical execution steps and methods are provided by decomposing complex tasks; in addition, ASR and TTS technologies are combined to achieve natural language interaction and improve user experience; interfaces are also opened to support third-party application calls and expand the application capabilities of the system.

[0100] In addition, a local large model information processing device is also provided, the device comprising:

[0101] Local deployment module, used to deploy local large models and task execution engines;

[0102] An input acquisition module is used to acquire user input, and the user input supports text, voice and file input;

[0103] An intention determination module, used to determine the user intention according to the user input;

[0104] a request conversion module, configured to convert the task request into a processing instruction in response to the user's intention indication being a task request;

[0105] The instruction feedback module is used to feedback the processing instruction, and the processing instruction includes text, voice and file.

[0106] Optionally, the local deployment module includes:

[0107] A first deployment unit, used to build the task execution engine;

[0108] A second deployment unit, configured to quantify the local large model and deploy it in a local environment and integrate the task execution engine;

[0109] A third deployment unit is used to execute a dynamic on-demand loading strategy for the local large model;

[0110] The fourth deployment unit is used to install the graphics card driver and software environment necessary for the local large model.

[0111] Optionally, the input acquisition module includes:

[0112] In response to the user's voice input, the automatic speech recognition technology ASR is used to convert the voice input content into text input content in real time;

[0113] The text input content is submitted after confirmation by the user, wherein the content input by the user is processed by a preprocessing module to ensure that the format and content are applicable.

[0114] Optionally, the intention determination module includes:

[0115] A first determination module is used to determine the user intention of the user input by using a prompt fine-tuning method based on the local large model, wherein the content of the user intention determination includes intention recognition, context enhancement, content decomposition, content simplification and normalization processing;

[0116] The second determination module is used to determine the user intention through the constructed task vector database.

[0117] Optionally, after the request conversion module, the device further includes:

[0118] A first processing module, configured to, in response to the user's intention indicating that the request is complex, decompose the complex request into subtasks through prompt fine-tuning;

[0119] The second processing module is used to parse the subtask through the task execution engine and execute corresponding instructions.

[0120] Optionally, the third deployment unit is further used to set up an embedded model that supports on-demand loading of specific functional models; set up a dynamic loading and unloading mechanism, which supports not actively loading all large models that may be needed, and supports selectively loading the model into the video memory based on task requirements and priorities; set up a task scheduling and resource management mechanism, which supports predicting models that will be needed based on real-time monitoring of task queues and video memory usage.

[0121] Optionally, the first deployment unit is also used to build an instruction set according to the system operation application scenario; implement the instruction interface through the API, third-party library, application program interface or script provided by the operating system; and process system commands or general script instructions using the default call method.

[0122] On the other hand, a server is provided, comprising a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the local large model-based information processing method as described in the above aspect.

[0123] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the local large model-based information processing method as described in the above aspects.

[0124] On the other hand, a computer program product is also provided, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the local large model-based information processing method described in the above aspect.

[0125] An embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement the local large model information processing method provided in the above embodiments.

[0126] Optionally, the computer readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid state drive (SSD), or an optical disk, etc. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM).

[0127] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0128] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0129] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing information based on a local large model, characterized in that: The method comprises: Deploy local large models and task execution engines; Obtain user input, where the user input supports text, voice and file input; Determining user intention according to the user input; In response to the user's intention indication being a task request, converting the task request into a processing instruction; The processing instruction is fed back, and the processing instruction is in a form of text, voice and file.

2. The method according to claim 1, characterized in that The deployment of the local large model and the task execution engine includes: Building the task execution engine; Quantifying the local large model and deploying it in the local environment and integrating it with the task execution engine; Executing a dynamic on-demand loading strategy on the local large model; Install the graphics card driver and software environment necessary for the local large model.

3. The method according to claim 1, characterized in that The obtaining of user input comprises: In response to the user's voice input, the automatic speech recognition technology ASR is used to convert the voice input content into text input content in real time; The text input content is submitted after confirmation by the user, wherein the content input by the user is processed by a preprocessing module to ensure that the format and content are applicable.

4. The method according to claim 1, characterized in that: The determining the user intention according to the user input includes: Based on the local large model, a prompt fine-tuning method is used to determine the user intention of the user input, wherein the user intention determination includes intention recognition, context enhancement, content decomposition, content simplification and normalization processing; The user intention is determined through the constructed task vector database.

5. The method according to claim 1, characterized in that After converting the task request into a processing instruction in response to the user's intention indication being a task request, the method further includes: In response to the user's intention indicating that the request is complex, the complex request is decomposed into subtasks through prompt fine-tuning; The task execution engine parses the subtasks and executes corresponding instructions.

6. The method according to claim 2, characterized in that The dynamic on-demand loading strategy includes: Setting embedding supports on-demand loading of specific functional models; Setting a dynamic loading and unloading mechanism, which supports not actively loading all large models that may be needed, and supports selectively loading the model into the video memory according to task requirements and priorities; A task scheduling and resource management mechanism is set up, which supports predicting the models that will be needed soon based on real-time monitoring of task queues and video memory usage.

7. The method according to claim 2, characterized in that The step of constructing the task execution engine includes: Build instruction sets based on system operation application scenarios; Implement the command interface through the API provided by the operating system, third-party libraries, application programming interfaces or scripts; System commands or general script instructions are processed using the default call method.

8. A local large model information processing device, characterized in that: The device comprises: Local deployment module, used to deploy local large models and task execution engines; An input acquisition module is used to acquire user input, and the user input supports text, voice and file input; An intention determination module, used to determine the user intention according to the user input; a request conversion module, configured to convert the task request into a processing instruction in response to the user's intention indication being a task request; The instruction feedback module is used to feedback the processing instruction, and the processing instruction includes text, voice and file.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the local large model information processing method as described in any one of claims 1 to 7 above.

Images