A method and device for processing information based on local large model

Abstract

The present application discloses a method and device for information processing based on a local large model, which belongs to the field of network technology. By constructing and deploying a task execution engine, the locally deployed large model can fully utilize the computing power of the graphics card to achieve efficient information processing. At the same time, model quantization processing significantly reduces the demand for video memory and improves the reasoning speed. The task execution engine simplifies some commonly used system interaction-related tasks, allowing users to complete various tasks more conveniently and improving the real-time response capability of the system; based on the local large model, users can conduct intelligent dialogues and task execution even in an off-network environment, ensuring data privacy and security, and realizing data privacy protection; users can initiate task requests without switching interfaces, improving operational efficiency and convenience; the task execution engine can achieve task execution that is deeply integrated with the system, simplifying the interaction between users and the system; by decomposing complex tasks, more practical execution steps and methods are provided.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of network technology, and in particular to a method and device for processing information based on a local large model. Background Art

[0002] In the field of artificial intelligence, large models have become a key driver of technological progress, particularly in natural language processing (NLP). Intelligent dialogue systems are widely used in customer service, personal assistants, and voice assistants. These systems simulate human conversational behavior to provide users with information queries and answers to their questions, significantly improving user experience and business efficiency.

[0003] Currently, most intelligent dialogue systems rely on large cloud-based models for natural language processing and response generation. While this model fully leverages cloud computing resources, it also presents challenges such as data privacy and security, network dependency, data transmission latency, and insufficient customization. Traditional dialogue systems typically only offer simple question-and-answer functionality, failing to meet complex user needs. Task execution often requires manual intervention, resulting in limited flexibility. Furthermore, the few systems that support both dialogue and task requests often require users to switch interfaces, compromising operational fluidity. These systems are limited to simple queries and basic command execution, lacking the capabilities for multi-step task planning and complex logical reasoning.

[0004] Therefore, there is an urgent need for a method based on a local large model to integrate intelligent dialogue, task request and execution to provide more secure, stable, efficient, convenient and intelligent services. Summary of the Invention

[0005] The present application provides a method and device for processing information based on a local large model. The technical solution is as follows:

[0006] In one aspect, a method for processing information based on a local large model is provided, the method comprising:

[0007] Deploy local large models and task execution engines;

[0008] Obtain user input, which supports text, voice, and file input;

[0009] Determining user intent based on the user input;

[0010] In response to the user intention indication being a task request, converting the task request into a processing instruction;

[0011] Feedback of the processing instruction, wherein the processing instruction includes text, voice and file.

[0012] Optionally, the deployment of the local large model and the task execution engine includes:

[0013] Building the task execution engine;

[0014] Quantifying the local large model and deploying it in the local environment and integrating it with the task execution engine;

[0015] Executing a dynamic on-demand loading strategy on the local large model;

[0016] Install the graphics card driver and software environment necessary for the local large model.

[0017] Optionally, obtaining user input includes:

[0018] In response to user voice input, the automatic speech recognition technology ASR is used to convert the voice input content into text input content in real time;

[0019] After the user confirms, the text input content is submitted, wherein the content input by the user is processed by a pre-processing module to ensure that the format and content are applicable.

[0020] Optionally, determining the user intention based on the user input includes:

[0021] Performing user intent determination on the user input using a prompt fine-tuning method based on the local large model, wherein the user intent determination includes intent recognition, context enhancement, content decomposition, content simplification, and normalization processing;

[0022] The user intention is determined through the constructed task vector database.

[0023] Optionally, in response to the user intention indication being a task request, after converting the task request into a processing instruction, the method further includes:

[0024] In response to the user indicating that the intention is a complex request, the complex request is disassembled into subtasks through prompt fine-tuning;

[0025] The task execution engine parses the subtasks and executes corresponding instructions.

[0026] Optionally, the dynamic on-demand loading strategy includes:

[0027] Setting embedding supports loading specific functional models on demand;

[0028] Setting up a dynamic loading and unloading mechanism that supports not actively loading all large models that may be needed, but supports selectively loading the models into the graphics memory based on task requirements and priorities;

[0029] A task scheduling and resource management mechanism is set up, which supports predicting the models that will be needed soon based on real-time monitoring of task queues and video memory usage.

[0030] Optionally, the constructing the task execution engine includes:

[0031] Build instruction sets based on system operation application scenarios;

[0032] Implement the command interface through the API provided by the operating system, third-party libraries, application programming interfaces or scripts;

[0033] System commands or general script instructions are processed using the default call method.

[0034] On the other hand, a device for processing information based on a local large model is also provided, the device comprising:

[0035] Local deployment module, used to deploy local large models and task execution engines;

[0036] An input acquisition module is used to acquire user input, and the user input supports text, voice and file input;

[0037] An intention determination module, configured to determine user intention based on the user input;

[0038] a request conversion module, configured to convert the task request into a processing instruction in response to the user intention indication being a task request;

[0039] The instruction feedback module is used to feedback the processing instruction, and the processing instruction includes a form of text, voice and file.

[0040] On the other hand, a server is provided, comprising a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the local large model-based information processing method as described in the above aspect.

[0041] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the local large model-based information processing method as described in the above aspects.

[0042] On the other hand, a computer program product is also provided, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the local large model-based information processing method described in the above aspect.

[0043] In an embodiment of the present application, the method constructs and deploys a task execution engine, and the locally deployed large model can fully utilize the computing power of the graphics card to achieve efficient information processing. At the same time, model quantization processing significantly reduces the demand for video memory and improves the inference speed. The task execution engine simplifies some common system interaction-related tasks, allowing users to complete various tasks more conveniently and improving the system's real-time responsiveness. Based on the local large model, users can conduct intelligent dialogues and task execution even in an offline environment, ensuring data privacy and security, and achieving data privacy protection. Users can initiate task requests without switching interfaces, improving operational efficiency and convenience. The task execution engine can achieve task execution that is deeply integrated with the system, simplifying the interaction between users and the system. By decomposing complex tasks, more practical execution steps and methods are provided. Based on the local large model, users can conduct intelligent dialogues and task execution even in an offline environment, ensuring data privacy and security, and achieving data privacy protection. Users can initiate task requests without switching interfaces, improving operational efficiency and convenience. The task execution engine can achieve task execution that is deeply integrated with the system, simplifying the interaction between users and the system. By decomposing complex tasks, more practical execution steps and methods are provided.

[0044] Among the other methods, by breaking down complex tasks, more practical execution steps and methods are provided; in addition, by combining ASR and TTS technologies, natural language interaction is achieved to enhance user experience; interfaces are also opened to support third-party application calls, expanding the application capabilities of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of a local large model information processing method provided by an exemplary embodiment of the present application is shown;

[0046] Figure 2 A schematic diagram of a dynamic on-demand loading model is shown;

[0047] Figure 3 shows a functional schematic diagram of a task execution engine;

[0048] Figure 4 A schematic diagram of user intention determination is shown. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0050] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0051] Please refer to Figure 1 , which shows a flowchart of a method for processing local large model information provided by an exemplary embodiment of the present application. This method is applicable to Figure 1 The live broadcast platform shown in FIG. 1 includes:

[0052] Step 101: deploy the local large model and task execution engine.

[0053] In a possible implementation, step 101 includes the following content.

[0054] Content 1: Build a task execution engine.

[0055] In order to simplify the complexity of some tasks related to the system and improve the real-time responsiveness of task execution, such as Figure 3 As shown, this method requires building and deploying a task execution engine in advance.

[0056] First, build an instruction set and define instructions and parameters based on common system operation application scenarios, such as searching for files, taking screenshots, printing files, sending emails, system settings, etc.

[0057] Then implement the command interface through the API, third-party library, application program interface or script instructions provided by the operating system. For system commands or general script instructions, such as Shell, Python, etc., the default call method is used to process them to be compatible with general script instructions to achieve general task execution capabilities.

[0058] The task execution engine specifically includes the following functions: instruction parsing, which is responsible for parsing the instructions entered by the user and converting them into specific operations; task scheduling, which schedules corresponding operations based on the instruction parsing results; instruction execution: calling the operating system API, third-party library, application program interface or script to complete specific instructions; result feedback: feeding back the execution results to the user, such as "music playback has started" or "email has been sent."

[0059] Content 2: Quantify the local large model, deploy it in the local environment, and integrate it with the task execution engine.

[0060] Content 3: Install the graphics card driver and software environment necessary for the local large model.

[0061] The purpose of this step is to deploy a large local model and task execution engine. The large model is quantified and deployed in the local environment. The task execution engine is integrated to ensure that the system can efficiently process various user requests. In addition, to accelerate the reasoning of the local large model, the necessary graphics card drivers and software environment can also be installed.

[0062] Furthermore, to deploy high-performance large models locally, you can collect graphics card hardware information, including video memory size, number of CUDA cores, and graphics card computing power. Based on the graphics card's performance, you can select the most appropriate large model parameters to ensure the graphics card can maximize its computing power. For example, for graphics cards with larger video memory, you can choose larger models with more parameters and greater complexity. Download and install the appropriate graphics card driver, CUDA, CUDNN libraries, and other necessary dependencies. These drivers and libraries provide key functionality required to accelerate deep learning model execution.

[0063] In order to further reduce the graphics memory required by the model and improve the inference speed, this application uses AutoGPTQ to quantize large models and convert the floating-point weights of large models into INT8 format. Through quantization, the graphics memory required by the model is significantly reduced, allowing large models to run efficiently on consumer-grade graphics cards while maintaining the performance of large models as much as possible.

[0064] In addition, considering that during each conversation, large models may have the problem of gradually increasing video memory usage, which may eventually lead to insufficient video memory, thereby affecting the reasoning ability of the next conversation or other models, such as voice wake-up, automatic speech recognition (ASR), text-to-speech (TTS) models, and even causing crashes. To solve this problem, this method introduces a strategy of dynamically loading models on demand, such as Figure 2 As shown, this optimizes video memory usage and improves the parameter capabilities and overall performance of large models. The specific implementation is as follows.

[0065] Modularity: Set up embedding to support on-demand loading of specific functional models rather than all required models to achieve on-demand loading of models.

[0066] The dynamic loading and unloading mechanism does not actively load all large models that may be needed. Only when a large model is actually needed, it is selectively loaded into the video memory according to task requirements and priority. After use, the model is immediately unloaded and the video memory resources occupied by it are reclaimed. For large models, since they are still needed after the conversation, the video memory usage is reduced by reloading.

[0067] Task scheduling and resource management, real-time monitoring of task queues and video memory usage, can predict the models that will be needed and load them in advance so as not to affect the user experience, but will only be executed when the system has sufficient video memory to avoid video memory overload.

[0068] At this point, through the above dynamic on-demand loading strategy, the video memory resources are optimized and the stability and responsiveness of the session process are ensured.

[0069] Step 102: Obtain user input. User input supports text, voice, and file input.

[0070] There are no restrictions on the format of user input; it can be text, voice, or files. If a user inputs via voice, the system will further utilize Automatic Speech Recognition (ASR) technology to convert the voice input into text in real time. After the user confirms the text input, the text input is submitted. The user input is pre-processed to ensure that its format and content are suitable for the next step of analysis and processing.

[0071] For example, if the user inputs data through a file, the text input content can be extracted using text recognition technology, which is not limited in this application.

[0072] Therefore, there are three main ways to obtain user input in step 102, namely text, real-time voice, and file. Among them, text input is the most common way. The user enters text content through the keyboard and then submits it through a button or shortcut key. In real-time voice input, the user starts recording through a button or shortcut key. This method uses the open source automatic speech recognition (ASR) technology whisper.cpp to convert voice into text in real time and display it in the text input box. The user confirms that the recognized text is correct or submits it after correction. In addition, the user can also select various types of files such as documents, pictures, audio or video through the upload interface. These files will be parsed and processed according to the conversation scenario. For example, when adding a PDF file to the knowledge base, mupdf is used to parse and process the PDF document to obtain the formatted text information therein; when extracting picture text, OpenCV is used for image processing and OCR technology for image text recognition; when setting the desktop background, it is only necessary to save the picture file path information as the corresponding instruction parameter. The above input methods can be selected or combined according to the specific application scenario and user needs.

[0073] This method also builds a server based on the Flask framework to handle HTTP requests and responses, making the large model's inference functionality available to third-party applications. The interface call process: The third-party application first sends a user authentication request to the / auth interface for authentication. Once authentication is successful, the third-party application calls the / api / function interface with a token and the required data. The server verifies the token's validity and, if successful, passes the request data to the large model's inference module for processing and returns the result. For example, WPS office software can implement a full-text summary function by calling the interface and sending the message "Please summarize this paragraph. Output a summary. The content you need to summarize is []," where [] contains the content to be summarized. Similarly, content polishing, document drafting, and other functions can be implemented. Third-party applications can call the interface to leverage the capabilities of the large model based on their functional needs. This method efficiently and securely opens the capabilities of the large model to third-party applications, thereby expanding its scope of application.

[0074] Step 103: Determine the user intention based on the user input.

[0075] Considering that there may be various task requests under different user intentions, step 103 includes the following content.

[0076] Content 4: Based on the local large model, the prompt fine-tuning method is used to determine the user intention of user input. The content of user intention determination includes intent recognition, context enhancement, content decomposition, content simplification and normalization.

[0077] Content 5. Determine user intention through the constructed task vector database.

[0078] In one possible implementation, the prompt fine-tuning method is used based on the local large model to perform a series of processing on the input content, such as intent recognition, context enhancement, content decomposition, content simplification, and normalization. Finally, the request corresponding to the user's intention is determined through the constructed task vector database, such as whether a corresponding request exists, so as to further accurately determine whether the user's intention is an existing request and provide support.

[0079] In addition, in response to the user's intention indicating a complex request, the complex request is disassembled into subtasks through Prompt fine-tuning, and the subtasks are parsed and the corresponding instructions are executed through the task execution engine.

[0080] Therefore, in step 103, the user intention is determined. This method performs a series of processing on the user's input content based on the large model through the Prompt fine-tuning method. The "[ ]" in the subsequent prompt words contains the text to be processed. These processing steps are as follows Figure 4 shown.

[0081] Intent recognition determines the intent of user input. Prompts include "You are a language expert and can determine intent based on the content. Intent can be categorized as information seeking, problem solving, social interaction, emotional support, suggestions and recommendations, opinion expression, learning and growth, transactions and purchases, operation requests, task requests, etc. Content: []".

[0082] Context enhancement: If the intent recognition is not an operation request or task request, based on the previous method, intent recognition is performed by combining conversation history and background information to ensure accurate understanding of the current input;

[0083] If the user intention is recognized as an operation request or task request, continue with subsequent processing.

[0084] Content decomposition: The input may contain complex sentence structures or long paragraphs. Therefore, it is necessary to decompose the complex and long sentences or paragraphs into multiple simple sentences to make subsequent processing more efficient. The prompt words may be "You are a language expert and are good at decomposing complex sentences or paragraphs into multiple simple sentences, one sentence per line. The content to be decomposed is: [ ]";

[0085] Simplify content, transforming complex expressions into more understandable and concise forms. This includes simplifying vocabulary, phrases, and syntactic structures for more accurate analysis. Prompts include, "You are a language expert and are skilled at simplifying sentences. Treat each line of a sentence separately, remove meaningless conjunctions and modifiers, and remove punctuation. The following content needs to be simplified: [ ]"

[0086] Normalization: This step aims to standardize various input forms, effectively reduce data redundancy, improve data consistency, and thus improve data processing efficiency. For example, noise words and invisible characters at the beginning and end are removed.

[0087] Task vector database matching: After completing the above steps, the processed information will be matched with the constructed task vector database to check whether the corresponding task exists and generate the corresponding instruction sequence. The task vector database is a vector database that contains all natural language instructions supported by the task execution engine, such as "turn on Bluetooth", "take a screenshot", "play music", etc. In this way, the system can more accurately determine whether the user's intention is a task request and determine whether the task execution engine can support it. The number of all natural language instructions supported by the task execution engine is huge. The use of vector database matching also solves the problem of the number of input tokens in the local large model, ensuring accurate identification of the intention of the user's operation request or task request.

[0088] Step 104 : In response to the user intention indicating that the task request is a task request, convert the task request into a processing instruction.

[0089] When the user's intent is a task request and the system supports it, the task request is processed and converted into corresponding instructions. For more complex tasks, the large model's prompt is fine-tuned to break down the task. Finally, the built task execution engine parses the instructions and executes the corresponding task. If the user's intent is not a task request or is not supported, the large model directly answers the user's input question based on the context.

[0090] When the user's intention is expressed as a task request, most tasks can be converted into one or more instructions directly supported by the task execution engine through step 104. For tasks that cannot be directly supported, the task request is decomposed into multiple simple and executable steps based on the large model prompt fine-tuning method, and executable Python or Shell script code is generated. The prompt words are such as "You are an experienced Linux system programmer who can reasonably decompose tasks into simple steps and can use the most appropriate language in Shell or Python to implement them according to task requirements. Task: [Help me move the files in my home directory whose file names contain work to the desktop]". According to the results of large model reasoning, the script code is extracted and used as parameters of bash or Python commands to form instructions.

[0091] Through the above method, most tasks are converted into instruction sequences directly supported by the task execution engine. The instruction sequence is then saved to the task execution engine's input file, and the task execution engine is launched. Each instruction is enclosed in curly braces ({}) to facilitate parsing. The task execution engine reads the contents of the input file, parses these instructions step by step, and executes the corresponding tasks. Instruction execution is implemented by calling operating system APIs, third-party libraries, application programming interfaces (APIs), or scripts. For example, the command corresponding to "open WiFi" is "open wifi." The open command supported by the task execution engine recognizes the parameter "wifi" and calls the command "nmcli radio wifi on" to turn on WiFi. Other methods for turning on the network include dbus communication and other more complex methods. The task execution engine selects the most appropriate method to implement the task function, improving real-time response and user experience.

[0092] If the user's intent isn't a task request, or the task execution engine doesn't support it, the large model will directly answer the user's input question based on the context. This way, whether the user's need is a complex task request or a general question, the system can provide effective responses and support.

[0093] Step 105: Feedback a processing instruction, which may be in the form of text, voice, or file.

[0094] The feedback can be in the form of text, voice or file to ensure that users can clearly understand the results, and voice playback is achieved through text-to-speech technology (TTS).

[0095] Therefore, in step 105, to ensure that users clearly understand the feedback question or task execution results, feedback can be provided in a variety of forms, including text, voice, or files. Text feedback uses intuitive text descriptions, which may include code and comments, making it easy for users to read and understand. For input that is not a task request, the macro model's answer is directly fed back. Otherwise, the task execution method and steps are returned. After completion, the execution status is also returned, such as "OK, Bluetooth is turned on" or "File burning failed, the disc does not support reading or writing." Voice feedback, as a supplement to text feedback, relies on text-to-speech (TTS) technology to enable users to perceive the results through hearing. This is suitable for situations with limited vision or where audio prompts are required. If the voice playback option is enabled, the text content will be played simultaneously with the text feedback when the user inputs voice. If the text content is too long, the text will be summarized based on the macro model and played back in voice, further improving the user experience. When the user right-clicks on a text, they can also choose to play the entire text to preserve their preferences. File feedback can include detailed content, charts, images, or other supplementary materials to provide more comprehensive information support. For example, when a user requests to "find Word documents in the home directory that have been modified in the past few days," a list of matching Word files will be displayed, allowing users to click to open the file. By combining these feedback forms, we can more effectively meet feedback needs from various inputs.

[0096] In addition to providing feedback on questions or task execution results, a local dynamic knowledge base will be built and maintained to store the model's self-evaluation and user feedback, enhancing its capabilities. The model's self-evaluation involves evaluating the quality of its responses or executions, conducting self-criticism. This includes analyzing the accuracy and relevance of responses, identifying potential errors and areas for improvement, and storing this information in the knowledge base. This information will be linked to the conversation in the knowledge base and provided to the model as context for similar questions in the future. Errors will be avoided in the next response, and improvements will be reflected in the next answer. User feedback: After each answer, users are provided with a user feedback option, allowing them to rate, evaluate, and provide suggestions on the answer or execution. The knowledge base records these user preferences, historical interactions, task methods, and tools. For example, if a user is dissatisfied with the answer and execution of "Move files containing the file name "work" in the home directory to the desktop," the knowledge base will link the conversation to the previous answer in the future and provide it to the model as context. The model will avoid the previous unsatisfactory response and, if satisfied, will directly retrieve the previous execution instruction, avoiding further steps such as intent determination, thereby accelerating answer and task response. Through the above methods, more accurate and personalized answers can be provided for future conversations, as well as more accurate and faster task responses.

[0097] Specifically, users can directly add new knowledge to the knowledge base, ensuring it's constantly updated and expanded. This can happen through a conversational process, such as entering a paragraph and being prompted to add the content to the knowledge base. It can also happen through files, such as dragging a file into an input box or button, then typing "Add the above file to the knowledge base." Users can also add, delete, or modify content or files directly within the knowledge base interface. Enhanced knowledge retrieval can help models provide more accurate and comprehensive information when answering questions.

[0098] In an embodiment of the present application, the method constructs and deploys a task execution engine, and the locally deployed large model can fully utilize the computing power of the graphics card to achieve efficient information processing. At the same time, model quantization processing significantly reduces the demand for video memory and improves the inference speed. The task execution engine simplifies some common system interaction-related tasks, allowing users to complete various tasks more conveniently and improving the system's real-time responsiveness. Based on the local large model, users can conduct intelligent dialogues and task execution even in an offline environment, ensuring data privacy and security, and achieving data privacy protection. Users can initiate task requests without switching interfaces, improving operational efficiency and convenience. The task execution engine can achieve task execution that is deeply integrated with the system, simplifying the interaction between users and the system. By decomposing complex tasks, more practical execution steps and methods are provided. Based on the local large model, users can conduct intelligent dialogues and task execution even in an offline environment, ensuring data privacy and security, and achieving data privacy protection. Users can initiate task requests without switching interfaces, improving operational efficiency and convenience. The task execution engine can achieve task execution that is deeply integrated with the system, simplifying the interaction between users and the system. By decomposing complex tasks, more practical execution steps and methods are provided.

[0099] Among the other methods, by breaking down complex tasks, more practical execution steps and methods are provided; in addition, by combining ASR and TTS technologies, natural language interaction is achieved to enhance user experience; interfaces are also opened to support third-party application calls, expanding the application capabilities of the system.

[0100] In addition, a local large model information processing device is provided, the device comprising:

[0101] Local deployment module, used to deploy local large models and task execution engines;

[0102] An input acquisition module is used to acquire user input, and the user input supports text, voice and file input;

[0103] An intention determination module, configured to determine user intention based on the user input;

[0104] a request conversion module, configured to convert the task request into a processing instruction in response to the user intention indication being a task request;

[0105] The instruction feedback module is used to feedback the processing instruction, and the processing instruction includes a form of text, voice and file.

[0106] Optionally, the local deployment module includes:

[0107] A first deployment unit, configured to construct the task execution engine;

[0108] A second deployment unit is used to quantify the local large model and deploy it in the local environment and integrate it with the task execution engine;

[0109] A third deployment unit is configured to execute a dynamic on-demand loading strategy for the local large model;

[0110] The fourth deployment unit is used to install the graphics card driver and software environment necessary for the local large model.

[0111] Optionally, the input acquisition module includes:

[0112] In response to user voice input, the automatic speech recognition technology ASR is used to convert the voice input content into text input content in real time;

[0113] After the user confirms, the text input content is submitted, wherein the content input by the user is processed by a pre-processing module to ensure that the format and content are applicable.

[0114] Optionally, the intention determination module includes:

[0115] A first determination module is configured to determine user intent of the user input based on the local large model using a prompt fine-tuning method, wherein the user intent determination includes intent recognition, context enhancement, content decomposition, content simplification, and normalization processing;

[0116] The second determination module is used to determine the user intention through the constructed task vector database.

[0117] Optionally, after the request conversion module, the device further includes:

[0118] A first processing module is configured to, in response to the user indicating that the intention is a complex request, decompose the complex request into subtasks through prompt fine-tuning;

[0119] The second processing module is configured to parse the subtasks through the task execution engine and execute corresponding instructions.

[0120] Optionally, the third deployment unit is further used to set up an embedded model that supports on-demand loading of specific functions; set up a dynamic loading and unloading mechanism, which supports not actively loading all large models that may be needed, and supports selectively loading the model into the video memory based on task requirements and priority; set up a task scheduling and resource management mechanism, which supports predicting the models that will be needed based on real-time monitoring of task queues and video memory usage.

[0121] Optionally, the first deployment unit is also used to build an instruction set based on the system operation application scenario; implement the instruction interface through the API, third-party library, application program interface or script provided by the operating system; and process system commands or general script instructions using the default call method.

[0122] On the other hand, a server is provided, comprising a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the local large model-based information processing method as described in the above aspect.

[0123] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the local large model-based information processing method as described in the above aspects.

[0124] On the other hand, a computer program product is also provided, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the local large model-based information processing method described in the above aspect.

[0125] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the local large model information processing method provided in the above embodiments.

[0126] Optionally, the computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid-state drive (SSD), or an optical disk, etc. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM).

[0127] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0128] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0129] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for processing information based on a local large model, characterized in that: The method comprises: Build a task execution engine; Quantifying the local large model and deploying it in the local environment and integrating it with the task execution engine; Executing a dynamic on-demand loading strategy on the local large model; Install the graphics card driver and software environment necessary for the local large model; Obtain user input, which supports text, voice, and file input; Performing user intent determination on the user input using a prompt fine-tuning method based on the local large model, wherein the user intent determination includes intent recognition, context enhancement, content decomposition, content simplification, and normalization processing; Determining the user intention through the constructed task vector database; In response to the user intention indication being a task request, converting the task request into a processing instruction; Feedback of the processing instruction, wherein the processing instruction may be in the form of text, voice, or file; After converting the task request into a processing instruction in response to the user's intention indication being a task request, the method further includes: in response to the user's intention indication being a complex request, decomposing the complex request into subtasks through prompt fine-tuning; parsing the subtasks and executing corresponding instructions through the task execution engine; Among them, the dynamic on-demand loading strategy includes: setting an embedded specific functional model that supports on-demand loading; setting a dynamic loading and unloading mechanism, which supports not actively loading all large models that may be needed, and supports selectively loading the model into the video memory according to task requirements and priority; setting a task scheduling and resource management mechanism, which supports predicting the models that will be needed based on real-time monitoring of task queues and video memory usage.

2. The method according to claim 1, characterized in that The obtaining of user input includes: In response to user voice input, the automatic speech recognition technology ASR is used to convert the voice input content into text input content in real time; After the user confirms, the text input content is submitted, wherein the content input by the user is processed by a pre-processing module to ensure that the format and content are applicable.

3. The method according to claim 1, characterized in that The step of constructing the task execution engine includes: Build instruction sets based on system operation application scenarios; Implement the command interface through the API provided by the operating system, third-party libraries, application programming interfaces or scripts; System commands or general script instructions are processed using the default call method.

4. A local large model information processing device, characterized in that: The device comprises: A local deployment module is used to build a task execution engine; quantify the local large model and deploy it in the local environment and integrate the task execution engine; implement a dynamic on-demand loading strategy for the local large model; install the graphics card driver and software environment necessary for the local large model; and also be used to set up an embedded model that supports on-demand loading of specific functions; set up a dynamic loading and unloading mechanism that supports not actively loading all large models that may be needed, and supports selectively loading the model into the video memory based on task requirements and priority; set up a task scheduling and resource management mechanism that supports predicting the models that will be needed based on real-time monitoring of task queues and video memory usage; An input acquisition module is used to acquire user input, and the user input supports text, voice and file input; An intent determination module is configured to determine user intent based on the local large model using a prompt fine-tuning method. The user intent determination includes intent recognition, context enhancement, content decomposition, content simplification, and normalization. The user intent is determined using a constructed task vector database. a request conversion module for, in response to the user's intention indicating a task request, converting the task request into a processing instruction; further for, in response to the user's intention indicating a complex request, decomposing the complex request into subtasks through prompt fine-tuning; parsing the subtasks through the task execution engine and executing corresponding instructions; The instruction feedback module is used to feedback the processing instruction, and the processing instruction includes a form of text, voice and file.

5. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the local large model information processing method according to any one of claims 1 to 3.

Images