Model loading method and terminal equipment

Abstract

The invention provides a model loading method and terminal equipment, and relates to the technical field of terminals. The method comprises the steps that under the condition that a preset language model is not operated, at least part of model files of the preset language model are loaded in a first memory pool; and receiving a first user instruction, wherein the first user instruction is used for triggering a voice assistant function of the first application. And in response to the first user instruction, under the condition that all the model files of the preset language model are loaded in the first memory pool, running the preset language model, and generating a semantic reasoning result of the first user instruction through the preset language model.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on January 16, 2024, with application number 202410063464.0 and invention name “A method for optimizing loading speed of large models”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a model loading method and a terminal device. Background Art

[0003] In order to provide users with a better user experience, the functions provided by terminal devices are becoming more and more diverse. For example, the voice assistant function, as a new function of terminal devices, can provide services such as voice interaction and voice control. When providing the voice assistant function, the terminal device needs to load the model file of the corresponding function model to provide the corresponding function through the corresponding function model.

[0004] However, for large models, the model files of large models are relatively large, and the data volume can reach thousands of megabytes. When using the large model function of the terminal device, the model files of large models are loaded slowly, and the startup time of large models is long, which affects the user experience. Summary of the invention

[0005] The present application provides a model loading method and a terminal device for improving the loading rate of large models, shortening the startup time of large models, and improving the user experience.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In the first aspect, the present application provides a model loading method, which is applied to a terminal device, in which a first application is installed, and the first application is used to provide a voice assistant function through a preset language model. A first memory pool is created in the terminal device, and the first memory pool is used to load model files of a preset language model, and the total data volume of the model files of the preset language model is greater than the first preset data volume. When the preset language model is not running, at least part of the model files of the preset language model are loaded in the first memory pool, and a first user instruction is received, and the first user instruction is used to trigger the voice assistant function of the first application. In response to the first user instruction, when all the model files of the preset language model are loaded in the first memory pool, the preset language model is run, and the semantic reasoning result of the first user instruction is generated through the preset language model.

[0008] In this way, by setting a first memory pool for a preset language model, the frequency of the terminal device applying for and releasing memory space for a large language model can be reduced, the fragmentation of the memory space can be reduced, and the utilization rate of the memory space can be improved. When the large language model is idle, the terminal device pre-loads at least part of the model file of the large language model into the first memory pool, which can reduce the memory pressure when the large language model is started and speed up the startup of the large language model.

[0009] In a possible implementation of the first aspect, when the preset language model is not running, available memory information of the terminal device is obtained, where the available memory information is used to indicate the size of the available memory space. If the available memory space is greater than a first preset memory threshold, at least part of the model file of the preset language model is loaded into the first memory pool, and the size of the available memory space is inversely proportional to the size of the first memory pool.

[0010] In this implementation, the available memory space of the terminal device is greater than the first preset memory threshold, indicating that the available memory space of the terminal device is sufficient and the memory status is good. In this case, preloading the model file of the large language model in the first memory pool has little effect on the memory pressure of the terminal device, and can also reduce the amount of data loaded by the model file when the large language model is started, thereby speeding up the startup of the large language model.

[0011] In another possible implementation of the first aspect, if the available memory space is less than a first preset memory threshold and greater than a second preset memory threshold, some loaded model files are unloaded in the first memory pool, and the second preset memory threshold is greater than the first preset memory threshold.

[0012] In this implementation, if the available memory space of the terminal device is less than the first preset memory threshold and greater than the second preset memory threshold, it means that the memory pressure of the terminal device increases and the available memory space decreases. In this case, unloading the model files of the large language model in the first memory pool can reduce the memory pressure of the terminal device. At the same time, some model files can also be retained in the preset cache pool, so that some model files of the large language model are loaded with model files in the preset cache pool when the model is started.

[0013] In another possible implementation of the first aspect, if the memory space occupied by the preset language model service is greater than a preset occupancy threshold and the available memory space is less than or equal to a third preset memory threshold, the preset language model service is set to a killable state, and the preset language model service is used to run the preset language model, and the third preset memory threshold is greater than the second preset memory threshold and less than the first preset memory threshold.

[0014] In this implementation, the available memory space of the terminal device is between the third preset memory threshold and the second preset memory threshold, indicating that the memory pressure of the terminal device is large and the available memory space is insufficient. In order to alleviate the memory pressure of the terminal device, the preset language model service can also be set to a killable state when the memory space occupied by the preset language model service is large. In this way, memory space can be further released by killing large language model services to alleviate memory pressure.

[0015] In another possible implementation of the first aspect, if the available memory space is less than or equal to the second preset memory threshold, all loaded model files are unloaded in the first memory pool. In this implementation, when the available memory space is less than or equal to the second preset memory threshold, it indicates that the memory pressure of the terminal device is high and the available memory space is seriously insufficient. In this case, unloading all loaded model files in the first memory pool can alleviate the memory pressure of the terminal device.

[0016] In another possible implementation of the first aspect, the preset language model service may also be set to a killable state. In this implementation, when the available memory space of the terminal device is seriously insufficient, regardless of whether the memory space occupied by the large language model service is greater than the preset occupancy threshold, the terminal device sets the large language model service to a killable state, thereby further releasing memory space and alleviating memory pressure by killing the large language model service.

[0017] In another possible implementation of the first aspect, in response to a first user instruction, if there is an unloaded model file, the first memory pool is pre-filled according to a first data amount, where the first data amount is the data amount of the unloaded model file. In response to a memory application request of a preset language model service, the unloaded model file is loaded in the pre-filled first memory pool. After all model files of the preset language model are loaded in the first memory pool, the preset language model is run, and a semantic reasoning result of the first user instruction is generated through the preset language model.

[0018] In this implementation, the terminal device fills the first memory pool without receiving a memory application request from the large language model service. This can prepare memory space for loading model files of the large language model in advance, further improving the startup speed of the large language model.

[0019] In another possible implementation of the first aspect, if the available memory space of the terminal device is less than the first data volume, the memory space allocated to the terminal device is reclaimed. In this implementation, if the available memory space is less than the first data volume, it means that the available memory space is insufficient to provide the first memory pool with memory space for loading the first data volume. In this case, the terminal device expands the available memory space by reclaiming memory so that there is enough space in the available memory space to provide the first memory pool.

[0020] In another possible implementation of the first aspect, after the terminal device is started, a first memory pool of the preset language model is created. In this implementation, the terminal device creates a dedicated memory pool for the large language model when it is started, so that there is no need to compete with other processes for memory resources during the loading of the large language model, and the frequency of the terminal device applying for and releasing memory space for the large language model can be reduced, thereby reducing the fragmentation of the memory space and improving the utilization rate of the memory space.

[0021] In a second aspect, the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory. When the processor runs the computer program, the electronic device executes the method described in the first aspect and any possible implementation thereof.

[0022] In a third aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by an electronic device, the electronic device executes the method described in the first aspect and any possible implementation thereof.

[0023] In a fourth aspect, the present application provides a computer program product including program instructions, including a computer program, which, when executed by an electronic device, enables the computer to execute the method described in the first aspect and any possible implementation thereof. For example, the computer may be the electronic device described above.

[0024] In a fifth aspect, the present application provides a chip system, which is applied to an electronic device. The chip system includes an interface circuit and a processor. The interface circuit and the processor are interconnected by a line. The interface circuit is used to receive a signal from a memory and send a signal to the processor, the signal including a computer instruction stored in the memory. When the processor executes the computer instruction, the electronic device executes the method described in the first aspect and any possible implementation thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of an interface of a terminal device provided in an embodiment of the present application;

[0026] Figure 2A hardware structure block diagram of a terminal device provided in an embodiment of the present application;

[0027] Figure 3 A software structure block diagram of a terminal device provided in an embodiment of the present application;

[0028] Figure 4 A flowchart of dynamically loading or unloading a model file provided in an embodiment of the present application;

[0029] Figure 5 A flowchart of another dynamic loading or unloading model file provided in an embodiment of the present application;

[0030] Figure 6 A communication diagram of a software module provided in an embodiment of the present application;

[0031] Figure 7 A flowchart of starting a large language model provided in an embodiment of the present application;

[0032] Figure 8 Another flowchart for starting a large language model provided in an embodiment of the present application;

[0033] Fig. 9 A communication diagram of another software module provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to provide users with a better user experience, the terminal device is equipped with a voice assistant application (or voice assistant). The voice assistant can provide the function of recognizing the user's voice commands and executing the corresponding operations of the voice commands. For example, the voice assistant can provide services such as voice dialogue, voice broadcast, and opening applications. The voice assistant can reduce the user's input dependence on input devices such as keyboards and mice, and provide users with a more convenient way of interaction.

[0035] For example, Figure 1 As shown in (1), the terminal device displays a desktop interface. When the terminal device displays the desktop interface, it receives a voice command of "What is the weather like today?". Figure 1 As shown in (2), the terminal device responds to the voice command, pops up a dialogue window of the voice assistant application on the desktop interface, and prompts the weather forecast content "Today's weather is sunny, the temperature is 0℃-10℃" in the dialogue window. At the same time, the terminal device can also voice broadcast "Today's weather is sunny, the temperature is 0℃-10℃".

[0036] The voice assistant application of the terminal device provides the voice assistant function through a large language model (LLM). When the voice assistant provides the voice assistant function through a large language model, the terminal device needs to load the model file of the large language model. After all the model files of the large language model are loaded, the terminal device can run the large language model. With the continuous improvement of the voice assistant function, the model files of the large language model are getting larger and larger. The data volume of the model files of the large language model can reach gigabytes. Accordingly, the terminal device loads the model files of the large language model slowly, and the voice assistant application takes a long time to respond to user commands, affecting the user experience.

[0037] In addition, since the model files of large language models are large, the terminal device is under great pressure to allocate memory space for large language models. When the terminal device loads the model files of large language models, other applications running in the background may be checked and killed due to insufficient available memory space on the terminal device, so that the terminal device has sufficient available memory space. After the terminal device checks and kills the application running in the background, the user data of the application may be lost, affecting the user experience.

[0038] In view of this, in the method provided in the embodiment of the present application, a first memory pool dedicated to loading model files of a large language model can be set in the terminal device. When the large language model is not running, the terminal device pre-loads at least part of the model files of the large language model in the first memory pool. When the terminal device receives a user instruction to trigger the voice assistant function, the terminal device responds to the user instruction, and when all model files are loaded in the first memory pool, runs the preset language model, and generates the semantic reasoning result of the user instruction through the preset language model.

[0039] In this way, by setting a first memory pool for a preset language model, the frequency of the terminal device applying for and releasing memory space for a large language model can be reduced, the fragmentation of the memory space can be reduced, and the utilization rate of the memory space can be improved. When the large language model is idle, the terminal device pre-loads at least part of the model file of the large language model in the first memory pool, which can reduce the memory pressure when the large language model is started and speed up the startup of the large language model. In addition, when the memory pressure is small when the large language model is started, the possibility of the terminal device killing background applications or processes also becomes smaller, which can reduce the probability of applications or processes being killed.

[0040] Exemplarily, the terminal device described in the embodiments of the present application may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR)\virtual reality (VR) device, a media player, a wearable device, etc. The embodiments of the present application do not impose any special restrictions on the specific form of the terminal device.

[0041] In the embodiment of the present application, the terminal device is a mobile phone 100 as an example, and the hardware structure of the terminal device is introduced through the mobile phone 100. Figure 2 As shown, the mobile phone 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0042] Among them, the processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural network processor (neural-network processing unit, NPU), a driver processor, etc. Among them, different processing units can be independent devices or integrated in one or more processors. The processor 110 can be the nerve center and command center of the mobile phone 100. The processor 110 can generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0043] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. In this way, repeated access may be avoided, the waiting time of the processor 110 may be reduced, and the efficiency of the system may be improved.

[0044] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, the mobile phone 100 saves files such as music and videos in an external memory card. In some embodiments, the mobile phone 100 stores the model file of the large language model in the external memory card. When the large language model is started, the mobile phone 100 loads the model file of the large language model into the memory.

[0045] The internal memory 121 may be used to store computer executable program codes, which may include instructions. The processor 110 executes various functional applications and data processing of the mobile phone 100 by running the instructions stored in the internal memory 121. For example, in an embodiment of the present application, the processor 110 may execute the instructions stored in the internal memory 121, and the internal memory 121 may include a program storage area and a data storage area.

[0046] Among them, the program storage area can store the operating system, the application required for at least one function (such as the sound playback function, the image playback function, etc.), the configuration file of the motor 191, etc. The data storage area can store the data created during the use of the mobile phone 100 (such as audio data, phone book, etc.). In addition, the internal memory 121 may include a high-speed random access memory and may also include a non-volatile memory. For example, the internal memory 121 may include at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0047] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193 and the wireless communication module 160. In some embodiments, the power management module 141 and the charging management module 140 can also be provided in the same device.

[0048] The wireless communication function of the mobile phone 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor. In some embodiments, the antenna 1 of the mobile phone 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the mobile phone 100 can communicate with the network and other devices through wireless communication technology.

[0049] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in mobile phone 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0050] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the mobile phone 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation.

[0051] The wireless communication module 160 can provide wireless communication solutions for the mobile phone 100, including wireless local area networks (WLAN) (such as Wi-Fi networks, Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0052] The mobile phone 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D and the application processor, etc. For example, music playing, recording, etc. In some implementations, the mobile phone 100 can collect the user's voice commands through the microphone 170C.

[0053] The sensor module 180 may include sensors such as a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a Hall sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor. The mobile phone 100 may collect various data through the sensor module 180 .

[0054] The mobile phone 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0055] The display screen 194 is used to display images, videos, etc. For example, the mobile phone 100 displays an application interface through the display screen 194. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), MiniLED, MicroLED, Micro-OLED, a quantum dot light emitting diode (QLED), etc.

[0056] In some implementations, the touch sensor may be disposed in the display screen 194, and the touch sensor and the display panel may form a touch screen, also known as a "touch screen". The touch sensor, also known as a "touch panel", is used to detect touch operations on or near it, such as click operations, sliding operations, etc. The touch sensor may transmit the detected touch operations to the application processor to determine the type of touch event. The mobile phone 100 may provide visual output related to the touch operation through the display screen 194.

[0057] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation on the terminal device. In other embodiments, the terminal device may also include more or fewer modules than those provided in the above embodiment, and the modules may also adopt different interface connection methods in the above embodiment, or a combination of multiple interface connection methods.

[0058] The software system of the terminal device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture or a cloud architecture. In the embodiment of the present application, the terminal device is the mobile phone 100, and the software system of the terminal device adopts the layered architecture of the Android system as an example to illustrate the software structure of the terminal device.

[0059] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system may include an application layer, an application framework layer, an Android runtime and system libraries, a hardware abstraction layer (HAL), and a kernel layer.

[0060] The application layer may include a series of application packages. For example, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, voice assistant, large language model service (LLMService), etc., and the embodiments of the present application do not impose any restrictions on this. Among them, the voice assistant is used to provide a voice assistant function. The large language model service is used to run a large language model. The voice assistant communicates with the large language model service and passes user instructions to the large language model service. The large language model service is used to run a large language model, analyze, reason, and other processes the user instructions through the large language model, obtain the semantic reasoning results corresponding to the user instructions, and return the semantic reasoning results obtained by the large language model to the voice assistant.

[0061] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some predefined functions. For example, the application framework layer may include a system service (System Service), a window manager, a content provider, a view system, a phone manager, a resource manager, and a notification manager, etc., and the embodiments of the present application do not impose any restrictions on this. Among them, the system service is used to start and manage some service processes of the operating system, such as starting the window manager, the content provider, etc. The system service is also used to obtain the memory status, such as the system service obtains the available memory information of the total memory space through the iaware program.

[0062] The Android runtime consists of a core library and a virtual machine. The Android runtime is responsible for scheduling and management of the Android system. The core library consists of two parts: one is the function that the Java language needs to call, and the other is the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0063] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0064] The HAL layer encapsulates the Linux kernel driver, provides an interface to the upper layer, and shields the implementation details of the underlying hardware. For example, the HAL layer may include camera HAL, Wi-Fi HAL, Bluetooth HAL, etc.

[0065] The kernel layer is the layer between hardware and software. The kernel layer includes drivers and system service programs. Drivers at least include memory management modules, display drivers, camera drivers, audio drivers, sensor drivers, etc. Among them, the memory management module is used to manage memory space, such as allocating and recycling memory space.

[0066] It is to be understood that the software modules included in each software layer in the software structure illustrated in this embodiment are only schematic illustrations and do not constitute a limitation on the software structure of the terminal device. In other embodiments, the terminal device may also include more or fewer software modules than those provided in the above embodiment, or the software layer where some software models are located may be different from the software layer provided in the above embodiment. For example, the above large language model may be located in the application framework layer.

[0067] In the following embodiments of the present application, the terminal device is the mobile phone 100, and the software system of the mobile phone 100 adopts the layered architecture of the Android system. Figure 3 The software architecture shown exemplifies the workflow of the software and hardware of the mobile phone 100.

[0068] The mobile phone 100 collects the user's voice command through the microphone 170C, and the voice command is used to trigger the voice assistant function of the voice assistant application. The voice assistant application of the mobile phone 100 responds to the voice command and pulls up the large language model service. The large language model service instructs the memory management module of the kernel layer to load the model file of the large language model through the system service. After the model file of the large language model is loaded, the large language model service runs the large language model, and performs semantic analysis and voice reasoning on the received voice command through the large language model to obtain the semantic reasoning result corresponding to the voice command. For example, in the voice command including the first preset instruction of "what day is today", the semantic reasoning result corresponding to the voice command can be obtained as "Friday" through the large language model. Further, the large language model service returns the semantic reasoning result to the voice assistant application. The voice assistant application further presents the semantic reasoning result to the user, such as playing the semantic reasoning result through the speaker 170A, or displaying the semantic reasoning result through the display screen 194.

[0069] As described above, a first memory pool for loading model files of a large language model (or a preset language model) is created in the terminal device. The first memory pool is a dedicated memory pool set for a large language model (or a preset language model). The terminal device can create a memory pool for loading model files for a large language model in advance, so that there is no need to compete with other processes for memory resources during the loading of the large language model. In this way, when the terminal device starts the large language model, it can directly use the memory space of the first memory pool, which can improve the loading rate of the large language model. In addition, the first memory pool can also reduce the frequency of applying for and releasing memory space for the large language model, reduce the fragmentation of the memory space, and improve the utilization of the memory space.

[0070] In order to further improve the loading rate of the large language model, the terminal device can also pre-load the model file of the large language model in the first memory pool without running the large language model. In some embodiments, the terminal device can dynamically load or unload the model file of the large language model in the first memory pool according to the available memory information of the terminal device, so as to realize dynamic adjustment of the amount of data loaded by the model file. For example, when the available memory information indicates that the available memory space of the terminal device is sufficient (such as the available memory space is greater than the preset memory threshold), the model file of the large language model is loaded in the first memory pool. When the available memory information indicates that the available memory space of the terminal device is insufficient (such as the available memory space is less than the preset memory threshold), the model file of the large language model is unloaded in the first memory pool.

[0071] The following takes the mobile phone 100 as an example to describe the process of dynamically adjusting the model file loaded in the first memory pool. Figure 4As shown, the method provided in the embodiment of the present application includes the following steps:

[0072] S401, the mobile phone 100 creates a first memory pool.

[0073] The mobile phone 100 sets a dedicated first memory pool for the large language model (or the preset language model). For example, the mobile phone 100 starts the operating system in response to the user's power-on operation. After the operating system is started, Figure 5 As shown, the memory management module of the mobile phone 100 creates a first memory pool (such as dmabuf) for the large language model.

[0074] The model file of the large language model is larger than the first preset data volume. For example, the model file of the large language model is larger than the first preset data volume of 1GB or 3GB. The model file of the large language model may include model parameters, execution code, etc.

[0075] S402: The mobile phone 100 obtains available memory information of the mobile phone 100 when the large language model is not running.

[0076] The available memory information is used to indicate the size of the available memory space. The available memory space is used to indicate the free part of the memory space of the mobile phone 100, which is the size of the physical memory in the mobile phone 100 that can be used by applications or processes. For example, if the total memory space size of the mobile phone 100 is 8 gigabytes (GB), and the operating system of the mobile phone 100 or other running applications or processes occupy 2GB of memory space (which can be called occupied memory space), then the available memory space of the mobile phone 100 is equal to the difference between the total memory space and the occupied memory space, that is, the available memory space is 6GB that is not used.

[0077] In some implementations, the mobile phone 100 may obtain available memory information at a certain frequency. Figure 5 As shown, the system service of the mobile phone 100 obtains the available memory information by polling.

[0078] Exemplarily, the mobile phone 100 is installed with a large language model service (or a preset language model service), which is a service process of a large language model and is used to run a large language model. Figure 6As shown, the large language model service can register memory management events with the system service during initialization (such as assigning values ​​to some variables, loading files, etc. when the large language model service is started). After the system service completes the registration of the memory management event, it periodically obtains the available memory information through polling (such as in a period of 5 minutes or 10 minutes). For example, the system service sends a memory status acquisition request to the memory management module through polling, and the memory status acquisition request is used to request to obtain the available memory information. In response to the memory status acquisition request, the memory management model performs memory pressure detection, calculates the size of the available memory space, and returns the available memory information to the system service.

[0079] S403, the mobile phone 100 adjusts the amount of data loaded from the model file in the first memory pool according to the available memory information.

[0080] The mobile phone 100 adjusts the amount of data loaded into the model file in the first memory pool according to the size of the available memory space indicated by the available memory information. For example, when the available memory information indicates that the size of the available memory space is greater than a preset memory threshold, the mobile phone 100 loads the model file of the large language model in the first memory pool. When the available memory information indicates that the size of the available memory space is equal to the preset memory threshold, the mobile phone 100 does not load or unload the model file of the large language model in the first memory pool. When the available memory information indicates that the size of the available memory space is less than the preset memory threshold, the mobile phone 100 unloads the model file of the large language model in the first memory pool.

[0081] For example, Figure 5 and Figure 6 As shown, after obtaining the available memory information, the system service determines the memory control strategy according to the size of the available memory space indicated by the available memory information, such as determining the memory control strategy for loading or unloading the model files of the large language model in the first memory pool. Further, the system service sends a control message to the large language model service, and the control message is used to indicate the memory control strategy, such as indicating the memory control strategy for loading or unloading the model files of the large language model in the first memory pool. The large language model service loads or unloads the model files of the large language model in the first memory pool according to the memory control strategy indicated by the control message.

[0082] In some implementations, in order to more accurately control the amount of data loaded into the model file in the first memory pool, the mobile phone 100 may use a plurality of preset memory thresholds as a reference condition for whether the available memory space is sufficient.

[0083] When the available memory space is greater than the first preset memory threshold, it means that the current available memory space of the mobile phone 100 is sufficient and the memory status is good, and the mobile phone 100 loads the model file of the large language model in the first memory pool. For example, the mobile phone 100 loads the model file of the large language model in the first memory pool at a preset loading rate (such as 5 megabytes per second) until the model file is completely loaded in the first memory pool, or until the available memory space of the mobile phone 100 is less than or equal to the first preset memory threshold. The first preset memory threshold can also be called a balance waterline. The first preset memory threshold can be set according to the actual application scenario or demand. For example, the first preset memory threshold can be set to 5000 megabytes (MB).

[0084] By preloading the model files of the large language model in the first memory pool when the available memory space of the mobile phone 100 is sufficient and the memory status is good, the amount of data loaded in the model files when the large language model is started can be reduced, thereby speeding up the startup of the large language model.

[0085] Exemplarily, in the process of loading the model files of the large language model in the first memory pool, the mobile phone 100 may preferentially load the model files of a preset type according to the type of the model files. For example, preferentially load the model files of the execution code class. In this way, the key files of the large language model may be preloaded in the first memory pool.

[0086] When the available memory space is less than the first preset memory threshold and greater than the second preset memory threshold, it means that the current memory pressure of the mobile phone 100 increases and the available memory space becomes smaller. In order to relieve the memory pressure of the mobile phone 100, the mobile phone 100 unloads part of the loaded model files in the preset cache pool. For example, the mobile phone 100 unloads the loaded model files in the first memory pool at a preset unloading rate (such as 5 megabytes per second) until the amount of loaded data of the model files in the first memory pool is reduced to the second preset data amount (such as 500MB, 600MB, etc.), or until the available memory space of the mobile phone 100 is greater than or equal to the first preset memory threshold. The second preset memory threshold can also be called the low memory emergency check waterline. The second preset memory threshold can be set according to the actual application scenario or demand. For example, the second preset memory threshold can be set to 900MB. The second preset data amount can also be set according to the actual application scenario or demand. For example, the second preset data amount can be set to 500MB.

[0087] By unloading the model files of the large language model in the first memory pool when the memory pressure of the mobile phone 100 increases and the available memory space becomes smaller, the memory pressure of the mobile phone 100 can be reduced. At the same time, some model files can also be reserved in the preset cache pool, so that the large language model is loaded with some model files in the preset cache pool when starting.

[0088] When the available memory space is less than or equal to the second preset memory threshold, it indicates that the current memory pressure of the mobile phone 100 is high and the available memory space is seriously insufficient, and the mobile phone 100 unloads all loaded model files in the first memory pool. For example, the mobile phone 100 deletes the loaded model files in the first memory pool at a preset unloading rate (such as 5 megabytes per second, 10 megabytes per second) until the amount of loaded data of the model files in the first memory pool is reduced to 0, or until the available memory space of the mobile phone 100 is greater than or equal to the first preset memory threshold.

[0089] When the memory pressure of the mobile phone 100 is high and the available memory space is seriously insufficient, the memory pressure of the mobile phone 100 can be relieved by unloading all loaded model files in the first memory pool.

[0090] It is understandable that the size of the memory space of the first memory pool changes dynamically with the amount of loaded data of the model file in the first memory pool. When loading the model file, if the space of the first memory pool is insufficient, such as when the first memory pool is insufficient to accommodate the model file to be loaded, the mobile phone 100 fills the first memory pool (such as filling the memory space in the available memory space into the first memory pool) so that there is enough memory space in the first memory pool to load the model file of the large language model. When unloading the model file, the mobile phone 100 can recycle the memory space in the first memory pool (such as recycling the memory space of the first memory pool to the available memory space) to increase the available memory space of the mobile phone 100. The first memory pool is the allocated memory space in the mobile phone 100. The size of the available memory space of the mobile phone 100 is inversely proportional to the size of the first memory pool. That is, the larger the first memory pool, the smaller the available memory space, and the smaller the first memory pool, the larger the available memory space.

[0091] In other implementations, in order to improve the utilization of memory space when the available memory space of the mobile phone 100 is insufficient, the mobile phone 100 can manage and control the large language model service used to run the large language model in addition to unloading the model file of the large language model in the first memory pool. For example, when the available memory space of the mobile phone 100 is insufficient, the mobile phone 100 sets the large language model service to a killable state. In this case, when the available memory space of the mobile phone 100 is less than a certain threshold or when the memory is recycled, the memory space can be released by killing the large language model service.

[0092] In one example, when the available memory space of the mobile phone 100 is less than or equal to the third preset memory threshold and greater than the second preset memory threshold, it indicates that the current memory pressure of the mobile phone 100 is large and the available memory space is insufficient. In order to alleviate the memory pressure of the mobile phone 100, the mobile phone 100 can also set the preset language model service to a killable state when the memory space occupied by the preset language model service is greater than the preset occupation threshold. In this way, the mobile phone 100 can further release memory space and alleviate memory pressure by killing large language model services.

[0093] The third preset memory threshold is between the first preset memory threshold and the second preset memory threshold. That is, the third preset memory threshold is greater than the second preset memory threshold and less than the first preset memory threshold. The third preset memory threshold can also be called a low memory kill waterline. The third preset memory threshold can be set according to actual application scenarios or requirements. For example, the third preset memory threshold can be set to 1800MB. The preset occupancy threshold can be set according to actual application scenarios or requirements. For example, the second preset memory threshold can be set to 500MB.

[0094] In another example, when the available memory space of the mobile phone 100 is less than or equal to the second preset memory threshold, the mobile phone 100 can not only unload all loaded model files in the preset cache pool, but also set the large language model service to a killable state. In this way, when the available memory space of the mobile phone 100 is seriously insufficient, regardless of whether the memory space occupied by the large language model service is greater than the preset occupancy threshold, the mobile phone 100 sets the large language model service to a killable state, thereby further freeing up memory space and alleviating memory pressure by killing the large language model service.

[0095] It is understandable that the mobile phone 100 sets the large language model service to a killable state by setting the process priority of the large language model service. The process priority of a process indicates the importance of a process. The higher the process priority of a process, the more important the process is. The lower the process priority of a process, the lower the importance of the process, and the process is in a killable state. For example, when the process priority of a process is a first value, it indicates that the process is in a resident state or an unkillable state. When the process priority of a process is a second value, it indicates that the process is in a killable state. The mobile phone 100 can set the process priority of the large language model service to the second value to put the large language model service in a killable state.

[0096] The following takes the first preset memory threshold (balanced waterline) of 4800MB, the second preset memory threshold (low memory emergency kill waterline) of 900MB, and the third preset memory threshold (low memory emergency kill waterline) of 1800MB as an example, and combines Table 1 to exemplarily introduce the memory control strategy.

[0097] Table 1 Memory management strategy corresponding to available memory space

[0098]

[0099] In the example, when the available memory space is greater than the balance waterline, the memory status is good, the system service sends a control message to the large language model service, and the large language model service executes the memory control strategy indicated by the control message: increase the amount of loaded data of the model file in the first memory pool, and set the large language model service to an unkillable state. When the available memory space is equal to the balance waterline, the memory control strategy executed by the large language model service is: set the large language model service to an unkillable state. When the available memory space is between the balance waterline and the low memory kill waterline, it indicates that the memory pressure is increasing, and the memory control strategy executed by the large language model service is: reduce the amount of loaded data of the model file in the first memory pool, and set the large language model service to an unkillable state. When the available memory space is less than or equal to the low memory kill waterline, and greater than the low memory emergency kill waterline, the memory control strategy executed by the large language model service is: reduce the amount of loaded data of the model file in the first memory pool, and set the large language model service to an unkillable state when the memory space occupied by the large language model service exceeds 500MB. When the available memory space is less than the low memory emergency kill waterline, the memory control strategy implemented by the large language model service is: all loaded data of the model files loaded in the first memory pool are unloaded, and the large language model service is set to a killable state.

[0100] In an embodiment of the present application, when the large language model is in an idle state, the mobile phone 100 can load or unload the model file of the large language model in the first memory pool according to the memory status (such as available memory space). When the available memory space is sufficient, the model file is pre-loaded in the preset cache pool, so that the large language model has some resident model files in the first memory pool, thereby improving the performance of the large language model when it is started. When the available memory space is insufficient, the model file of the large language model in the preset cache pool is unloaded, and the memory space of the first memory pool is returned to the available memory space of the mobile phone 100. In this way, the amount of data loaded by the model file in the first memory pool is dynamically adjusted when the large language model is idle, thereby improving the startup speed of the large language model.

[0101] In the above embodiment, the mobile phone 100 performs a memory control strategy at a certain frequency. In some embodiments, the mobile phone 100 can also immediately perform a memory control strategy for a large language model when starting the second application to reduce the impact of preloading the model file in the first memory pool on the startup of the second application.

[0102] Exemplarily, if the system service of the mobile phone 100 detects that the second application is started, the system service immediately determines the memory control strategy according to the available memory space of the mobile phone 100, and sends a control message to the large language model service, indicating the determined memory control strategy through the control message. The large language model service executes the memory control strategy of the large language model according to the control message, such as loading or unloading the model file of the large language model in the first memory pool.

[0103] It is understandable that the second application is a large memory application with a large memory space demand. For example, the second application is a camera application, a game application, etc. When the system service detects that the second application is started, it can determine whether the second application is a large memory application. For example, a preset application list may be provided in the mobile phone 100. The preset application list is used to record large memory applications with large memory space demands. If the second application is in the preset application list, it can be considered that the second application is a large memory application, and the system service immediately sends a control message to the large language model service so that the mobile phone 100 can give priority to providing sufficient available memory space for the second application.

[0104] In the above, the process of dynamically adjusting the amount of data loaded by the model file in the first memory pool when the large language model is not running is exemplified by taking the terminal device as the mobile phone 100. The process of starting the large language model by the terminal device is further described below.

[0105] In some implementations, if the large language model is not running, and part of the model files of the large language model are loaded in the first memory pool, the terminal device pre-fills the first memory pool in response to the first user instruction to prepare memory space for loading the model files of the large language model in advance, further improving the startup speed of the large language model. Furthermore, the terminal device adds the model files of the large language model to the pre-filled first memory pool. After all the model files are loaded, the terminal device runs the large language model and generates the semantic reasoning result of the first user instruction through the large language model.

[0106] The following is an example of a process in which a terminal device starts a large language model when a part of the model files of the large language model are loaded in the first memory pool. Figure 7As shown, the terminal device calls the large language model service (i.e., enters the semantic reasoning entrance) in response to the first user instruction. The terminal device further calculates the memory requirements of the large language model, and performs pre-filling of the first memory pool according to the memory requirements of the large language model. At the same time, the terminal device determines whether the available memory space is insufficient according to the memory requirements of the large language model. If the available memory space is insufficient, the available memory space is expanded through a memory recycling operation. Furthermore, the terminal device loads the remaining unloaded model files in the pre-filled first memory pool to generate the semantic reasoning result of the first user instruction through the large language model.

[0107] In the following embodiments, the terminal device is also taken as a mobile phone 100, and the first memory pool is loaded with some model files of the large language model as an example to further introduce the startup process of the large language model. Figure 8 As shown, the method provided in the embodiment of the present application includes the following steps:

[0108] S801, the mobile phone 100 determines the data amount of the unloaded model file in response to the first user instruction.

[0109] The first user instruction is used to trigger the voice assistant function of the first application. The first user instruction can be a voice instruction, a user operation instruction, etc. In the case where the first user instruction is a voice instruction, the mobile phone 100 can also recognize the first user instruction from the voice data as text data, and display the text data of the first user instruction in the application interface of the first application.

[0110] The first application is used to provide a voice assistant function through a large language model. For example, the first application is a voice assistant application, a smart device control application, etc. For example, the first application can obtain a semantic reasoning result corresponding to the first user instruction through a large language model, and further implement voice assistant functions such as voice dialogue and intelligent question and answer based on the semantic reasoning result.

[0111] In order to run a large language model, the mobile phone 100 needs to load all model files of the large language model into the memory (such as the first memory pool). In response to the first user instruction, the mobile phone 100 calculates the data volume of the model files not loaded in the first memory pool.

[0112] For example, the first user instruction is "generate birthday video". Fig. 9As shown, the first application of the mobile phone 100 receives the first user instruction of "generate birthday video" reported by the underlying module (such as audio driver). In response to the first user instruction of "generate birthday video", the first application notifies the large language model service to start the large language model. The large language model service further calculates the memory requirement of the large language model, that is, calculates the data amount of the model files that are not loaded in the preset cache pool. For ease of description, the data amount of the unloaded model files can be referred to as the first data amount. The first data amount is equal to the difference between the total data amount of the model file and the second data amount. The second data amount is the data amount of the model files that have been loaded in the first memory pool. The second data amount can also be called a resident amount.

[0113] S802, the mobile phone 100 pre-fills a first memory pool according to the first data volume.

[0114] After determining the data volume of the unloaded model file, that is, after determining the first data volume, the mobile phone 100 pre-fills the first memory pool according to the first data volume. For example, the mobile phone 100 fills part of the memory space of the available memory space into the first memory pool, so that the first memory pool has enough memory space to load the model file of the first data volume.

[0115] It is understandable that after part of the available memory space is filled into the first memory pool, this part of the memory space is the allocated memory space and no longer belongs to the available memory space, and the available memory space of the mobile phone 100 is reduced. The pre-filling of the first memory pool by the mobile phone 100 can be understood as the mobile phone 100 filling the first memory pool without receiving a memory application request from the large language model service. Compared with the solution of filling the first memory pool in response to a memory application request from the large language model service, pre-filling the first memory pool can prepare memory space for loading the model file of the large language model in advance, further improving the startup speed of the large language model.

[0116] In some implementations, the available memory space of the mobile phone 100 may be insufficient. In this case, the mobile phone 100 may also expand the available memory space by memory recycling so that the available memory space can provide enough memory space to fill the first memory pool. Specifically, after determining the first data amount, the mobile phone 100 determines whether the available memory space is less than the first data amount. If the available memory space is less than the first data amount, it means that the available memory space may be insufficient, and the mobile phone 100 recycles the allocated memory space to expand the available memory space.

[0117] For example, Fig. 9As shown, after calculating the memory demand (i.e., the first data volume), the large language model service pre-requests memory supply from the system service. If the available memory space is less than the first data volume, the system service instructs the memory management module to reclaim memory. The memory management module can quickly reclaim the allocated memory space by compressing, dumping, and killing processes loaded in the memory to expand the available memory space. The system service further instructs the memory management module to pre-fill the first memory pool through a pre-fill indication. The memory management module pre-fills the first memory pool in response to the pre-fill indication.

[0118] In the embodiment of the present application, after detecting the memory demand for loading a large language model, the mobile phone 100 can reclaim memory on demand and pre-fill the first memory pool dedicated to the large language model to prepare in advance the memory space for loading the model file, respond to the subsequent memory requests corresponding to the large language model, reduce the impact of the startup of the large language model on the performance of the operating system, and improve the startup speed of the large language model.

[0119] S803, the mobile phone 100 loads the model file in the first memory pool.

[0120] In response to the memory application request of the large language model service, the mobile phone 100 loads the model file of the large language model into the first memory pool. Fig. 9 As shown, after the first memory pool is pre-filled, the large language model service sends a memory application request to the memory management module, and requests the memory management module to load the memory space of the model file through the memory application request. In response to the memory application request, the memory management module loads the model file of the large language model in the pre-filled first memory pool.

[0121] S804: When all the model files are loaded, the mobile phone 100 runs the large language model to generate a semantic reasoning result of the first user instruction through the large language model.

[0122] After all the model files are loaded into the first memory pool, the large language model service of the mobile phone 100 runs the large language model based on the model files loaded into the first memory pool. For example, the large language model service inputs the text data of the first user instruction into the large language model, and uses the large language model to perform semantic analysis, semantic reasoning and other operations on the first user instruction to obtain the semantic reasoning result of the first user instruction. The mobile phone 100 can also prompt the user with the semantic reasoning result.

[0123] For example, Fig. 9As shown, after the memory management module of the mobile phone 100 loads all the model files in the first memory pool, it returns a memory application success response message to the large language model service. In response to the memory application success response message, the large language model service runs the large language model and uses the large language model to obtain the semantic reasoning result of the first user instruction. After obtaining the semantic reasoning result of the first user instruction, the large language model service returns the semantic reasoning result to the first application. The first application can further send a prompt message to the user based on the semantic reasoning result. For example, the first user instruction is "generate a birthday video", and the semantic reasoning result can be an image related to the birthday theme. The first application generates a prompt message based on the semantic reasoning result, and the prompt message triggers the mobile phone 100 to display a preview window of images related to the birthday theme to the user in the application interface of the first application.

[0124] In the above embodiment, the terminal device is a mobile phone 100 as an example, and the startup process of the large language model is introduced when the first memory pool is pre-loaded with some model files of the large language model. In other implementations, if the large language model is not running, all model files of the large language model are loaded in the first memory pool, and the terminal device responds to the first user instruction and directly runs the large language model, which can greatly improve the startup speed of the large language model.

[0125] The method provided in the embodiment of the present application pre-loads at least part of the model files of the large language model into the first memory pool, so that the model files of the large language model are partially resident in the first memory pool, thereby making full use of memory resources and optimizing the startup time of the large language model. By pre-filling the first memory pool, all model files can be loaded into the first memory pool more quickly, so that the large language model can continue to perform speech content reasoning. In addition, when there is sufficient available memory space, the model files do not need to be repeatedly loaded into the first memory pool, and the performance requirements for storage devices such as general flash storage are lower.

[0126] It is understandable that the solution provided in the embodiment of the present application is introduced by taking a large language model as an example. The solution provided in the embodiment of the present application can also be applied to other large models.

[0127] In some other embodiments of the present application, a terminal device is provided, including: a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, the terminal device can perform each function or step in the above method embodiment. Of course, the terminal device can also include other hardware structures. For example, the terminal device also includes hardware structures such as sensors and communication modules. The structure of the terminal device can refer to Figure 2 The structure of the mobile phone 100 is shown.

[0128] The embodiment of the present application also provides a chip system, which is applied to a terminal device. The chip system includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected by lines. For example, the interface circuit can be used to receive signals from other devices (such as memory). For another example, the interface circuit can be used to send signals to other devices (such as processors). Exemplarily, the interface circuit can read a computer program stored in the memory and send the computer program to the processor. When the computer program is executed by the processor, the terminal device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiment of the present application.

[0129] An embodiment of the present application also provides a computer-readable storage medium, which includes a computer program. When the computer program runs on the above-mentioned terminal device, the terminal device executes each function or step in the above-mentioned method embodiment.

[0130] The present application also provides a computer program product, which includes a computer program, and when the computer program is run on a computer, the computer executes each function or step in the above method embodiment. For example, the computer may be the above terminal device.

[0131] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0132] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0134] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0136] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered within the protection scope of the present application.

Claims

1. A model loading method, characterized in that: Applied to a terminal device, wherein a first application is installed in the terminal device, and the first application is used to provide a voice assistant function through a preset language model; a first memory pool is created in the terminal device, and the first memory pool is used to load a model file of the preset language model, and the total data volume of the model file of the preset language model is greater than a first preset data volume; the method comprises: When the preset language model is not running, loading at least part of the model files of the preset language model into the first memory pool; receiving a first user instruction, where the first user instruction is used to trigger a voice assistant function of the first application; In response to the first user instruction, when all model files of the preset language model are loaded in the first memory pool, the preset language model is run to generate a semantic reasoning result of the first user instruction through the preset language model.

2. The method according to claim 1, characterized in that When the preset language model is not running, loading at least part of the model files of the preset language model into the first memory pool includes: In a case where the preset language model is not run, obtaining available memory information of the terminal device, wherein the available memory information is used to indicate the size of the available memory space; If the available memory space is greater than a first preset memory threshold, at least part of the model files of the preset language model are loaded into the first memory pool, and the size of the available memory space is inversely proportional to the size of the first memory pool.

3. The method according to claim 2, characterized in that The method further comprises: If the available memory space is less than a first preset memory threshold and greater than a second preset memory threshold, some loaded model files are unloaded in the first memory pool; wherein the second preset memory threshold is greater than the first preset memory threshold.

4. The method according to claim 3, characterized in that The method further comprises: If the memory space occupied by the preset language model service is greater than the preset occupancy threshold, and the available memory space is less than or equal to the third preset memory threshold, the preset language model service is set to a killable state, wherein the preset language model service is used to run the preset language model, and the third preset memory threshold is greater than the second preset memory threshold and less than the first preset memory threshold.

5. The method according to any one of claims 2 to 4, characterized in that: The method further comprises: If the available memory space is less than or equal to a second preset memory threshold, all loaded model files are unloaded from the first memory pool.

6. The method according to claim 5, characterized in that The method further comprises: The preset language model service is set to a killable state, wherein the preset language model service is used to run the preset language model.

7. The method according to any one of claims 1 to 6, characterized in that In response to the first user instruction, when all model files of the preset language model are loaded in the first memory pool, the preset language model is run, and a semantic reasoning result of the first user instruction is generated by the preset language model, including: In response to the first user instruction, if there is an unloaded model file, pre-filling the first memory pool according to a first data amount, wherein the first data amount is the data amount of the unloaded model file; In response to a memory application request of a preset language model service, loading the unloaded model file into a pre-filled first memory pool; After all model files of the preset language model are loaded into the first memory pool, the preset language model is run to generate a semantic reasoning result of the first user instruction through the preset language model.

8. The method according to claim 7, characterized in that Before pre-filling the first memory pool, the method further includes: If the available memory space of the terminal device is less than the first data amount, the allocated memory space of the terminal device is recycled.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: After the terminal device is started, a first memory pool of the preset language model is created.

10. A terminal device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, and when the processor runs the computer program, the terminal device executes the method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a terminal device, the terminal device executes the method according to any one of claims 1 to 9.

12. A computer program product, characterized in that The method comprises a computer program, which, when executed by a terminal device, enables the terminal device to execute the method according to any one of claims 1 to 9.

Images