Model loading method and terminal device

Abstract

The application provides a model loading method and a terminal device, and relates to the technical field of terminals. The method comprises the following steps: loading at least part of model files of a preset language model in a first memory pool in the case that the preset language model is not running. A first user instruction is received, and the first user instruction is used for triggering a voice assistant function of a first application. In response to the first user instruction, in the case that all model files of the preset language model are loaded in the first memory pool, the preset language model is run, and a semantic reasoning result of the first user instruction is generated through the preset language model.

Description

[0001] The present application claims priority to the Chinese Patent Application No. 202410063464.0, filed on January 16, 2024, and entitled "A large model loading speed optimization method", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminal devices, and in particular to a model loading method and a terminal device. BACKGROUND

[0003] In order to provide better user experience, the functions provided by terminal devices are increasingly diversified. For example, as a new type of function of terminal devices, voice assistant functions can provide voice interaction, voice control and other services. When providing voice assistant functions, terminal devices need to load model files of corresponding function models to provide corresponding functions through the corresponding function models.

[0004] However, for large models, the model files of large models are relatively large, and the data volume can reach the order of gigabytes. When using the large model function of the terminal device, the model files of the large model are loaded slowly, and the large model takes a long time to start, which affects the user experience. SUMMARY

[0005] The present application provides a model loading method and a terminal device for improving the loading rate of large models, shortening the starting time of large models, and improving the user experience.

[0006] To achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, the present application provides a model loading method applied to a terminal device, wherein the terminal device is installed with a first application, and the first application is used to provide a voice assistant function through a preset language model. A first memory pool is created in the terminal device, and the first memory pool is used to load model files of the preset language model. The total data volume of the model files of the preset language model is greater than a first preset data volume. In the case where the preset language model is not running, at least part of the model files of the preset language model are loaded in the first memory pool. A first user instruction is received, and the first user instruction is used to trigger the voice assistant function of the first application. In response to the first user instruction, in the case where the entire model files of the preset language model are loaded in the first memory pool, the preset language model is run, and a semantic reasoning result of the first user instruction is generated through the preset language model.

[0008] In this way, by setting the first memory pool for the preset language model, the frequency of the terminal device applying for and releasing memory space for the large language model can be reduced, the fragmentation of the memory space can be reduced, and the utilization rate of the memory space can be improved. When the large language model is idle, the terminal device preloads at least part of the model files of the large language model in the first memory pool, which can reduce the memory pressure when the large language model starts and accelerate the starting speed of the large language model.

[0009] In a possible implementation of the first aspect, when the preset language model is not running, available memory information of the terminal device is acquired, the available memory information being used to indicate the size of the available memory space. If the available memory space is greater than a first preset memory threshold, at least part of the model files of the preset language model are loaded in the first memory pool, and the size of the available memory space is inversely proportional to the size of the first memory pool.

[0010] In this implementation, the available memory space of the terminal device is greater than the first preset memory threshold, indicating that the available memory space of the terminal device is sufficient and the memory state is good. In this case, the model files of the large language model are preloaded in the first memory pool, which has little effect on the memory pressure of the terminal device and can also reduce the data amount of the model file loading when the large language model starts and accelerate the starting speed of the large language model.

[0011] In another possible implementation of the first aspect, if the available memory space is less than the first preset memory threshold and greater than a second preset memory threshold, part of the loaded model files are unloaded in the first memory pool, and the second preset memory threshold is greater than the first preset memory threshold.

[0012] In this implementation, the available memory space of the terminal device is less than the first preset memory threshold and greater than the second preset memory threshold, indicating that the memory pressure of the terminal device increases and the available memory space becomes smaller. In this case, the model files of the large language model are unloaded in the first memory pool, which can reduce the memory pressure of the terminal device. At the same time, part of the model files can be reserved in the preset cache pool, so that part of the model files of the large language model are loaded in the preset cache pool when the large language model starts.

[0013] In another possible implementation of the first aspect, if the memory space occupied by the preset language model service is greater than a preset occupation threshold and the available memory space is less than or equal to a third preset memory threshold, the preset language model service is set to a killable state, the preset language model service is used to run the preset language model, the third preset memory threshold is greater than the second preset memory threshold and less than the first preset memory threshold.

[0014] In the implementation, the available memory space of the terminal device is between the third preset memory threshold and the second preset memory threshold, which indicates that the memory pressure of the terminal device is relatively large, and the available memory space is insufficient. In order to relieve the memory pressure of the terminal device, the preset language model service can also be set to the killable state when the memory space occupied by the preset language model service is relatively large. In this way, the memory space can be further released by killing the large language model service, and the memory pressure can be relieved.

[0015] In a possible implementation of the first aspect, if the available memory space is less than or equal to the second preset memory threshold, all loaded model files are unloaded in the first memory pool. In this implementation, if the available memory space is less than or equal to the second preset memory threshold, it indicates that the memory pressure of the terminal device is large, and the available memory space is severely insufficient. In this case, all loaded model files are unloaded in the first memory pool, which can relieve the memory pressure of the terminal device.

[0016] In a possible implementation of the first aspect, the preset language model service can also be set to the killable state. In this implementation, in the case where the available memory space of the terminal device is severely insufficient, the terminal device sets the large language model service to the killable state regardless of whether the memory space occupied by the large language model service is greater than the preset occupation threshold, so as to further release the memory space by killing the large language model service and relieve the memory pressure.

[0017] In a possible implementation of the first aspect, in response to the first user instruction, if there is an unloaded model file, the first memory pool is pre-filled according to a first data amount, and the first data amount is a data amount of the unloaded model file. In response to a memory application request of the preset language model service, the unloaded model file is loaded in the pre-filled first memory pool. After loading all model files of the preset language model in the first memory pool, the preset language model is run to generate a semantic reasoning result of the first user instruction by the preset language model.

[0018] In this implementation, the terminal device fills the first memory pool without receiving the memory application request of the large language model service. In this way, the memory space can be prepared in advance for the model file loading of the large language model, and the startup speed of the large language model can be further improved.

[0019] In a possible implementation of the first aspect, the first memory pool is created after the terminal device is started. In this implementation, the terminal device creates a dedicated memory pool for the large language model when it is started, so that the terminal device does not need to compete for memory resources with other processes during loading of the large language model, and the frequency of application and release of memory space by the terminal device for the large language model is reduced, the fragmentation of memory space is reduced, and the utilization of memory space is improved.

[0020] In a possible implementation of the first aspect, the first memory pool of the preset language model is created after the terminal device is started. In this implementation, the terminal device creates a dedicated memory pool for the large language model when it is started, so that the terminal device does not need to compete for memory resources with other processes during loading of the large language model, and the frequency of application and release of memory space by the terminal device for the large language model is reduced, the fragmentation of memory space is reduced, and the utilization of memory space is improved.

[0021] In a possible implementation of the first aspect, the first memory pool of the preset language model is created after the terminal device is started. In this implementation, the terminal device creates a dedicated memory pool for the large language model when it is started, so that the terminal device does not need to compete for memory resources with other processes during loading of the large language model, and the frequency of application and release of memory space by the terminal device for the large language model is reduced, the fragmentation of memory space is reduced, and the utilization of memory space is improved.

[0022] In a possible implementation of the first aspect, the first memory pool of the preset language model is created after the terminal device is started. In this implementation, the terminal device creates a dedicated memory pool for the large language model when it is started, so that the terminal device does not need to compete for memory resources with other processes during loading of the large language model, and the frequency of application and release of memory space by the terminal device for the large language model is reduced, the fragmentation of memory space is reduced, and the utilization of memory space is improved.

[0023] In a possible implementation of the first aspect, the first memory pool of the preset language model is created after the terminal device is started. In this implementation, the terminal device creates a dedicated memory pool for the large language model when it is started, so that the terminal device does not need to compete for memory resources with other processes during loading of the large language model, and the frequency of application and release of memory space by the terminal device for the large language model is reduced, the fragmentation of memory space is reduced, and the utilization of memory space is improved.

[0024] In a possible implementation of the first aspect, the first memory pool of the preset language model is created after the terminal device is started. In this implementation, the terminal device creates a dedicated memory pool for the large language model when it is started, so that the terminal device does not need to compete for memory resources with other processes during loading of the large language model, and the frequency of application and release of memory space by the terminal device for the large language model is reduced, the fragmentation of memory space is reduced, and the utilization of memory space is improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A schematic diagram of an interface of a terminal device provided by an embodiment of the present application;

[0026] Figure 2A hardware structure block diagram of a terminal device provided in an embodiment of this application;

[0027] Figure 3 A software structure block diagram of a terminal device provided in an embodiment of this application;

[0028] Figure 4 A flowchart illustrating the dynamic loading or unloading of model files is provided in this application embodiment;

[0029] Figure 5 A flowchart illustrating another dynamic loading or unloading of model files provided in this application embodiment;

[0030] Figure 6 A communication diagram of a software module provided in an embodiment of this application;

[0031] Figure 7 A flowchart for starting a large language model is provided as an embodiment of this application;

[0032] Figure 8 Another flowchart for launching a large language model provided in this application embodiment;

[0033] Figure 9 This is a communication diagram of another software module provided in an embodiment of this application. Detailed Implementation

[0034] To provide a better user experience, terminal devices are equipped with voice assistant applications (or voice assistants). Voice assistants can recognize user voice commands and execute corresponding operations. For example, voice assistants can provide services such as voice dialogue, voice broadcasting, and launching applications. Voice assistants can reduce user reliance on input devices such as keyboards and mice, providing a more convenient interaction method.

[0035] For example, such as Figure 1 As shown in (1), the terminal device displays a desktop interface. With the desktop interface displayed, the terminal device receives the voice command "What's the weather like today?". Figure 1 As shown in (2), in response to the voice command, the terminal device pops up a dialog window of the voice assistant application on the desktop interface and displays the weather forecast content "Today's weather is sunny, temperature 0℃-10℃" in the dialog window. At the same time, the terminal device can also broadcast "Today's weather is sunny, temperature 0℃-10℃" via voice.

[0036] A voice assistant application of a terminal device provides a voice assistant function through a large language model (LLM). When the voice assistant provides the voice assistant function through the large language model, the terminal device needs to load model files of the large language model. After loading all the model files of the large language model, the terminal device can run the large language model. With the continuous improvement of the voice assistant function, the model files of the large language model are also getting larger and larger. The data volume of the model files of the large language model can reach the level of gigabytes. Accordingly, the terminal device loads the model files of the large language model at a slower speed, and the voice assistant application takes a longer time to respond to a user instruction, affecting the user experience.

[0037] In addition, due to the large size of the model files of the large language model, the terminal device has a large pressure on allocating memory space for the large language model. When the terminal device loads the model files of the large language model, other applications running in the background may be killed due to insufficient available memory space of the terminal device, so that the terminal device has sufficient available memory space. However, after the terminal device kills the applications running in the background, the user data of the applications may be lost, affecting the user experience.

[0038] Therefore, in the method provided by the embodiments of the present application, a first memory pool dedicated to loading the model files of the large language model can be set in the terminal device. In the case where the large language model is not running, the terminal device preloads at least part of the model files of the large language model in the first memory pool. When the terminal device receives a user instruction triggering the voice assistant function, the terminal device responds to the user instruction, and in the case where all the model files are loaded in the first memory pool, runs a preset language model, and generates a semantic reasoning result of the user instruction through the preset language model.

[0039] In this way, by setting the first memory pool for the preset language model, the frequency of applying and releasing memory space for the large language model by the terminal device can be reduced, the fragmentation of the memory space can be reduced, and the utilization rate of the memory space can be improved. When the large language model is idle, the terminal device preloads at least part of the model files of the large language model in the first memory pool, which can reduce the memory pressure when the large language model starts and accelerate the starting speed of the large language model. In addition, when the memory pressure is small when the large language model starts, the possibility of killing the background applications or processes by the terminal device is also small, which can reduce the probability of killing the applications or processes.

[0040] Exemplarily, the terminal device described in the embodiments of the present application can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) \ virtual reality (VR) device, a media player, a wearable device, and the like. The embodiments of the present application do not specially limit the specific form of the terminal device.

[0041] In the embodiments of the present application, the terminal device is taken as a mobile phone 100, and the hardware structure of the terminal device is introduced through the mobile phone 100. As shown in Figure 2 The mobile phone 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, and the like.

[0042] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), a driving processor, and the like. Different processing units can be independent devices, or can be integrated in one or more processors. The processor 110 can be the nerve center and command center of the mobile phone 100. The processor 110 can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching instructions and executing instructions.

[0043] The processor 110 can also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can hold instructions or data that the processor 110 has recently used or is likely to use again. If the processor 110 needs to use the instructions or data again, it can be retrieved directly from the memory. In this way, repeated accesses can be avoided, the latency of the processor 110 can be reduced, and the efficiency of the system can be improved.

[0044] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to extend the storage capacity of the mobile phone 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, the mobile phone 100 stores music, video, and other files in the external memory card. In some embodiments, the mobile phone 100 stores the model file of the large language model in the external memory card. When the large language model is started, the mobile phone 100 loads the model file of the large language model into the memory.

[0045] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the mobile phone 100 by running the instructions stored in the internal memory 121. For example, in the embodiments of the present application, the processor 110 can execute the instructions stored in the internal memory 121, and the internal memory 121 can include a storage program area and a storage data area.

[0046] The storage program area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), a configuration file of the motor 191, etc. The storage data area can store data (such as audio data, a phonebook, etc.) created during the use of the mobile phone 100, etc. In addition, the internal memory 121 can include a high-speed random access memory and can also include a non-volatile memory. For example, the internal memory 121 can include at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0047] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives the input of the battery 142 and / or the charging management module 140 to supply power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160, etc. In some embodiments, the power management module 141 and the charging management module 140 can also be arranged in the same device.

[0048] The wireless communication function of the mobile phone 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, the baseband processor, and the like. In some embodiments, the antenna 1 and the mobile communication module 150 of the mobile phone 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the mobile phone 100 can communicate with the network and other devices through wireless communication technology.

[0049] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile phone 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0050] The mobile communication module 150 can provide a solution applied to the mobile phone 100 including 2G / 3G / 4G / 5G and the like wireless communication. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and perform filtering, amplification, and the like on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation.

[0051] The wireless communication module 160 can provide a solution applied to the mobile phone 100 including wireless local area networks (WLAN) such as Wi-Fi network, Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), and the like wireless communication.

[0052] The mobile phone 100 can implement an audio function through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, the application processor, and the like. For example, music playing, recording, and the like. In some implementations, the mobile phone 100 can collect voice instructions of a user through the microphone 170C.

[0053] The sensor module 180 can include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a Hall sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor, and the like. The mobile phone 100 can collect various data through the sensor module 180.

[0054] The mobile phone 100 implements display functions through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0055] The display screen 194 is used to display images, videos, etc. For example, the mobile phone 100 displays an application interface through the display screen 194. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flex light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light emitting diode (QLED), etc.

[0056] In some implementations, the above-mentioned touch sensor can be arranged in the display screen 194, and a touch screen composed of the touch sensor and the display panel is also called a “touch screen”. The touch sensor is also called a “touch panel”, which is used to detect touch operations such as click operations, sliding operations, etc. acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the touch event type. The mobile phone 100 can provide visual output related to the touch operation through the display screen 194.

[0057] It can be understood that the interface connection relationship between the modules shown in the embodiments is only illustrative and does not constitute a structural limitation of the terminal device. In other embodiments, the terminal device can include more or fewer modules than those provided in the above embodiments, and the modules can be connected by different interfaces or a combination of different interface connection manners.

[0058] The software system of the terminal device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiments of the present application take the terminal device as the above-mentioned mobile phone 100, and the software system of the terminal device adopts a layered architecture of an Android system as an example to exemplarily illustrate the software structure of the terminal device.

[0059] A layered architecture divides software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system can include an application layer, an application framework layer, an Android runtime and system library, a hardware abstraction layer (HAL), and a kernel layer.

[0060] The application layer can include a series of application packages. For example, the application packages can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, voice assistant, large language model service (LLMService), etc. application, the embodiments of the present application do not make any limitation thereto. Among them, the voice assistant is used to provide voice assistant function. The large language model service is used to run the large language model. The voice assistant communicates with the large language model service and delivers user instructions to the large language model service. The large language model service is used to run the large language model, analyze, infer, etc. the user instruction through the large language model, obtain the semantic inference result corresponding to the user instruction, and return the semantic inference result obtained by the large language model to the voice assistant.

[0061] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications of the application layer. The application framework layer includes some pre-defined functions. For example, the application framework layer can include system service, window manager, content provider, view system, phone manager, resource manager, and notification manager, etc., and the embodiments of the present application do not make any limitation thereto. Among them, the system service is used to start and manage some service processes of the operating system, such as starting the window manager, content provider, etc. The system service is also used to obtain the memory state, such as the system service obtaining the available memory information of the total memory space through the iaware program.

[0062] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core library contains two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform the management of object life cycle, stack management, thread management, security and exception management, and garbage collection, etc.

[0063] The system library can include a plurality of functional modules. For example, a surface manager, media libraries, a three-dimensional graphics processing library (e.g., OpenGL ES), a two-dimensional graphics engine (e.g., SGL), and the like.

[0064] The HAL layer encapsulates Linux kernel drivers, provides an interface upward, and shields the implementation details of the underlying hardware. For example, the HAL layer can include a camera HAL, a Wi-Fi HAL, a Bluetooth HAL, and the like.

[0065] The kernel layer is a layer between hardware and software. The kernel layer includes drivers and system services. The drivers include at least a memory management module, a display driver, a camera driver, an audio driver, a sensor driver, and the like. The memory management module is configured to manage memory space, such as allocating and recycling memory space.

[0066] It can be understood that the software modules included in each software layer in the software structure illustrated in the embodiments are only illustrative and do not limit the software structure of the terminal device. In other embodiments, the terminal device can include more or fewer software modules than those provided in the above embodiments, or the software layers of some software modules can be different from the software layers provided in the above embodiments. For example, the large language model described above can be located in the application framework layer.

[0067] The following embodiments of the present application take the terminal device as the above-mentioned mobile phone 100, the software system of the mobile phone 100 adopts the layered architecture of the above-mentioned Android system as an example, and illustrate the working process of the software and hardware of the mobile phone 100 in combination with the software architecture shown in the above-mentioned Android system. Figure 3

[0068] ​The mobile phone 100 collects a voice instruction of a user through the microphone 170C, and the voice instruction is used to trigger a voice assistant function of a voice assistant application. The voice assistant application of the mobile phone 100 pulls up a large language model service in response to the voice instruction. The large language model service instructs a memory management module of a kernel layer to load a model file of the large language model through a system service. After the model file of the large language model is loaded, the large language model service runs the large language model, and performs semantic analysis and voice reasoning on the received voice instruction through the large language model to obtain a semantic reasoning result corresponding to the voice instruction. For example, when the voice instruction includes a first preset instruction “What day is today?”, the semantic reasoning result corresponding to the voice instruction is “Friday” through the large language model. Further, the large language model service returns the semantic reasoning result to the voice assistant application. The voice assistant application further presents the semantic reasoning result to the user, such as playing the semantic reasoning result through the loudspeaker 170A or displaying the semantic reasoning result through the display screen 194.

[0069] As described above, the first memory pool for loading the model file of the large language model (or referred to as a preset language model) is created in the terminal device. The first memory pool is a dedicated memory pool set for the large language model (or referred to as the preset language model). The terminal device can create the memory pool for loading the model file of the large language model in advance, so that the memory resource does not need to be competed with other processes in the process of loading the large language model. In this way, when the large language model is started, the terminal device directly uses the memory space of the first memory pool, which can improve the loading rate of the large language model. In addition, the first memory pool can also reduce the frequency of applying and releasing the memory space for the large language model, reduce the fragmentation of the memory space, and improve the utilization rate of the memory space.

[0070] In order to further improve the loading rate of the large language model, the terminal device can also load the model file of the large language model in the first memory pool in advance without running the large language model. In some embodiments, the terminal device can dynamically load or unload the model file of the large language model in the first memory pool according to the available memory information of the terminal device, to realize dynamic adjustment of the data amount of the model file. For example, when the available memory information indicates that the available memory space of the terminal device is sufficient (such as the available memory space is greater than a preset memory threshold), the model file of the large language model is loaded in the first memory pool. When the available memory information indicates that the available memory space of the terminal device is insufficient (such as the available memory space is less than the preset memory threshold), the model file of the large language model is unloaded in the first memory pool.

[0071] The process of dynamically adjusting the model file loaded in the first memory pool is exemplarily introduced below taking the mobile phone 100 as an example. As shown in FIG. 6, the mobile phone 100 collects a voice instruction of a user through the microphone 170C, and the voice instruction is used to trigger a voice assistant function of a voice assistant application. The voice assistant application of the mobile phone 100 pulls up a large language model service in response to the voice instruction. The large language model service instructs a memory management module of a kernel layer to load a model file of the large language model through a system service. After the model file of the large language model is loaded, the large language model service runs the large language model, and performs semantic analysis and voice reasoning on the received voice instruction through the large language model to obtain a semantic reasoning result corresponding to the voice instruction. For example, when the voice instruction includes a first preset instruction “What day is today?”, the semantic reasoning result corresponding to the voice instruction is “Friday” through the large language model. Further, the large language model service returns the semantic reasoning result to the voice assistant application. The voice assistant application further presents the semantic reasoning result to the user, such as playing the semantic reasoning result through the loudspeaker 170A or displaying the semantic reasoning result through the display screen 194. Figure 4As shown, the method provided by the embodiments of the present application includes the following steps:

[0072] S401, the mobile phone 100 creates a first memory pool.

[0073] The mobile phone 100 sets a dedicated first memory pool for a large language model (or a preset language model). For example, the mobile phone 100 starts an operating system in response to a user's booting operation. After the operating system is started, the mobile phone 100 creates the first memory pool for the large language model. Figure 5 As shown, the memory management module of the mobile phone 100 creates a first memory pool (such as dmabuf) for the large language model.

[0074] The model file of the large language model is greater than the first preset data amount. For example, the model file of the large language model is greater than 1 GB or 3 GB of the first preset data amount. The model file of the large language model can include model parameters, execution code, etc.

[0075] S402, the mobile phone 100 acquires available memory information of the mobile phone 100 without running the large language model.

[0076] The available memory information is used to indicate the size of the available memory space. The available memory space is used to represent the idle part of the memory space of the mobile phone 100, and is the size of the physical memory that can be used by the application program or process of the mobile phone 100. For example, if the total memory space size of the mobile phone 100 is 8 gigabytes (GB), and the operating system or other running application programs or processes of the mobile phone 100 occupy 2 GB of memory space (which can be referred to as occupied memory space), then the available memory space of the mobile phone 100 is equal to the difference between the total memory space and the occupied memory space, that is, the available memory space is 6 GB that is not used.

[0077] In some implementations, the mobile phone 100 can acquire the available memory information at a certain frequency. For example, as shown in Figure 5 The system service of the mobile phone 100 acquires the available memory information through polling.

[0078] For example, the mobile phone 100 is installed with a large language model service (or a preset language model service), and the large language model service is a service process of the large language model and is used to run the large language model. For example, Figure 6As shown, the large language model service can register a memory management event to the system service during initialization (e.g., assigning values to some variables, loading files, etc. when the large language model service starts). After the memory management event is registered, the system service periodically (e.g., with a period of 5 minutes, 10 minutes, etc.) acquires available memory information. For example, the system service sends a memory state acquisition request to the memory management module through polling, and the memory state acquisition request is used to request the available memory information. In response to the memory state acquisition request, the memory management module performs memory pressure detection, calculates the size of the available memory space, and returns the available memory information to the system service.

[0079] S403, the mobile phone 100 adjusts the amount of data loaded by the model file in the first memory pool according to the available memory information.

[0080] The mobile phone 100 adjusts the amount of data loaded by the model file in the first memory pool according to the size of the available memory space indicated by the available memory information. For example, when the size of the available memory space indicated by the available memory information is greater than the preset memory threshold, the mobile phone 100 loads the model file of the large language model in the first memory pool. When the size of the available memory space indicated by the available memory information is equal to the preset memory threshold, the mobile phone 100 does not load or unload the model file of the large language model in the first memory pool. When the size of the available memory space indicated by the available memory information is less than the preset memory threshold, the mobile phone 100 unloads the model file of the large language model in the first memory pool.

[0081] As shown in the examples of FIGS. 6A and 6B, the system service determines a memory management policy according to the size of the available memory space indicated by the available memory information after acquiring the available memory information, such as determining a memory management policy for loading or unloading the model file of the large language model in the first memory pool. Further, the system service sends a management message to the large language model service, and the management message is used to indicate the memory management policy, such as indicating the memory management policy for loading or unloading the model file of the large language model in the first memory pool. The large language model service loads or unloads the model file of the large language model in the first memory pool according to the memory management policy indicated by the management message. Figure 5 Figure 6 As shown in the examples of FIGS. 6A and 6B, the system service determines a memory management policy according to the size of the available memory space indicated by the available memory information after acquiring the available memory information, such as determining a memory management policy for loading or unloading the model file of the large language model in the first memory pool. Further, the system service sends a management message to the large language model service, and the management message is used to indicate the memory management policy, such as indicating the memory management policy for loading or unloading the model file of the large language model in the first memory pool. The large language model service loads or unloads the model file of the large language model in the first memory pool according to the memory management policy indicated by the management message.

[0082] In some implementations, in order to more accurately control the amount of data loaded by the model file in the first memory pool, the mobile phone 100 can use multiple preset memory thresholds as reference conditions for determining whether the available memory space is sufficient.

[0083] ​In a case that the available memory space is greater than the first preset memory threshold, it indicates that the available memory space of the current mobile phone 100 is sufficient, and the memory state is good. The mobile phone 100 loads the model file of the large language model in the first memory pool. For example, the mobile phone 100 loads the model file of the large language model in the first memory pool at a preset loading rate (such as 5 megabytes per second), until the model file is completely loaded in the first memory pool, or until the available memory space of the mobile phone 100 is less than or equal to the first preset memory threshold. The first preset memory threshold can also be referred to as the balance waterline. The first preset memory threshold can be set according to actual application scenarios or requirements. For example, the first preset memory threshold can be set to 5000 megabytes (MB).

[0084] By preloading the model file of the large language model in the first memory pool in a case that the available memory space of the mobile phone 100 is sufficient and the memory state is good, the data amount of the model file loading when the large language model starts can be reduced, and the starting speed of the large language model can be accelerated.

[0085] For example, during the process of loading the model file of the large language model in the first memory pool, the mobile phone 100 can preferentially load the model file of a preset type according to the type of the model file. For example, the model file of the execution code type is preferentially loaded. In this way, the key file of the large language model can be preloaded in the first memory pool.

[0086] In a case that the available memory space is less than the first preset memory threshold and greater than the second preset memory threshold, it indicates that the memory pressure of the current mobile phone 100 increases, and the available memory space becomes smaller. In order to relieve the memory pressure of the mobile phone 100, the mobile phone 100 unloads part of the loaded model file in the preset cache pool. For example, the mobile phone 100 unloads the loaded model file in the first memory pool at a preset unloading rate (such as 5 megabytes per second), until the loaded data amount of the model file in the first memory pool is reduced to the second preset data amount (such as 500 MB, 600 MB, etc.), or until the available memory space of the mobile phone 100 is greater than or equal to the first preset memory threshold. The second preset memory threshold can also be referred to as the low memory emergency killing waterline. The second preset memory threshold can be set according to actual application scenarios or requirements. For example, the second preset memory threshold can be set to 900 MB. The second preset data amount can also be set according to actual application scenarios or requirements. For example, the second preset data amount can be set to 500 MB.

[0087] By unloading the model file of the large language model in the first memory pool in a case that the memory pressure of the mobile phone 100 increases and the available memory space becomes smaller, the memory pressure of the mobile phone 100 can be reduced. At the same time, part of the model file can be retained in the preset cache pool, so that part of the model file of the large language model is loaded in the preset cache pool when the large language model starts.

[0088] In a case where the available memory space is less than or equal to the second preset memory threshold, it indicates that the memory pressure of the mobile phone 100 is large, and the available memory space is seriously insufficient. The mobile phone 100 uninstalls all the loaded model files in the first memory pool. For example, the mobile phone 100 deletes the loaded model files in the first memory pool at a preset uninstallation rate (such as 5 megabytes per second, 10 megabytes per second), until the amount of loaded data of the model files in the first memory pool is reduced to 0, or until the available memory space of the mobile phone 100 is greater than or equal to the first preset memory threshold.

[0089] By uninstalling all the loaded model files in the first memory pool in a case where the memory pressure of the mobile phone 100 is large and the available memory space is seriously insufficient, the memory pressure of the mobile phone 100 can be relieved.

[0090] It can be understood that the memory space size of the first memory pool dynamically changes with the amount of loaded data of the model files in the first memory pool. When loading a model file, if the space of the first memory pool is insufficient, such as when the first memory pool is insufficient to accommodate the model file to be loaded, the mobile phone 100 fills the first memory pool (such as filling the memory space in the available memory space to the first memory pool), so that there is enough memory space in the first memory pool to load the model file of the large language model. When uninstalling a model file, the mobile phone 100 can recycle the memory space in the first memory pool (such as recycling the memory space of the first memory pool to the available memory space), so as to increase the available memory space of the mobile phone 100. The first memory pool is the memory space that has been allocated in the mobile phone 100. The size of the available memory space of the mobile phone 100 is inversely proportional to the size of the first memory pool. That is, the larger the first memory pool, the smaller the available memory space, and the smaller the first memory pool, the larger the available memory space.

[0091] In other implementations, in order to improve the utilization rate of memory space when the available memory space of the mobile phone 100 is insufficient, the mobile phone 100 can control the large language model service for running the large language model in addition to uninstalling the model files of the large language model in the first memory pool. For example, when the available memory space of the mobile phone 100 is insufficient, the mobile phone 100 sets the large language model service to a killable state. In this case, the mobile phone 100 can release memory space by killing the large language model service when the available memory space is less than a certain threshold or when the memory is recycled.

[0092] In one example, when the available memory space of the mobile phone 100 is less than or equal to the third preset memory threshold and greater than the second preset memory threshold, it indicates that the memory pressure of the mobile phone 100 is relatively large, and the available memory space is insufficient. In order to alleviate the memory pressure of the mobile phone 100, the mobile phone 100 can also set the preset language model service to a killable state when the memory space occupied by the preset language model service is greater than the preset occupation threshold. In this way, the mobile phone 100 can further release the memory space by killing the large language model service, thereby alleviating the memory pressure.

[0093] The third preset memory threshold is between the first preset memory threshold and the second preset memory threshold. That is, the third preset memory threshold is greater than the second preset memory threshold and less than the first preset memory threshold. The third preset memory threshold can also be referred to as a low memory kill waterline. The third preset memory threshold can be set according to actual application scenarios or requirements. For example, the third preset memory threshold can be set to 1800MB. The preset occupation threshold can be set according to actual application scenarios or requirements. For example, the second preset memory threshold can be set to 500MB.

[0094] In another example, when the available memory space of the mobile phone 100 is less than or equal to the second preset memory threshold, the mobile phone 100 can not only unload all loaded model files in the preset cache pool, but also set the large language model service to a killable state. In this way, when the available memory space of the mobile phone 100 is severely insufficient, the mobile phone 100 sets the large language model service to a killable state regardless of whether the memory space occupied by the large language model service is greater than the preset occupation threshold, thereby further releasing the memory space by killing the large language model service and alleviating the memory pressure.

[0095] It can be understood that the mobile phone 100 sets the large language model service to a killable state, which can be achieved by setting the process priority of the large language model service. The process priority of a process indicates the importance of the process. The higher the process priority of a process, the more important the process. The lower the process priority of a process, the less important the process, and the process is in a killable state. For example, when the process priority of a process is a first value, it indicates that the process is in a resident state or an unkillable state. When the process priority of a process is a second value, it indicates that the process is in a killable state. The mobile phone 100 can set the process priority of the large language model service to the second value to make the large language model service in a killable state.

[0096] The following takes the first preset memory threshold (balance waterline) as 4800MB, the second preset memory threshold (low memory emergency killing waterline) as 900MB, and the third preset memory threshold (low memory killing waterline) as 1800MB as an example, and the memory management strategy corresponding to the available memory space is exemplarily introduced in combination with Table 1.

[0097] Table 1: Memory management strategy corresponding to available memory space

[0098]

[0099] In the example, when the available memory space is greater than the balance waterline, the memory state is good, the system service sends a management message to the large language model service, and the large language model service executes the memory management strategy indicated by the management message: increasing the loaded data amount of the model file in the first memory pool, and setting the large language model service to an unkillable state. When the available memory space is equal to the balance waterline, the memory management strategy executed by the large language model service is to set the large language model service to an unkillable state. When the available memory space is between the balance waterline and the low memory killing waterline, it indicates that the memory pressure increases, and the memory management strategy executed by the large language model service is to reduce the loaded data amount of the model file in the first memory pool, and set the large language model service to an unkillable state. When the available memory space is less than or equal to the low memory killing waterline and greater than the low memory emergency killing waterline, the memory management strategy executed by the large language model service is to reduce the loaded data amount of the model file in the first memory pool, and set the large language model service to a killable state when the memory space occupied by the large language model service exceeds 500MB. When the available memory space is less than the low memory emergency killing waterline, the memory management strategy executed by the large language model service is to unload all the loaded data amount of the model file in the first memory pool, and set the large language model service to a killable state.

[0100] In the embodiment of the application, when the large language model is in an idle state, the mobile phone 100 can load or unload the model file of the large language model in the first memory pool according to the memory state (such as the available memory space). When the available memory space is sufficient, the model file is preloaded in the preset cache pool, so that the large language model has a resident part of the model file in the first memory pool, and the performance of the large language model when starting is improved. When the available memory space is insufficient, the model file of the large language model in the preset cache pool is unloaded, and the memory space of the first memory pool is returned to the available memory space of the mobile phone 100. In this way, the data amount of the model file loaded in the first memory pool is dynamically adjusted when the large language model is idle, and the speed of starting the large language model is improved.

[0101] In the above embodiments, the mobile phone 100 performs the memory management strategy at a certain frequency. In some embodiments, the mobile phone 100 can also perform the memory management strategy of the large language model immediately in the case of starting the second application, so as to reduce the influence of preloading the model file of the large language model in the first memory pool on the starting of the second application.

[0102] For example, if the system service of the mobile phone 100 detects the starting of the second application, the system service immediately determines the memory management strategy according to the available memory space of the mobile phone 100, and sends a management message to the large language model service, which indicates the determined memory management strategy through the management message. The large language model service executes the memory management strategy of the large language model according to the management message, such as loading or unloading the model file of the large language model in the first memory pool.

[0103] It can be understood that the second application is a large memory application with high demand for memory space. For example, the second application is a camera application, a game application, etc. In the case where the system service detects the starting of the second application, it can be judged whether the second application is a large memory application. For example, the mobile phone 100 can be provided with a preset application list. The preset application list is used to record large memory applications with high demand for memory space. If the second application is in the preset application list, it can be considered that the second application is a large memory application, and the system service immediately sends a management message to the large language model service, so that the mobile phone 100 can provide sufficient available memory space for the second application in priority.

[0104] In the above, the terminal device is taken as the mobile phone 100 as an example, and the process of dynamically adjusting the data amount of the model file loaded in the first memory pool when the large language model is not running is exemplarily introduced. The process of starting the large language model by the terminal device is further introduced below.

[0105] In some implementations, if part of the model file of the large language model is loaded in the first memory pool when the large language model is not running, the terminal device pre-fills the first memory pool in response to the first user instruction, so as to prepare memory space in advance for the model file loading of the large language model, and further improve the starting speed of the large language model. Further, the terminal device loads the model file of the large language model in the pre-filled first memory pool. After the model file is completely loaded, the terminal device runs the large language model, and generates the semantic reasoning result of the first user instruction through the large language model.

[0106] The process of starting the large language model by the terminal device in the case where part of the model file of the large language model is loaded in the first memory pool is summarized below through an example. Exemplarily, as shown in FIG. 6, the terminal device is taken as the mobile phone 100 as an example. Figure 7As shown, the terminal device invokes the large language model service (i.e., enters the semantic reasoning entry) in response to the first user instruction. The terminal device further calculates the memory requirement of the large language model, and performs pre-filling of the first memory pool according to the memory requirement of the large language model. At the same time, the terminal device determines whether the available memory space is insufficient according to the memory requirement of the large language model, and if the available memory space is insufficient, expands the available memory space through a memory recycling operation. Further, the terminal device loads the remaining unloaded model files in the pre-filled first memory pool to generate a semantic reasoning result of the first user instruction through the large language model.

[0107] In the following embodiments, the terminal device is also taken as a mobile phone 100, and the large language model has part of the model files loaded in the first memory pool. The starting process of the large language model is further introduced. As shown in Figure 8 The method provided by the embodiments of the present application includes the following steps:

[0108] S801, the mobile phone 100 determines the data amount of the unloaded model files in response to the first user instruction.

[0109] The first user instruction is used to trigger the voice assistant function of the first application. The first user instruction can be a voice instruction, a user operation instruction, etc. In the case where the first user instruction is a voice instruction, the mobile phone 100 can also recognize the first user instruction from voice data to text data, and display the text data of the first user instruction in the application interface of the first application.

[0110] The first application is used to provide a voice assistant function through a large language model. For example, the first application is a voice assistant application, a smart device control application, etc. For example, the first application can obtain a semantic reasoning result corresponding to the first user instruction through the large language model, and further implement voice assistant functions such as voice dialogue and intelligent question answering based on the semantic reasoning result.

[0111] In order to run the large language model, the mobile phone 100 needs to load all the model files of the large language model in the memory (such as the first memory pool). The mobile phone 100 calculates the data amount of the unloaded model files in the first memory pool in response to the first user instruction.

[0112] For example, taking the first user instruction as "generate a birthday video" as an example. As shown in Figure 9As shown, the first application of the mobile phone 100 receives a first user instruction of "generating a birthday video" reported by a bottom module (such as an audio driver). The first application responds to the first user instruction of "generating a birthday video" and notifies the large language model service to start the large language model. The large language model service further calculates the memory requirement of the large language model, that is, calculates the data amount of the model file that is not loaded in the preset cache pool. For ease of description, the data amount of the model file that is not loaded can be referred to as a first data amount. The first data amount is equal to the difference between the total data amount of the model file and a second data amount. The second data amount is the data amount of the model file that is loaded in the first memory pool. The second data amount can also be referred to as a resident amount.

[0113] S802, the mobile phone 100 pre-fills the first memory pool according to the first data amount.

[0114] After determining the data amount of the model file that is not loaded, that is, after determining the first data amount, the mobile phone 100 pre-fills the first memory pool according to the first data amount. For example, the mobile phone 100 fills part of the memory space of the available memory space into the first memory pool, so that the first memory pool has sufficient memory space to load the model file of the first data amount.

[0115] It can be understood that after part of the memory space in the available memory space is filled into the first memory pool, this part of the memory space is allocated memory space and no longer belongs to the available memory space, and the available memory space of the mobile phone 100 is reduced. The pre-filling of the first memory pool by the mobile phone 100 can be understood as that the mobile phone 100 fills the first memory pool without receiving a memory application request from the large language model service. Compared with the scheme of filling the first memory pool in response to the memory application request of the large language model service, the pre-filling of the first memory pool can prepare memory space for loading the model file of the large language model in advance, and further improve the startup speed of the large language model.

[0116] In some implementations, the available memory space of the mobile phone 100 can be insufficient, in which case the mobile phone 100 can also expand the available memory space by memory recycling to make the available memory space provide sufficient memory space to fill the first memory pool. Specifically, after determining the first data amount, the mobile phone 100 determines whether the available memory space is less than the first data amount. If the available memory space is less than the first data amount, it indicates that the available memory space can be insufficient, and the mobile phone 100 recycles the allocated memory space to expand the available memory space.

[0117] For example, as shown in FIG. 8, the mobile phone 100 can determine the available memory space of the mobile phone 100, and determine whether the available memory space is less than the first data amount. If the available memory space is less than the first data amount, the mobile phone 100 can recycle the allocated memory space to expand the available memory space. Figure 9As shown, after the large language model service calculates the memory requirement (i.e., the first data amount), the system service pre-requests the memory supply from the memory management module. If the available memory space is less than the first data amount, the system service instructs the memory management module to perform memory recovery. The memory management module can quickly recover the allocated memory space by compressing, dumping, and killing processes in the memory to expand the available memory space. The system service further instructs the memory management module to pre-fill the first memory pool by a pre-fill instruction. The memory management module pre-fills the first memory pool in response to the pre-fill instruction.

[0118] In the embodiment of the application, after detecting the memory requirement of loading the large language model, the mobile phone 100 can perform memory recovery on demand and pre-fill the first memory pool specific to the large language model to prepare the memory space for loading the model file in advance, cope with subsequent memory applications of the large language model, reduce the impact of the startup of the large language model on the performance of the operating system, and improve the speed of the startup of the large language model.

[0119] S803, the mobile phone 100 loads the model file in the first memory pool.

[0120] The mobile phone 100 loads the model file of the large language model in the first memory pool in response to the memory application request of the large language model service. For example, the mobile phone 100 loads the model file of the large language model in the first memory pool in response to the memory application request of the large language model service. Figure 9 As shown, after the pre-filling of the first memory pool starts, the large language model service sends a memory application request to the memory management module to request the memory space for loading the model file from the memory management module. The memory management module loads the model file of the large language model in the pre-filled first memory pool in response to the memory application request.

[0121] S804, in the case where the model file is completely loaded, the mobile phone 100 runs the large language model to generate the semantic reasoning result of the first user instruction through the large language model.

[0122] After the model file is completely loaded in the first memory pool, the large language model service of the mobile phone 100 runs the large language model based on the model file loaded in the first memory pool. For example, the large language model service inputs the text data of the first user instruction into the large language model, performs semantic analysis and semantic reasoning on the first user instruction by using the large language model, and obtains the semantic reasoning result of the first user instruction. The mobile phone 100 can also prompt the user with the semantic reasoning result.

[0123] For example, as shown in FIG. 8, the mobile phone 100 can run the large language model to generate the semantic reasoning result of the first user instruction through the large language model. Figure 9As shown, after the memory management module of the mobile phone 100 loads all the model files in the first memory pool, the memory management module returns a memory application success response message to the large language model service. In response to the memory application success response message, the large language model service runs the large language model, and obtains the semantic reasoning result of the first user instruction by using the large language model. After obtaining the semantic reasoning result of the first user instruction, the large language model service returns the semantic reasoning result to the first application. The first application can further send a prompt information to the user according to the semantic reasoning result. For example, the first user instruction is "generate a birthday video", and the semantic reasoning result can be a birthday theme related image. The first application generates a prompt information according to the semantic reasoning result, and the prompt information triggers the mobile phone 100 to display a preview window of the birthday theme related image in the application interface of the first application.

[0124] In the above embodiment, the terminal device is taken as an example of the mobile phone 100, and the starting process of the large language model in the case where part of the model files of the large language model are pre-loaded in the first memory pool is introduced. In another implementation manner, if all the model files of the large language model are loaded in the first memory pool when the large language model is not running, the terminal device can directly run the large language model in response to the first user instruction, which can greatly improve the starting speed of the large language model.

[0125] The method provided in the embodiments of the present application loads at least part of the model files of the large language model in the first memory pool in advance, so that the model files of the large language model are partially resident in the first memory pool, thereby fully utilizing the memory resources and optimizing the starting time of the large language model. By pre-filling the first memory pool, all the model files can be loaded in the first memory pool more quickly, so that the large language model continuously performs voice content reasoning. In addition, in the case where the available memory space is sufficient, the model files do not need to be repeatedly loaded into the first memory pool, and the performance requirement of the general flash memory storage and other storage devices is lower.

[0126] It can be understood that the scheme provided in the embodiments of the present application is introduced by taking the large language model as an example. The scheme provided in the embodiments of the present application can also be applicable to other large models.

[0127] In some other embodiments of the present application, a terminal device is also provided, which includes a memory, a processor and a computer program stored in the memory. When the computer program is executed by the processor, the terminal device can perform each function or step in the above method embodiments. Of course, the terminal device can also include other hardware structures. For example, the terminal device also includes a sensor, a communication module and other hardware structures. The structure of the terminal device can refer to the structure of the mobile phone 100 shown in the above. Figure 2 The structure of the mobile phone 100 shown in the above.

[0128] The embodiment of the present application further provides a chip system applied to the terminal device. The chip system comprises at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected through a line. For example, the interface circuit can be used to receive signals from other devices (for example, a memory). For another example, the interface circuit can be used to send signals to other devices (for example, a processor). Illustratively, the interface circuit can read a computer program stored in the memory and send the computer program to the processor. When the computer program is executed by the processor, the terminal device can execute various steps in the above embodiment. Of course, the chip system can also include other discrete devices, and the embodiment of the present application does not make specific limitations thereto.

[0129] The embodiment of the present application further provides a computer readable storage medium, which comprises a computer program. When the computer program is run on the terminal device, the terminal device can execute various functions or steps in the above method embodiment.

[0130] The embodiment of the present application further provides a computer program product, which comprises a computer program. When the computer program is run on a computer, the computer can execute various functions or steps in the above method embodiment. For example, the computer can be the terminal device.

[0131] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0132] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiment described above is only schematic. For example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0133] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or multiple physical units, that is, can be located in one place or distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0134] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0135] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, including a plurality of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage program codes.

[0136] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application.

Claims

1. A model loading method, characterized in that, The method is applied to a terminal device, wherein a first application is installed on the terminal device, and the first application is used to provide voice assistant functionality through a preset language model; a first memory pool is created in the terminal device, and the first memory pool is used exclusively for loading model files of the preset language model, wherein the total data size of the model files of the preset language model is greater than a first preset data size; the method includes: After the terminal device is started, the first memory pool is created; Without running the preset language model, the available memory space of the terminal device is monitored, and at least a portion of the model files of the preset language model are dynamically loaded or unloaded in the first memory pool based on the dynamic changes in the available memory space. The size of the first memory pool dynamically changes with the amount of loaded data in the model files within the first memory pool. A larger first memory pool results in a smaller available memory space after loading model files, and a smaller first memory pool results in a larger available memory space after unloading model files. The model files include model parameters and executable code. Receive a first user instruction, which is used to trigger the voice assistant function of the first application; In response to the first user instruction, if all model files of the preset language model are loaded in the first memory pool, the preset language model is run, and the semantic reasoning result of the first user instruction is generated through the preset language model.

2. The method according to claim 1, characterized in that, Without running the preset language model, the available memory space of the terminal device is monitored, and based on the dynamic changes in the available memory space, at least a portion of the model files of the preset language model are dynamically loaded or unloaded from the first memory pool, including: Without running the preset language model, obtain the available memory information of the terminal device, wherein the available memory information is used to indicate the size of the available memory space; If the available memory space is greater than a first preset memory threshold, then at least a portion of the model files of the preset language model are loaded into the first memory pool.

3. The method according to claim 2, characterized in that, The method further includes: If the available memory space is less than a first preset memory threshold and greater than a second preset memory threshold, then some of the loaded model files are unloaded from the first memory pool; wherein the second preset memory threshold is less than the first preset memory threshold.

4. The method according to claim 3, characterized in that, The method further includes: If the memory space occupied by the preset language model service is greater than the preset memory space threshold, and the available memory space is less than or equal to the third preset memory threshold, then the preset language model service is set to a detectable state. The preset language model service is used to run the preset language model, and the third preset memory threshold is greater than the second preset memory threshold and less than the first preset memory threshold.

5. The method according to any one of claims 2-4, characterized in that, The method further includes: If the available memory space is less than or equal to the second preset memory threshold, then all loaded model files are unloaded from the first memory pool.

6. The method according to claim 5, characterized in that, The method further includes: Set the preset language model service to a detectable state, wherein the preset language model service is used to run the preset language model.

7. The method according to any one of claims 1-4, characterized in that, In response to the first user instruction, and assuming all model files of the preset language model are loaded in the first memory pool, the preset language model is run to generate the semantic reasoning result of the first user instruction, including: In response to the first user instruction, if there are unloaded model files, the first memory pool is pre-filled according to the first data amount, wherein the first data amount is the data amount of the unloaded model files; In response to a memory request from a preset language model service, the unloaded model file is loaded into a pre-filled first memory pool; After loading all model files of the preset language model into the first memory pool, the preset language model is run to generate the semantic reasoning result of the first user instruction.

8. The method according to claim 7, characterized in that, Before pre-populating the first memory pool, the method further includes: If the available memory space of the terminal device is less than the first data amount, then the memory space already allocated to the terminal device is reclaimed.

9. A terminal device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory, wherein when the processor runs the computer program, the terminal device performs the method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The device stores a computer program that, when executed by a terminal device, causes the terminal device to perform the method according to any one of claims 1-8.

11. A computer program product, characterized in that, It includes a computer program, which, when run by a terminal device, causes the terminal device to perform the method according to any one of claims 1-8.

Images