Model update method, communication system and related device

Abstract

The embodiments of the present application provide a model update method, a communication system and a related device. A LoRA model can be updated by means of an end-cloud collaboration mechanism. An end side can download from a cloud side a LoRA model adapted for an application program and an end-side foundation model, thereby realizing the update and upgrade of an end-side large model. Thus, there is no need for an application program developer to train a LoRA model for different versions of foundation models, thereby significantly reducing the development cost; there is no need for an application program to package the LoRA model, resulting in a lightweight installation package; and the application program developer only hosts the LoRA model on a cloud side, which is more cost-effective compared with the solution of hosting an entire large model on the cloud side.

Description

Model updating method, communication system and related device

[0001] The present application claims priority to the Chinese patent application No. 202410808080.7, filed on June 20, 2024, with the State Intellectual Property Office of China, and entitled “Model updating method, communication system and related device”, the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the electronic technical field, in particular to a model updating method, a communication system and related device. BACKGROUND

[0003] The parameter quantity of a large model is large, and the level is more, resulting in a large binary file after the whole model training. The large model is about 1G after sinking to the end side, for example, the 1.5 billion hummingbird large model is about 1.3G after sinking to the end side. The current mainstream end-side large model is mainly pre-installed in the read only memory (ROM) of a terminal device such as a mobile phone, and the storage requirement of the end-side device is very high. Moreover, the full weight parameter of the large model is large, and the model upgrade is difficult. SUMMARY

[0004] The embodiments of the present application provide a model updating method, a communication system and related device, which can realize the upgrade of the end-side large model through the LoRA and other model fine-tuning technologies, meet the diversity output demand of the large model for different application programs, and obviously reduce the burden of the end-side and the application program development party without packing the LoRA model.

[0005] In a first aspect, the embodiments of the present application provide a communication system, which can include a terminal device and a server. The server can store a plurality of fine-tuning models, and the plurality of fine-tuning models can be divided into a plurality of groups, and one group of fine-tuning models corresponds to one base model. The terminal device can store a first base model and install a first program. The first base model belongs to the plurality of base models corresponding to the plurality of groups of fine-tuning models. The terminal device can be configured to obtain a first fine-tuning model from the server, and the first fine-tuning model belongs to the plurality of fine-tuning models. The first fine-tuning model can be used for the first program. The terminal device is further configured to fuse the first base model and the first fine-tuning model.

[0006] In the first aspect, the first program can be an artificial intelligence program or an artificial intelligence service or an artificial intelligence function. In the first aspect, the communication system, the terminal device and the server can be the communication system 10, the terminal device 20 and the server 30 in the subsequent embodiments respectively. The fine-tuning model mentioned in the first aspect can be a LoRA model, and the first fine-tuning model can be, for example, the first LoRA model mentioned in the subsequent embodiments.

[0007] In the first aspect, the group of fine-tuned models corresponding to the same base model can be further divided into LoRA models trained by different training sample sets. The different training sample sets can be training sample sets corresponding to different programs respectively, and are used to reflect the diverse output requirements of different programs on the large model.

[0008] In the first aspect, the first fine-tuned model is adapted to the first program and also adapted to the first base model. The first program can use a large model fused by the first fine-tuned model and the first base model to perform inference operation, so as to output a specific artificial intelligence service to a user.

[0009] The communication system described in the first aspect can realize updating of the fine-tuned model through an end-cloud collaboration mechanism. The terminal side can download the fine-tuned model adapted to the application program and the base model from the cloud side, and then realize updating and upgrading of the large model. In this way, the developer of the application program does not need to train fine-tuned models for different versions of base models, and the development cost is obviously reduced. The application program does not need to package the fine-tuned model, and the installation package is light. The fine-tuned model hosted by the application program developer on the cloud side is more cost-saving compared with the scheme that the whole large model is hosted on the cloud side.

[0010] In combination with the first aspect, in some embodiments, the terminal device is configured to obtain the first fine-tuned model from the server, and specifically can include: the terminal device is configured to send a first request to the server, then receive a download address of the first fine-tuned model returned by the server, and download the first fine-tuned model according to the download address. The first request can carry the following information: program information of the first program, model information of the first base model. The information carried in the first request can be used for the server to match the first fine-tuned model from a plurality of fine-tuned models. The first request can be a LoRA model updating request in subsequent embodiments.

[0011] In combination with the first aspect, in some embodiments, the terminal device can also be configured to use a first fusion model fused by the first base model and the first fine-tuned model to perform inference operation on user input received by the first program, to obtain a first inference result.

[0012] In combination with the first aspect, in some embodiments, the first program does not include the first fine-tuned model, and the first fine-tuned model is not packaged in the first program. The first program can include program code for calling a first interface, the first interface being used to load and fuse the first base model and the first fine-tuned model. The information carried in the first request, such as the program information of the first program and the model information of the first base model, can be used as the incoming parameter of the first interface. The first program can also include program code for calling a second interface, the second interface being used to call the fused model to perform inference operation on user input. The user input received by the first program can be used as the incoming parameter of the second interface.

[0013] With reference to the first aspect, in some embodiments, the terminal device can be further configured to return the first inference result to the server, and the returned first inference result can be used as a training sample of the fine-tuned model corresponding to the first program.

[0014] In a second aspect, the embodiments of the present application provide a model updating method, which can be applied to a terminal device in a first communication system. The first communication system can be the communication system described in the first aspect. The first communication system can include a terminal device and a server. The server stores a plurality of fine-tuned models. The plurality of fine-tuned models are divided into a plurality of groups. One group of fine-tuned models corresponds to one base model. The terminal device stores a first base model and installs a first program. The first base model belongs to a plurality of base models corresponding to a plurality of groups of fine-tuned models.

[0015] The method can include: the terminal device obtaining a first fine-tuned model from the server, and fusing the first base model and the first fine-tuned model. The first fine-tuned model belongs to the plurality of fine-tuned models. The first fine-tuned model is used for the first program.

[0016] In the second aspect, the first fine-tuned model is a fine-tuned model adapted to the first program and adapted to the first base model.

[0017] In the second aspect, the first program can be an artificial intelligence program or an artificial intelligence service or an artificial intelligence function. In the second aspect, the communication system, the terminal device, and the server can be the communication system 10, the terminal device 20, and the server 30 in subsequent embodiments. The fine-tuned model mentioned in the first aspect can be a LoRA model. The first fine-tuned model can be, for example, the first LoRA model mentioned in subsequent embodiments.

[0018] Implementing the method of the second aspect can upgrade the end-side large model through model fine-tuning technologies such as LoRA, meet the diverse output needs of different application programs for large models, and significantly reduce the burden on the end-side and application program developers without the need to package LoRA models.

[0019] With reference to the second aspect, in some embodiments, the terminal device obtains the first fine-tuned model from the server, which can specifically include: the terminal device sends a first request to the server, then receives a download address of the first fine-tuned model returned by the server, and downloads the first fine-tuned model according to the download address. The first request can carry the following information: program information of the first program and model information of the first base model. The information carried in the first request is used for the server to match the first fine-tuned model from the plurality of fine-tuned models. The first request can be a LoRA model updating request in subsequent embodiments.

[0020] With reference to the second aspect, in some embodiments, the model updating method can further include: the terminal device performing inference operation on the user input received by the first program by using the first fusion model generated by fusing the first base model and the first fine-tuned model, to obtain a first inference result.

[0021] With reference to the second aspect, in some embodiments, the terminal device can further return the first inference result to the server, and the returned first inference result can be used as a training sample of the fine-tuned model corresponding to the first program.

[0022] With reference to the second aspect, in some embodiments, the first program does not include the first fine-tuned model, and the first fine-tuned model is not packaged in the first program. The first program can include program code for calling the first interface, the first interface being used to load and fuse the first base model and the first fine-tuned model. The information carried in the first request, such as the program information of the first program and the model information of the first base model, can be used as the incoming parameter of the first interface. The first program can further include program code for calling the second interface, the second interface being used to call the fused model to perform inference operation on the user input. The user input received by the first program can be used as the incoming parameter of the second interface.

[0023] With reference to the second aspect, in some embodiments, when the first fine-tuned model is matched from the plurality of fine-tuned models, the server can consider the fine-tuned model matching strategy of the first program in addition to considering the adaptation of the first program and the first base model. The first fine-tuned model can specifically be a fine-tuned model found from the plurality of fine-tuned models stored on the server according to the fine-tuned model matching strategy.

[0024] The fine-tuned model matching strategy can be the LoRA model matching strategy mentioned in subsequent embodiments. The terminal device can send the matching strategy of the first program to the server. For example, the fine-tuned model matching strategy of the first program is carried in the first request.

[0025] With reference to the second aspect, in some embodiments, the fine-tuned model matching strategy of the first program can be basic matching, that is, finding a fine-tuned model that adapts to the first base model and the first program from the plurality of fine-tuned models stored on the server without additionally considering other user experience requirements.

[0026] In combination with the second aspect, in some embodiments, the fine-tuned model matching strategy of the first program can be: exact matching, that is, finding a fine-tuned model from the plurality of fine-tuned models stored on the server, which is adapted to the first base model and the first program, and is also adapted to the device capability of the terminal device. In the embodiment adopting the matching strategy of exact matching, the first request sent by the terminal device to the server can further carry device capability information of the terminal device, such as version of the operating system, model of the processor chip, etc., so that the server can learn the device capability of the terminal device based on the device capability information, and realize exact matching of the fine-tuned model.

[0027] In combination with the second aspect, in some embodiments, the fine-tuned model matching strategy of the first program can also be: finding a minimum fine-tuned model from the plurality of fine-tuned models stored on the server 30, which is adapted to the first base model and the first program. This matching strategy can reduce the download amount of the fine-tuned model and improve the download speed.

[0028] In combination with the second aspect, in some embodiments, before the terminal device sends the first request to the server, the model updating method can further include: the terminal device runs the first program, finds a third fine-tuned model locally, which is adapted to the first base model and the first program, fuses the third fine-tuned model with the first base model to obtain a third fused model, and then uses the third fused model to perform inference operation on the user input received by the first program to obtain a third inference result. The version of the third fine-tuned model can be older than the version of the first fine-tuned model.

[0029] That is, in response to the demand of the user running the first program, if there is a fine-tuned model locally on the terminal side, which is adapted to the first program, even if the version is old, the terminal device first uses the local fine-tuned model and the first base model to fuse to perform inference operation as soon as possible, and then updates the LoRA model. In this way, the timely output of the inference result and the updating of the LoRA can be taken into account.

[0030] In combination with the second aspect, in some embodiments, fusing the first base model and the first fine-tuned model can specifically include: the terminal device 20 converts the first base model and the first LoRA model into model structure and model weight, loads the model structure and the model weight into the memory, and then fuses the weight of the first base model and the weight of the first LoRA model based on the model structure in the memory to obtain the weight of the first fused model.

[0031] In combination with the second aspect, in some embodiments, the model updating method can further include: quantizing the weight of the first fused model, and storing the quantized weight of the first fused model in a cache queue. The quantized weight of the first base model is also stored in the cache queue. In this way, once the fused model is generated, the terminal device can store the quantized weight of the fused model in the cache, which can also create conditions for the subsequent rapid switching of artificial intelligence services.

[0032] In conjunction with the second aspect, in some embodiments, the model update method may further include: the terminal device detects an operation switching from the first program to the second program, determines whether the cache queue contains the quantization weights of the second fusion model formed by fusing the second fine-tuning model and the first base model, and if so, dequantizes the quantization weights of the second fusion model, replacing the weights of the first fusion model in memory with the weights of the second fusion model. Then, the terminal device can use the second fusion model to perform inference operations on the user input received by the second program to obtain a second inference result. The second fine-tuning model is a fine-tuning model adapted to both the second program and the first base model.

[0033] Switching from the first program to the second program is to switch between artificial intelligence services. The operation that triggers this switch can be an operation that jumps to the second program while the first program is running, such as returning to the desktop from the first program and then clicking the desktop icon of the second program. The implementation of this operation is not limited in this embodiment.

[0034] The model update method described in the second aspect can significantly reduce the number of operations and computational load during program switching by setting up a cache queue, thereby significantly improving the switching response speed of artificial intelligence services.

[0035] In conjunction with the second aspect, in some embodiments, similar to the first program, the second program can also be a system application or a third-party application capable of providing artificial intelligence services. The installation package of the second program does not include the second fine-tuning model. The second program may also include program code that calls the second interface. Switching from the first program to the second program causes the code calling the second interface in the second program to be executed, thereby triggering the terminal device to perform inference operations using the second fusion model. The second program may also include program code that calls the first interface. When the program code calling the second interface is executed, the second program can trigger a large model upgrade, that is, trigger the terminal device to send a fine-tuning model update request to the server to obtain a fine-tuned model adapted to the second program and the first base model.

[0036] Thirdly, embodiments of this application provide a model update method. This method can be applied to a server in a first communication system. The first communication system can be the communication system described in the first aspect above. The first communication system may include: a terminal device and a server. The server stores multiple fine-tuning models, which are divided into multiple groups, with each group of fine-tuning models corresponding to a base model. The terminal device stores a first base model and has a first program installed on it. The first base model belongs to multiple base models corresponding to multiple groups of fine-tuning models.

[0037] The method may include: the server matching a first fine-tuning model from multiple fine-tuning models and sending the first fine-tuning model to the terminal device.

[0038] In the third aspect, the first fine-tuning model is a fine-tuned model adapted to the first program and the first base model. The first LoRA model can be used in the first program. The first program can utilize the large model formed by fusing the first LoRA model and the first base model to perform inference operations, so as to output specific artificial intelligence services to users, such as providing high-quality text summarization, context-aware intelligent response, advanced grammar correction and other artificial intelligence functions.

[0039] In the third aspect, these multiple fine-tuned models can be generated by the server using different training sample sets based on different base models, thus adapting to different base models and different programs. Different base models include the first base model. Different training sample sets correspond to different programs, which can consist of, but are not limited to, user input data and inference results from different artificial intelligence programs. Different programs include the first program.

[0040] In the third aspect, the communication system, terminal device, and server can be the communication system 10, terminal device 20, and server 30 in subsequent embodiments, respectively. The fine-tuning model mentioned in the first aspect can be a LoRA model, and the first fine-tuning model can be, for example, the first LoRA model mentioned in subsequent embodiments.

[0041] Implementing the third approach allows for the upgrading of large-scale models on the client side through model fine-tuning techniques such as LoRA, meeting the diverse output requirements of different applications for large-scale models. Moreover, applications do not need to package the LoRA model, significantly reducing the burden on the client side and application developers.

[0042] In conjunction with the third aspect, in some embodiments, the server sends the first fine-tuning model to the terminal device, specifically including: the server returning a download address of the first fine-tuning model to the terminal device. The download address is used by the terminal device to download the first fine-tuning model.

[0043] In conjunction with the third aspect, in some embodiments, the server can receive the first inference result returned by the terminal device and use the first inference result as a training sample corresponding to the first program.

[0044] In conjunction with the third aspect, in some embodiments, before the server matches the first fine-tuning model from multiple fine-tuning models, the model update method may further include: the server receiving a first request sent by the terminal device, wherein the server can specifically find the first fine-tuning model that adapts to the first program and the first base model from multiple fine-tuning models based on the information carried in the first request. The first request may carry the following information: program information of the first program and model information of the first base model.

[0045] In conjunction with the third aspect, in some embodiments, the first request may also carry: a fine-tuning model matching strategy for the first program. Specifically, the server may also match the first fine-tuning model from multiple fine-tuning models according to the matching strategy.

[0046] That is, when matching the first fine-tuning model from multiple fine-tuning models, in addition to considering the adaptation to the first program and the first base model, the server can also consider the fine-tuning model matching strategy of the first program. The fine-tuning model matching strategy can be the LoRA model matching strategy mentioned in subsequent embodiments. The terminal device can send the matching strategy of the first program to the server.

[0047] In conjunction with the third aspect, in some embodiments, the fine-tuning model matching strategy of the first program can be: basic matching, that is: finding the fine-tuning model that is suitable for the first basic model and the first program from multiple fine-tuning models stored on the server, without considering other user experience requirements.

[0048] In conjunction with the third aspect, in some embodiments, the fine-tuning model matching strategy of the first program can be: precise matching, that is: finding a fine-tuning model from multiple fine-tuning models stored on the server that is compatible with the first base model and the first program, and also compatible with the device capabilities of the terminal device. In embodiments employing this precise matching strategy, the first request sent by the terminal device to the server may further carry: device capability information of the terminal device, such as the operating system version, processor chip model, etc., so that the server can understand the device capabilities of the terminal device based on the device capability information and achieve precise matching of the fine-tuning model.

[0049] In conjunction with the third aspect, in some embodiments, the fine-tuning model matching strategy of the first program can also be: finding the smallest fine-tuning model that fits both the first base model and the first program from multiple fine-tuning models stored on the server 30. This matching strategy can reduce the amount of fine-tuning model downloaded and improve download speed.

[0050] Fourthly, this application provides a terminal device including a processor and a memory; wherein the memory is coupled to the processor and is used to store computer program code, the computer program code including computer instructions, and when the processor executes the computer instructions, the terminal device performs the method described in the second aspect and any possible implementation thereof.

[0051] Fifthly, this application provides a server including a processor and a memory; wherein the memory is coupled to the processor and is used to store computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the server to perform the methods described in the third aspect and any possible implementation thereof.

[0052] In a sixth aspect, embodiments of this application provide a chip system applied to a terminal device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the terminal device to perform the methods described in the second aspect and any possible implementation thereof.

[0053] In a seventh aspect, embodiments of this application provide a chip system applied to a server. The chip system includes one or more processors, which are configured to invoke computer instructions to cause the server to perform the methods described in the third aspect and any possible implementation thereof.

[0054] Eighthly, this application provides a computer-readable storage medium having instructions stored thereon that, when executed on a terminal device, cause the terminal device to perform the method described in the second aspect and any possible implementation thereof.

[0055] Ninthly, this application provides a computer-readable storage medium having instructions stored thereon that, when executed on a terminal device, cause the terminal device to perform the method described in the third aspect and any possible implementation thereof.

[0056] In a tenth aspect, this application provides a computer program product including instructions that, when executed on a terminal device, cause the terminal device to perform the method described in the second aspect and any possible implementation thereof.

[0057] In one aspect, this application provides a program product including instructions that, when the computer program product is run on a server, cause the server to perform the method described in the third aspect and any possible implementation thereof. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0059] Figure 1 illustrates a mainstream fine-tuning technique for large models;

[0060] Figure 2 shows the AI ​​Core that Google released for the Pixel 8 Pro phone;

[0061] Figure 3 illustrates the problems with LoRA technology in AI Core;

[0062] Figure 4 shows the communication system provided in an embodiment of this application;

[0063] Figure 5 illustrates a usage scenario of the model update method provided in the embodiments of this application;

[0064] Figure 6 illustrates the overall flow of the model update method provided in the embodiments of this application;

[0065] Figure 7 illustrates the user interface for enabling the automatic upgrade function of the end-side model;

[0066] Figure 8 illustrates the specific implementation process of downloading the LoRA model from the cloud side on the device side;

[0067] Figure 9 illustrates the technical effects of the model update method provided in the embodiments of this application;

[0068] Figure 10 illustrates a specific implementation flow of the model update method provided in an embodiment of this application;

[0069] Figure 11 illustrates the loading and fusion process of the LoRA model and the base model;

[0070] Figure 12 illustrates a traditional model switching method used when switching AI services;

[0071] Figure 13 illustrates the process further included in the model update method provided in the embodiments of this application;

[0072] Figure 14 illustrates the terminal device provided in an embodiment of this application;

[0073] Figure 15 shows a server provided in an embodiment of this application. Detailed Implementation

[0074] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be a limitation of this application.

[0075] Different third-party applications have different requirements for large models, such as demanding different outputs, including answering questions, summarizing data, and writing code. This diversity of needs is difficult to balance with a single large model, making it challenging to find a single model that can solve the problems that various third-party applications expect to address, especially specialized problems in certain professional fields. To improve this situation, the base model of a large model can be fine-tuned using industry-specific training datasets to perform specific tasks within that industry.

[0076] Figure 1 illustrates a mainstream fine-tuning technique for large language models: Low-Rank Adaptation of Large Language Models (LoRA). LoRA is a lightweight model fine-tuning method that can replace the overall upgrade of the base model with model fine-tuning, and is particularly suitable for the rapid adaptation of large pre-trained models.

[0077] As shown in Figure 1, the LoRA training process can include:

[0078] Step 1. Add a bypass next to the pre-trained weights and perform a low-rank decomposition operation of dimensionality reduction and then dimensionality increase to simulate the weight update amount ΔW;

[0079] Step 2. Initialize matrix A with a random Gaussian distribution, and initialize matrix B with a zero matrix;

[0080] Step 3. During training, freeze the pre-training weights and train only the reduced-dimensional matrix A and the increased-dimensional matrix B;

[0081] Step 4. After training is complete, multiply matrix B by matrix A and add the result to the pre-trained weights W0 to obtain the fine-tuned model weights, which can be expressed as: W0 + ΔW = W0 + BA;

[0082] Thus, during inference, the inference result h is obtained using the fine-tuned model weights: h = W0x + ΔWx = W0x + BAx, where x represents the input data.

[0083] The main features of LoRA technology are as follows: 1. Pre-trained weights are shared, and different LoRA models can be set to adapt to different tasks; 2. Only local weights are trained, resulting in fast training speed; 3. The capabilities of large models can be personalized with a small amount of industry data, thereby enhancing their capabilities.

[0084] A LoRA model is a smaller model obtained by fine-tuning the base model of a larger model using the LoRA method. It can be merged with the larger model. The LoRA model of the larger model needs to match the version of the base model to be correctly loaded and run.

[0085] In this embodiment, the large model can be an artificial intelligence model pre-trained based on a large amount of data, also known as a pre-trained model. It is characterized by a large number of parameters and strong generation capabilities.

[0086] Figure 2 illustrates Google's AI Core for the Pixel 8 Pro. As shown in Figure 2, AI Core allows applications to access the underlying model Gemini Nano running on the device. Specifically, applications can access AI Core through a series of interfaces within the software development kit (SDK). AI Core also provides a LoRA fine-tuning module to further optimize the adaptation performance of the underlying model Gemini Nano to applications. This allows application developers to create customized LoRA fine-tuning modules using specific training datasets. LoRA fine-tuning modules can also be referred to as LoRA models.

[0087] However, since the LoRA model is a small model fine-tuned from the base model Gemini Nano, it needs to match the version of the base model Gemini Nano to be loaded and run correctly. To adapt to edge devices with different hardware and software capabilities, the base model Gemini Nano is continuously upgraded, resulting in multiple versions. For example, as shown in Figure 3, to adapt to N device configurations represented by Device 1, Device 2... Device N, the base model evolved into Base Model 1, Base Model 2,... Base Model N. This leads to applications needing LoRA models adapted to different versions of the base model, resulting in high development costs. For example, as shown in Figure 3, in addition to its own business logic, the application "App-1" also needs to package LoRA models 1, 2, ... N, respectively, to adapt to Base Model 1, Base Model 2,... Base Model N. Moreover, an application's installation package needs to fully package the LoRA model; with the increasing number of LoRA model versions, the application's installation package becomes increasingly large.

[0088] This application provides a model update method, communication system, and related equipment. It enables the upgrading of large-scale models on the client side through model fine-tuning techniques such as LoRA, meeting the diverse output requirements of different applications for large models. Furthermore, applications do not need to package LoRA models, and the client side does not need to pre-install many LoRA models, significantly reducing the burden on both the client side and application developers. The embodiments of this application are not limited to LoRA models; other fine-tuning models can also be applied, and the type of fine-tuning model is not limited. The following will describe each example using the LoRA model.

[0089] Figure 4 illustrates the communication system 10 provided in an embodiment of this application.

[0090] The communication system 10 can update the LoRA model through an edge-cloud collaboration mechanism. The edge device can download the LoRA model adapted to the application and the edge device's base model from the cloud, thereby achieving the update and upgrade of the large model. In this way, application developers do not need to train LoRA models for different versions of the base model, significantly reducing development costs; applications do not need to package the LoRA model, making the installation package lightweight; and application developers only host the LoRA model on the cloud, which is more cost-effective than the solution of hosting the entire large model on the cloud.

[0091] As shown in Figure 4, the communication system 10 may include: a terminal device 20 and a server 30. Wherein:

[0092] Terminal device 20 refers to the end-side device or end-side device mentioned in the embodiments of this application. It can be a mobile phone, tablet computer, smart screen device, virtual reality (VR) device, augmented reality (AR) device, etc.

[0093] The terminal device 20 may store a first basic model, such as "basic model-V1.0". The first basic model may be a basic model that adapts to the device configuration or device capabilities of the terminal device 20.

[0094] The terminal device 20 may also have a first program installed, such as "App-1". That is, the first program can be an artificial intelligence program, an artificial intelligence service, or an artificial intelligence function. The first program can be a system application or a third-party application that can provide artificial intelligence services, or it can be an artificial intelligence service provided by a system application or a third-party application. A third-party application, as opposed to a system application, refers to an application downloaded and installed from an app store. The first program can call a large model to perform inference operations on user input to provide high-quality text summarization, context-aware intelligent response, advanced grammar correction, and other artificial intelligence functions or services. The large model called by the first program can be generated by fusing a first base model and a first LoRA model. The first LoRA model can be trained using the training sample set of the LoRA model corresponding to the first program; it is a LoRA model adapted to the first program, allowing the fused large model to reflect the personalized output requirements of the first program for the large model. In this embodiment, the first LoRA model is not packaged in the first program, nor is it pre-installed in the terminal device 20; instead, the terminal device 20 requests it from the server 30. This will be explained in detail below and will not be elaborated here.

[0095] Server 30 refers to the cloud-side or cloud-side server mentioned in the embodiments of this application. Server 30 may store multiple LoRA models, such as "LoRA-1-V1.0", "LoRA-1-V1.1", "LoRA-1-V2.0", "LoRA-2-V1.0", "LoRA-2-V1.1", and "LoRA-2-V2.0".

[0096] These LoRA models can be divided into multiple groups. Each group of LoRA models includes at least one LoRA model, and each group of LoRA models corresponds to a base model. For example, the group of LoRA models "LoRA-1-V1.0" and "LoRA-2-V1.0" corresponds to a base model: "Base Model-V1.0"; the group of LoRA models "LoRA-1-V1.1" and "LoRA-2-V1.1" corresponds to another base model: "Base Model-V1.1"; and the group of LoRA models "LoRA-1-V2.0" and "LoRA-2-V2.0" corresponds to yet another base model: "Base Model-V2.0".

[0097] A set of LoRA models corresponding to the same base model can be further divided into LoRA models corresponding to training sample sets for different applications. For example, in the set of LoRA models corresponding to "Base Model-V1.0", "LoRA-1-V1.0" and "LoRA-2-V1.0" correspond to the training sample sets "Industry-Specific Dataset-1" and "Industry-Specific Dataset-2", respectively. Similarly, in the set of LoRA models corresponding to "Base Model-V1.1", "LoRA-1-V1.1" and "LoRA-2-V1.1" correspond to the training sample sets "Industry-Specific Dataset-1" and "Industry-Specific Dataset-2", respectively. And again, in the set of LoRA models corresponding to "Base Model-V2.0", "LoRA-1-V2.0" and "LoRA-2-V2.0" correspond to the training sample sets "Industry-Specific Dataset-1" and "Industry-Specific Dataset-2", respectively. In these examples, "Industry-Specific Dataset-1" and "Industry-Specific Dataset-2" are the training sample sets for applications "App-1" and "App-2", respectively.

[0098] The multiple LoRA models stored on server 30 can be trained by server 30 using training sample sets corresponding to different applications, based on different base models. They are adapted to different base models and different applications. Specifically, server 30 can train a first LoRA model based on the first base model using the training sample set of the LoRA model corresponding to the first program. The first base model belongs to the multiple base models corresponding to the aforementioned multiple sets of LoRA models.

[0099] Terminal device 20 can be used to send a LoRA model update request to server 30 to request the first LoRA model from server 30. The first LoRA model can be used by a first program, that is, the first program can use the large model formed by fusing the first LoRA model and the first base model to perform inference operations in order to output specific artificial intelligence services to the user.

[0100] Correspondingly, after receiving the LoRA model update request sent by the terminal device 20, the server 30 can search for the first LoRA model that adapts to the first program and the first basic model from multiple LoRA models stored in the cloud based on the information carried in the LoRA model update request, and can return the download address of the first LoRA model to the terminal device 20.

[0101] A LoRA model update request, also known as a first request, carries the following information: program information of the first program and model information of the first base model. In this way, server 30 can search for the first LoRA model that matches both the first program and the first base model from multiple LoRA models stored in the cloud, based on the program information of the first program and the model information of the first base model. Server 30 can also consider the LoRA model matching strategy of the first program when searching for a LoRA model; this will be explained in detail later.

[0102] After downloading and obtaining the first LoRA model, the terminal device 20 can also be used to fuse the first basic model and the first LoRA model to achieve the update and upgrade of the large model on the terminal side.

[0103] The specific implementation details of the terminal side and cloud side in the communication system 10 will be explained in detail below.

[0104] 1. Specific implementation on the client side

[0105] As shown in Figure 4, the terminal device 20 may include a large model service module 21. The large model service module 21 may include a model loading module 23, a model fusion module 24, and a model inference module 25. Among them, the model loading module 23 and the model fusion module 24 can be used to provide large model update and upgrade services, and the model inference module 25 can be used to provide large model inference and computation services.

[0106] The main execution flow of each module can be summarized as follows:

[0107] Step 1: Model loading module 23 loads the first basic model into memory;

[0108] Step 2: Model loading module 23 loads the LoRA model corresponding to the first program into memory;

[0109] Step 3: The model fusion module 24 performs model fusion on the first basic model and the LoRA model corresponding to the first program that have been loaded into memory to obtain the fused large model;

[0110] Step 4: The model reasoning module 25 uses the fused large model to perform reasoning operations on the user input received by the application and obtain the reasoning results.

[0111] Step 5: The model inference module 25 can further send the inference results to the cloud. In this way, the inference results flow back to the cloud to be used as training samples for the LoRA model in the first procedure, thereby achieving automatic updates of the training samples. Prior to this, the large model service module 21 can also perform data screening on the inference results to remove problematic data involving user privacy, etc.

[0112] In this embodiment, loading the model into memory can refer to reading the model's structure, weights, and other parameters into memory. The model structure, weights, and other parameters of the LoRA model on the terminal device 20 can be saved as LoRA model files, such as model files "lora-a-v1.1" and "lora-b-v1.1". The LoRA model files can be stored in a specified path, such as the path " / data / ai / ".

[0113] As shown in Figure 4, the large model service module 21 may also include: service interface 22.

[0114] Service interface 22 may include a first interface and a second interface. The first interface can be used to load and merge the first base model and the downloaded first LoRA model; the second interface can be used to call the merged model to perform inference operations on user input and obtain inference results.

[0115] As shown in Figure 5, applications on terminal device 20 (such as applications "App-1" and "App-2") can use large model update and upgrade services, large model inference and computation services, etc., by calling the first interface and the second interface. Specifically, the first program can achieve the following:

[0116] (1) The first program includes program code that calls the first interface, such as "let engine = awaitllmInterface.createEngine(); engine.loadModel()", to load and merge the first base model and the downloaded LoRA model. The information carried in the LoRA model update request, such as the program information of the first program and the model information of the first base model, can be used as the input parameters of the first interface. In this example, "createEngine()" is used to invoke the SDK, and "engine.loadModel()" is used to call the loadModel() interface provided by the SDK. The loadModel() interface is an example of the first interface.

[0117] (2) The first program also includes program code that calls the second interface, such as "let engine = await llmInterface.createEngine(); engine.generate(queryInfo);", to call the fused model to perform inference operations on the user input. The user input received by the first program can be used as the input parameter of the second interface. In this example, "createEngine()" is used to invoke the SDK, and "engine.generate(queryInfo)" is used to call the generate() interface provided by the SDK. The generate() interface is an example of the second interface, where queryInfo represents the search term, which is an example of user input.

[0118] When the program code that calls the first interface is run, the first program can trigger the terminal device 20 to obtain the LoRA model adapted to the first program and the first basic model from the server 30, load and merge the LoRA model and the first basic model to upgrade the large model on the terminal side; when the program code that calls the second interface is run, the first program can trigger the inference operation of the large model, that is, trigger the terminal side to run the merged large model to perform inference operation and obtain the inference result.

[0119] The large model service module 21 may further include a LoRA upgrade management module 26. The LoRA upgrade management module 26 is responsible for the upgrade management of the LoRA model, sending LoRA model update requests to the server 30 to request the download of a new version of the LoRA model for one or more applications. When the first application calls the large model service, the LoRA upgrade management module 26 can perform information matching based on the LoRA information data on the client side, that is, verify whether the client side already has the LoRA model corresponding to the first application. If so, it first merges the existing LoRA model with the first basic model, and then triggers a silent upgrade of the LoRA model corresponding to the first application; if not, it directly triggers the download of the LoRA model.

[0120] The edge-side LoRA information data can be stored on the terminal device 20. The edge-side LoRA information data can include, as shown in Table 1, the model information of the LoRA model corresponding to different programs, the version information of the basic model adapted to the LoRA model, the edge-side processor chip model, etc.

[0121] Table 1

[0122] Table 1 is only used to explain the embodiments of this application. The edge LoRA information data may also include more information, and the information may be presented in other forms. The embodiments of this application do not limit this.

[0123] Each time a different LoRA model is obtained from the server 30, the terminal device 20 can record the model information of the LoRA model and its corresponding program information, basic model information, etc. in the terminal LoRA information data.

[0124] 2. Specific implementation on the cloud side

[0125] As shown in Figure 4, the server 30 may include a LoRA download management module 31. The LoRA download management module 31 may include: a LoRA model download module 311, a LoRA model search module 312, and a LoRA model information management module 313.

[0126] The LoRA model download module 311 can receive LoRA model update requests sent by the terminal device 20 and trigger the LoRA model search module 312 to search for LoRA models. The LoRA model search module 312 can obtain model information for all cloud-side LoRA models from the LoRA model information management module 313 and search for suitable LoRA models from multiple LoRA models stored in the cloud based on the information carried in the LoRA model update request. When performing a LoRA model search for an application, the LoRA model search module 312 can also consider the application's LoRA model matching strategy. Before this, the terminal device 20 needs to inform the server 30 of the application's LoRA model matching strategy, for example, by carrying the matching strategy in the LoRA model update request.

[0127] The LoRA model download module 311 can also be used to send the LoRA models searched by the LoRA model search module 312, such as the first LoRA model adapted to the first program and the first base model, to the terminal device 20. In this way, the terminal device 20 can use the LoRA model and the base model to merge them to achieve the update of the large model.

[0128] Furthermore, as shown in Figure 4, the server 30 may also include: a model training module 32 and a model storage module 33. Among them,

[0129] The model training module 32 can train LoRA models adapted to different base models and different programs using training data. The training data can include different training sample sets, such as "Industry-Specific Dataset-1" and "Industry-Specific Dataset-2". "Industry-Specific Dataset-1" and "Industry-Specific Dataset-2" can correspond to programs "App-1" and "App-2" respectively. "Industry-Specific Dataset-1" is specifically the LoRA model training sample set for "App-1", and "Industry-Specific Dataset-2" is specifically the LoRA model training sample set for "App-2".

[0130] The model training module 32 can also be used to collect user input, inference results, and other data from various programs, and use them to generate training sample sets for the LoRA models corresponding to different applications. As the collected training samples are continuously updated, the LoRA models trained by the model training module 32 will also be continuously updated, presenting new versions.

[0131] The LoRA model trained by the model training module 32 can be transferred to the model storage module 33 for storage. When storing the LoRA model, the model storage module 33 can also record model information, such as the application the LoRA model is compatible with, the base model the LoRA model is compatible with, the size of the LoRA model, the hardware capabilities of the processor chip and other devices compatible with the LoRA model, and the software capabilities of the operating system and other devices compatible with the LoRA model. This model information can be used by the LoRA model search module 312 to perform model searches. The application and base model of the LoRA model can be reflected in information such as the name and version of the LoRA model.

[0132] For example, the version information of the first LoRA model is represented as "LoRA-1-V1.0", where "V1.0" indicates that the base model adapted by the first LoRA model is "base model-V1.0" and the number "1" between the two hyphens "-" indicates that the first LoRA model was trained by "industry-specific dataset-1" and the application "App-1" corresponding to "industry-specific dataset-1" is the application adapted by the first LoRA model.

[0133] In this embodiment, server 30 may include multiple server devices. The LoRA download management module 31, model training module 32, and model storage module 33 may be deployed on these multiple server devices. Alternatively, server 30 may be a single server device, meaning that the LoRA download management module 31, model training module 32, and model storage module 33 are all deployed on the same server.

[0134] Figure 6 illustrates the overall flow of the model update method provided in the embodiments of this application.

[0135] This method can be applied to the communication system 10 shown in Figure 4. The communication system 10 may include: a terminal device 20 and a server 30. The server 30 may store multiple LoRA models, which may be divided into multiple groups, with each group of LoRA models corresponding to a basic model. The terminal device 20 may store a first basic model and install a first program. The first basic model belongs to the multiple basic models corresponding to the aforementioned groups of LoRA models.

[0136] As shown in Figure 6, the model update method may include the following steps:

[0137] S10 and server 30 match the first LoRA model from multiple LoRA models stored in the cloud.

[0138] The first LoRA model is a LoRA model adapted to the first program and the first basic model. The first LoRA model can be used in the first program.

[0139] These multiple LoRA models stored in the cloud can be generated by servers 30 using different training sample sets based on different base models, thus adapting to different base models and different programs. Different training sample sets correspond to different programs and can consist of, but are not limited to, user input data and inference results from different artificial intelligence programs.

[0140] S11. Terminal device 20 obtains the first LoRA model from server 30. Correspondingly, server 30 sends the first LoRA model to terminal device 20.

[0141] The first LoRA model may belong to multiple LoRA models stored on server 30. The first LoRA model is a LoRA model adapted to the first base model and the first program. Specifically, terminal device 20 can obtain the LoRA model adapted to the first program from a set of LoRA models corresponding to the first base model, and this LoRA model is the first LoRA model.

[0142] In one implementation, terminal device 20 may proactively request server 30 to obtain a first LoRA model. Server 30 may respond to the request and send the first LoRA model to terminal device 20. In another implementation, terminal device 20 may passively obtain the first LoRA model from server 30. For example, server 30 may periodically push the first LoRA model to terminal device 20 at midnight to avoid peak network hours. Alternatively, server 30 may proactively push the latest version of the first LoRA model to terminal device 20 when a new version is available. These examples are merely illustrative of embodiments of this application and should not be construed as limiting the scope of the application.

[0143] S12, Terminal device 20 integrates the first basic model and the first LoRA model to obtain the first fused model.

[0144] The fusion can be implemented as follows: the terminal device 20 converts the first basic model and the first LoRA model into a model structure and model weights, loads the model structure and model weights into memory, and then performs fusion processing on the weights of the first basic model and the weights of the first LoRA model in memory based on the model structure to obtain the weights of the first fused model.

[0145] The first LoRA model can be used in the first program. Specifically, the first program can use the large model formed by fusing the first LoRA model and the first base model to perform inference operations to output specific artificial intelligence services to the user, such as providing high-quality text summarization, context-aware intelligent response, and advanced grammar correction. The first program may include program code that calls the first interface. The first interface can be used to obtain the LoRA model from the server 30, load and fuse the first base model and the downloaded first LoRA model. When the program code that calls the first interface is run, the first program can trigger a large model update and upgrade, that is, trigger the terminal device 20 to obtain the first LoRA model from the server 30.

[0146] Since the first LoRA model is adapted to the first program, the large model obtained by fusing the first LoRA model and the first basic model can meet the first program's personalized output requirements for the large model.

[0147] Furthermore, in the model update method provided in this application embodiment, the first program does not need to package the first LoRA model; the first LoRA model is obtained by the terminal device 20 from the server 30. This not only makes the installation package of the first program lightweight but also reduces the development cost and shortens the development cycle. In addition, the developer of the first program only hosts the LoRA model on the cloud side. Compared with the solution of hosting the entire large model on the cloud side, the model update method provided in this application embodiment is also more cost-effective in terms of cloud usage.

[0148] Specifically, the model update method provided in this application embodiment can be implemented as an automatic upgrade function for the end-side model on the terminal device 20. For example, the user can turn on the switch 71 of the automatic upgrade function for the end-side model in the user interface 70 shown in Figure 7 to enable LoRA model upgrade under Wi-Fi. In this way, after the user triggers the running of the artificial intelligence service, the terminal device 20 can automatically update the LoRA model under the condition of Wi-Fi, that is, obtain the updated version of the LoRA model from the server 30. Specifically, the terminal device 20 can send a LoRA model update request to the server 30 to trigger the cloud side to determine whether there is an updated version of the LoRA model adapted to the terminal device 20 and the first program. The next time the artificial intelligence service is run, the terminal device 20 can use the updated version of the LoRA model to merge with the first basic model, thereby upgrading the end-side large model and improving the end-side artificial intelligence service experience.

[0149] As shown in Figure 8, terminal device 20 can obtain the first LoRA model from server 30 by requesting a download. S11 in Figure 6 can specifically include S21-S24:

[0150] S21, Terminal device 20 sends a first request to server 30. The first request may carry the following information: program information of the first program and model information of the first base model, which can be used to request the download of a LoRA model adapted to the first program and the first base model. The first request is also referred to herein as a LoRA model update request.

[0151] Correspondingly, server 30 can receive the first request sent by terminal device 20.

[0152] The information carried in the first request can be used by server 30 to match the first LoRA model from multiple LoRA models stored on it. The program information of the first program may include, for example, the package name, program identifier (ID), version, etc. The model information of the first base model may include, for example, the model version.

[0153] The first request may also include the version of the existing LoRA model on the terminal device 20, so that the server 30 can determine whether there is an updated version of the LoRA model on the cloud side that is compatible with the first base model and the first program. If it is determined that there is no updated version of the LoRA model on the cloud side, the server 30 can return specific response information to the terminal device 20 to inform the terminal side that there is currently no updated version of the LoRA model on the cloud side. In this case, after the user triggers the execution of the first program, the terminal device 20 can merge the existing local LoRA model and the first base model to update the large model on the terminal side, and use the merged model to perform inference operations to obtain inference results, so as to output artificial intelligence services to the user.

[0154] S22. Server 30 searches for the optimal LoRA model. That is, after receiving the first request, server 30 can match the first LoRA model from the multiple LoRA models stored on server 30.

[0155] The first LoRA model is compatible with the first program and the first basic model.

[0156] Furthermore, when matching the first LoRA model from multiple LoRA models, in addition to considering the adaptation of the first program and the first base model, the server 30 can also consider the LoRA model matching strategy of the first program. The terminal device 20 can send the matching strategy of the first program to the server 30. For example, the LoRA model matching strategy of the first program can be carried in the first request.

[0157] The LoRA model matching strategy for the first program can be: basic matching, that is: finding the LoRA model that matches the first basic model and the first program from multiple LoRA models stored on server 30, without considering other user experience requirements.

[0158] The LoRA model matching strategy for the first program can also be exact matching, that is, finding a LoRA model from multiple LoRA models stored on server 30 that is compatible with the first basic model and the first program, and also compatible with the device capabilities of terminal device 20. In an embodiment employing this exact matching strategy, the first request sent by terminal device 20 to server 30 may further carry device capability information of terminal device 20, such as the operating system version, processor chip model, etc., so that server 30 can understand the device capabilities of terminal device 20 based on this device capability information and achieve exact matching of LoRA models.

[0159] Not limited to the two matching strategies mentioned above, the LoRA model matching strategy for the first program can also be: finding the smallest LoRA model that fits both the first base model and the first program from multiple LoRA models stored on server 30. This matching strategy can reduce the download size of the LoRA model and improve the download speed.

[0160] The LoRA model matching strategy for the first program can also be: to find the latest LoRA model that fits both the first base model and the first program from among multiple LoRA models stored on server 30. This matching strategy is beneficial for updating the LoRA model to the latest version.

[0161] The first application can determine its LoRA model matching strategy from the aforementioned matching strategies based on user experience requirements. The first application's LoRA model matching strategy can change as user experience requirements evolve.

[0162] S23, Server 30 can return the download address of the first LoRA model to Terminal Device 20. This download address can be used by Terminal Device 20 to download the first LoRA model.

[0163] Correspondingly, terminal device 20 can receive the download address of the first LoRA model returned by server 30.

[0164] S24. After receiving the download address, the terminal device 20 downloads the first LoRA model according to the download address.

[0165] At this point, terminal device 20 obtains the first LoRA model from server 30. Next, terminal device 20 can merge the first LoRA model and the first base model to achieve updates and upgrades to the larger model.

[0166] The technical effects of the model update method provided in this application embodiment are shown in Figure 9. The program providing AI services only needs to integrate its own business logic, without needing to package the LoRA model. This solves the problem that such programs must carry all LoRA models to be compatible with the underlying large model, which leads to increasingly large program packages. Moreover, the program developer only hosts the LoRA model on the cloud side. Compared to solutions where the entire large model is hosted on the cloud side, the model update method provided in this application embodiment saves on cloud usage costs.

[0167] Figure 10 illustrates a specific implementation flow of the model update method provided in an embodiment of this application. The details are as follows.

[0168] S31, Terminal device 20 runs the first program.

[0169] Terminal device 20 can run the first program after detecting a user operation that runs the first program. This user operation could be, for example, clicking the desktop icon of the first program, or navigating to the first program from another application. This embodiment does not limit the implementation of this operation. Not limited to running the first program in response to a user operation, terminal device 20 can also run the first program based on internal events, such as activating a smart voice assistant when the remaining battery power is high, enabling companionship and conversation. This embodiment does not limit the conditions for starting the first program.

[0170] As mentioned above, the first program may include program code that calls the second interface. Starting the first program can cause the code calling the second interface within the first program to be executed, thereby triggering the terminal device 20 to perform inference operations using the fused large model. The first program may also include program code that calls the first interface. When the program code calling the first interface is executed, the first program can trigger a large model update and upgrade, that is, trigger the terminal device 20 to send a LoRA model update request to the server 30 to obtain a LoRA model adapted to the first program and the first base model, so as to upgrade the terminal-side large model.

[0171] The information carried in the LoRA model update request can be used as the input parameter for the first interface. The user input received by the first program can be used as the input parameter for the second interface. The first and second interfaces have been described in detail above, and will not be repeated here.

[0172] S32, Terminal device 20 can determine whether there is a LoRA model adapted to the first basic model and the first program on terminal device 20. If there is, then S33-S40 can be executed, that is: first inference operation and then model upgrade; if not, then S41-S46 can be executed, that is, first model upgrade and then inference operation.

[0173] First perform inference calculations, then upgrade the LoRA model (S33-S40).

[0174] S33, Terminal device 20 can integrate the first basic model with the existing LoRA model to obtain a fused model.

[0175] S34. Terminal device 20 uses the fusion model obtained in S33 to perform inference operations.

[0176] Specifically, the terminal device 20 can use the fusion model to perform inference operations on the user input received by the first program and obtain the inference result.

[0177] In this embodiment, the existing LoRA model on the terminal device 20 can be referred to as the third LoRA model, and its model version can be older than the first LoRA model. The terminal device 20 can trigger a silent model update after performing inference operations, updating and upgrading the LoRA model without the user's awareness, as shown in S35-S39. That is, when running the first program, if there is a LoRA model adapted to the first program locally on the terminal side, even if the version is old, the local LoRA model is first used to merge with the first base model to perform inference operations as quickly as possible, and then the LoRA model is updated later. This balances the timely output of inference results with the updating of the LoRA model.

[0178] S35, the terminal device 20 can return the inference result generated in S34 to the server 30, so as to use the inference result as training data for the LoRA model corresponding to the first program.

[0179] S36. Terminal device 20 may send a first request to server 30. Correspondingly, server 30 may receive the first request sent by terminal device 20.

[0180] The first request may carry the following information: program information of the first program and model information of the first basic model, which are used to obtain a LoRA model adapted to the first program and the first basic model from the server 30.

[0181] S37 and server 30 can match the first LoRA model from multiple LoRA models stored in the cloud. The first LoRA model can be adapted to the first program and the first basic model. The version of the first LoRA model can be newer than the versions of the existing LoRA models mentioned above.

[0182] Multiple LoRA models stored in the cloud can be trained by server 30. As shown in S30, server 30 can train LoRA models adapted to different base models and different programs using different training sample sets based on different base models. This point will be discussed further in the preceding text and will not be repeated here.

[0183] S38. Server 30 can return the download address of the first LoRA model to terminal device 20. Correspondingly, terminal device 20 can receive the download address of the first LoRA model returned by server 30.

[0184] This download link can be used to download the first LoRA model on terminal device 20.

[0185] S39, Terminal device 20 can download the first LoRA model from this download address.

[0186] At this point, the silent upgrade of the LoRA model to be adapted to the first program is complete.

[0187] Specifically, the silent upgrade of the LoRA model can be implemented as a LoRA model upgrade under Wi-Fi. For example, as shown in Figure 7, the user can turn on the switch 71 of the automatic model upgrade function on the terminal in the user interface 70. In this way, after the user runs the first program, the terminal device 20 can automatically update the LoRA model under the condition of Wi-Fi, that is, obtain the updated version of the LoRA model from the server 30.

[0188] As shown in S40, when the first program is run again, the terminal device 20 can merge the first LoRA model and the first basic model downloaded in S39 to obtain the first fused model. The first fused model is used to perform inference operations to obtain inference results, thereby upgrading the terminal-side large model and improving the terminal-side artificial intelligence service experience.

[0189] For specific details regarding LoRA model downloading and LoRA model matching described in S36-S39, please refer to S21-S24 in the embodiment of Figure 8, which will not be repeated here.

[0190] Upgrade the large edge model first, then perform inference operations (S41-S46)

[0191] S41. Terminal device 20 may send a first request to server 30. Correspondingly, server 30 may receive the first request sent by terminal device 20.

[0192] The first request may carry the following information: program information of the first program and model information of the first basic model, which are used to obtain a LoRA model adapted to the first program and the first basic model from the server 30.

[0193] S42 and server 30 can match the first LoRA model from multiple LoRA models stored in the cloud. The first LoRA model can be adapted to the first program and the first basic model.

[0194] S43. Server 30 can return the download address of the first LoRA model to terminal device 20. Correspondingly, terminal device 20 can receive the download address of the first LoRA model returned by server 30.

[0195] This download link can be used to download the first LoRA model on terminal device 20.

[0196] S44, Terminal device 20 can download the first LoRA model from this download address.

[0197] S45, Terminal device 20 can merge the first LoRA model and the first basic model to obtain the first fused model.

[0198] S46. Terminal device 20 uses the first fusion model to perform inference operations.

[0199] Specifically, terminal device 20 can use the first fusion model to perform inference operations on the user input received by the first program to obtain inference results. Afterwards, terminal device 20 can also return the inference results to server 30, as shown in S47, to use the inference results as training data for the LoRA model corresponding to the first program.

[0200] For specific details regarding LoRA model downloading and LoRA model matching described in S41-S44, please refer to S21-S24 in the embodiment of Figure 8, which will not be repeated here.

[0201] The loading and fusion process of the base model and the LoRA model can be illustrated in Figure 11. The base model and the local LoRA model are loaded into memory, converted into model structures and weights, and then, using computing resources such as the central processing unit (CPU), graphics processing unit (GPU), and neural network processing unit (NPU), the weights of the base model and the LoRA model loaded into memory are fused to obtain the weights of the fused model, thus completing the model fusion. However, the fused model weights consume a significant amount of memory, which can affect the overall smoothness of the machine's operation.

[0202] Therefore, in this embodiment, once the fusion model is generated, the terminal device 20 can store its quantized weights in a cache, which also creates conditions for rapid switching of subsequent artificial intelligence services. The weight cache queue in Figure 11 can be used to store the quantized weights.

[0203] As shown in Figure 12, the model update method provided in this embodiment may further include:

[0204] S51, Terminal device 20 can quantify the weights of the first fusion model.

[0205] S52, Terminal device 20 can store the quantization weights of the first fusion model in a cache queue.

[0206] In the cache queue shown in Figure 11, W0 can represent the quantization weight of the first base model, Quantize(Dequantize(W0)+W A1 *W B1 ) can represent the quantization weights of the fusion model of the first base model and the LoRA model "LoRA-1", Quantize(Dequantize(W0)+W A2 *W B2 The ) can represent the quantization weights of the fusion model of the first base model and the LoRA model "LoRA-2". In Figure 11, "Dequantize" means to perform dequantization and "Quantize" means to perform quantization.

[0207] Quantifying the weights of the fusion model and then storing them in a cache queue can reduce the amount of data for the model weights.

[0208] S53, Terminal device 20 can detect the operation of switching from the first program to the second program.

[0209] Similar to the first program, the second program can also be a system application or a third-party application that provides artificial intelligence services, or it can be an artificial intelligence service provided by a system application or a third-party application. The artificial intelligence service provided by the second program can be different from the artificial intelligence service provided by the first program.

[0210] Switching from the first program to the second program is to switch between artificial intelligence services. The operation that triggers this switch can be an operation that jumps to the second program while the first program is running, such as returning to the desktop from the first program and then clicking the desktop icon of the second program. The implementation of this operation is not limited in this embodiment.

[0211] S54, Terminal device 20 can determine whether the cache queue contains the quantization weights of the second fused model, which is formed by fusing the second LoRA model and the first basic model. The second LoRA model is a LoRA model adapted to both the second program and the first basic model.

[0212] If so, then terminal device 20 can execute S55-S57.

[0213] If not, as shown in Figure 13, the terminal device 20 needs to first unload the first LoRA model ("LoRA-1" represents the first LoRA model) and the first base model from memory, then load the second LoRA model ("LoRA-2" represents the second LoRA model) into memory, and then perform model fusion calculation on the second LoRA model and the first base model to obtain the second fused model. However, this scheme takes a long time, for example more than 1 second, to complete the switching of the fused models, which affects the switching response speed of the artificial intelligence service.

[0214] The second LoRA model is a LoRA model adapted to the second program and the first basic model.

[0215] S55, the terminal device 20 can dequantize the quantization weights of the second fusion model to obtain the weights of the second fusion model, that is, to recover the complete weights.

[0216] S56, Terminal device 20 can replace the weights of the first fusion model in memory with the weights of the second fusion model.

[0217] S57. The terminal device 20 can perform inference operations on the user input received by the second program through the second fusion model to obtain the inference result. This inference result can be called the second inference result, and it can also be returned by the terminal device 20 to the server 30 as training data for the LoRA model corresponding to the second program.

[0218] The installation package of the second program does not include the second LoRA model. The second program may also include program code that calls the second interface. Switching from the first program to the second program causes the code calling the second interface in the second program to be executed, thereby triggering the terminal device 20 to perform inference operations using the second fusion model. The second program may also include program code that calls the first interface. When the program code calling the second interface is executed, the second program can trigger a large model upgrade, that is, trigger the terminal device 20 to send a LoRA model update request to the server 30 to obtain a LoRA model adapted to the second program and the first base model.

[0219] The embodiments of this application can significantly reduce the number of operations and computational load when switching programs by setting up a cache queue, thereby significantly improving the switching response speed of artificial intelligence services.

[0220] Figure 14 shows a terminal device 200 provided in an embodiment of this application.

[0221] Terminal device 200 may be the terminal device 20 mentioned in the foregoing embodiments. The device type of terminal device 200 may be any of the following: mobile phone, tablet computer, handheld computer, desktop computer, laptop computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), as well as smart home devices such as smart screens and smart speakers, wearable devices such as smart bracelets, smartwatches, and smart glasses, extended reality (XR) devices such as augmented reality (AR), virtual reality (VR), and mixed reality (MR), in-vehicle devices, or smart city devices, etc.

[0222] As shown in Figure 14, the terminal device 200 may include: a processor 110, a memory 120, a display 130, a display driver integrated circuit (DDIC), antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a gyroscope sensor 180B, an accelerometer sensor 180E, and a touch sensor 180K, etc. The various components in the terminal device 200 can be connected via a bus.

[0223] The processor 110 can be one or more, and they can be integrated into an integrated circuit of a system-on-a-chip (SOC). An SOC is a system-on-a-chip. The processor 110 may include a central processing unit (CPU), a graphics processing unit (GPU), and a display driver integrated circuit (DDIC). The CPU can be an application processor (AP). The CPU and GPU can be used to render and composite the image to be displayed on the monitor 130. The processor 110 may also include a neural network processing unit (NPU), a modem processor, etc.

[0224] The processor 110 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0225] The processor 110 may include a cache memory, which can be used to store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can directly retrieve it from the cache memory, which can reduce the waiting time of the processor 110 and improve the program running efficiency.

[0226] The memory 120 may include a program storage area and a user data storage area. The program storage area may store the operating system and one or more applications (such as games), while the data storage area may store data created by the user during use of the terminal device 200 (such as photos and contacts). The memory 120 may be a high-speed random access memory or a non-volatile memory, such as a hard disk, flash memory, or universal flash storage (UFS). The memory 120 may also be an external memory card, such as a Micro SD card.

[0227] The memory 120 may also store the implementation program of the model update method provided in the embodiments of this application on the terminal side. When the processor 110 reads and runs the program from the memory 120, the terminal device 200 can execute the method.

[0228] The memory 120 can also be integrated with the processor 110 into the integrated circuit of the SOC.

[0229] The terminal device 200 can realize display functions through SOC, DDIC 140, and display 130.

[0230] The display 130 may have a local frequency conversion function and may be the display 10 described in the preceding embodiments. The display 130 may include a display panel, a timing controller (TCON), etc. The display panel may include multiple pixel units, and may be an organic light-emitting diode (OLED) display panel, with each pixel unit including an OLED. The TCON is mainly used to connect the GPU or SOC to the display panel. After receiving image data or control signals from the GPU or SOC, it sets the timing of the control signals and data signals according to the relevant data or signals, and transmits the control signals and data signals to the display driver circuit according to the timing sequence, thereby driving the display panel to perform graphic display.

[0231] The display driver integrated circuit (DDIC) 140 serves as the control core of the display 130, driving the display 130 to operate and receiving data from the SOC (processor 110), such as image data and some instructions. The DDIC 140 can send drive signals and data to the display panel of the display 130 in the form of electrical signals, thereby controlling the screen brightness and color, enabling image information such as letters and pictures to be displayed on the screen and completing the screen refresh.

[0232] The image data to be displayed sent by the SOC to the DDIC 140 can be stored in the frame buffer to complete the display sending (or image sending). Then, the DDIC 140 retrieves the image data from the frame buffer and drives the display 130 to display it.

[0233] The wireless communication function of the terminal device 200 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0234] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 200 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0235] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the terminal device 200. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0236] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display 130. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0237] The wireless communication module 160 can provide solutions for wireless communication applications on the terminal device 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0238] In some embodiments, antenna 1 of terminal device 200 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling terminal device 200 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0239] The terminal device 200 can perform shooting functions through an ISP, camera 193, video codec, GPU, display 130, and application processor.

[0240] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0241] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the terminal device 200 may include one or N cameras 193, where N is a positive integer greater than 1.

[0242] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when the terminal device 200 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0243] Video codecs are used to compress or decompress digital video. Terminal device 200 may support one or more video codecs. Thus, terminal device 200 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0244] NPU stands for Neural Network (NN) Computing Processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in terminal devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0245] Terminal device 200 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0246] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0247] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The terminal device 200 can listen to music or make hands-free calls through the speaker 170A.

[0248] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the terminal device 200 receives a telephone call or voice message, the receiver 170B can be brought close to the listener's ear to receive the voice message.

[0249] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Terminal device 200 may be equipped with at least one microphone 170C. In some embodiments, terminal device 200 may be equipped with two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, terminal device 200 may be equipped with three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0250] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0251] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. Terminal device 200 can receive button input and generate key signal inputs related to user settings and function control of terminal device 200. Motor 191 can generate vibration alerts. SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with terminal device 200.

[0252] The structure illustrated in Figure 14 does not constitute a specific limitation on the terminal device 200. The terminal device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.

[0253] Figure 15 shows the server 300 provided in an embodiment of this application.

[0254] Server 300 may be the server 30 mentioned in the foregoing embodiments. As shown in FIG15, server 300 may include: processor 210, memory 220, input / output device 230, communication module 240, etc., and these components may be coupled via a bus.

[0255] Server 300 may have powerful computing resources, and its processor 210 may include one or more powerful processors, such as central processing unit (CPU), neural network processing unit (NPU), graphics processing unit (GPU), etc.

[0256] The processor 210 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0257] The processor 210 may have a cache memory, which can be used to store instructions or data that the processor 210 has just used or that are used repeatedly. If the processor 210 needs to use the instruction or data again, it can directly retrieve it from the cache memory, which can reduce the waiting time of the processor 210 and improve the program running efficiency.

[0258] The processor 210 can also connect to external memory. This memory can be high-speed random access memory or non-volatile memory, such as a hard disk, flash memory, universal flash memory (UFS), etc. The memory can also be an external memory card, such as a Micro SD card.

[0259] The processor 210 is the computing core of the server 300, possessing powerful computing capabilities. Coupled with the memory 220, it can read and execute computer-readable instructions stored in the memory 220, running the operating system and various programs. Specifically, the CPU 210 can call programs stored in the memory 220, such as the cloud-side implementation of the model update method provided in this embodiment, and execute the instructions contained in that program.

[0260] The memory 220 may include high-speed random access memory, non-volatile memory, such as disk, flash memory, or other non-volatile solid-state storage devices. The memory 220 can be used to store various software programs and multiple sets of instructions. The memory 220 can store an operating system, such as Linux. The memory 220 can also store one or more programs, such as programs involved in patch creation, such as compilers and linkers. The memory 220 can also store the implementation program of the edge-cloud collaborative computing power scheduling method provided in the embodiments of this application.

[0261] Input / output device 230 may include devices such as a display screen, keyboard, and mouse, and can be used to receive user input and output program execution results to the user.

[0262] The communication module 240 may include a wired communication module and a wireless communication module. The wired communication module supports wired communication protocols such as Universal Serial Bus (USB), serial port, and Ethernet, communicating with other devices via physical communication cables. The wireless communication module may include 2G / 3G / 4G / 5G wireless communication modules, Wi-Fi communication modules, etc. The wireless communication module receives electromagnetic waves via an antenna, modulates and filters the electromagnetic wave signals, and sends the processed signal to the CPU 210. The wireless communication module can also receive signals to be transmitted from the CPU 210, modulate and amplify them, and then convert them into electromagnetic waves for radiation via the antenna.

[0263] The structure illustrated in Figure 15 does not constitute a limitation on server 300. Server 300 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.

[0264] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, read-only optical discs (CD-ROMs), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Additionally, the ASIC can reside in a transceiver or relay device. Alternatively, the processor and storage medium can exist as discrete components in a wireless access network device or user equipment device.

[0265] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0266] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A communication system, characterized in that, include: The terminal device and the server, wherein: The server stores multiple fine-tuning models, which are divided into multiple groups, with each group of fine-tuning models corresponding to a base model. The terminal device stores a first basic model and has a first program installed; the first basic model belongs to multiple basic models corresponding to the multiple sets of fine-tuning models. The terminal device is used to obtain a first fine-tuning model from the server. The first fine-tuning model belongs to the plurality of fine-tuning models and is used in the first program. The terminal device is also used to fuse the first basic model and the first fine-tuning model.

2. The communication system as described in claim 1, characterized in that, The terminal device is used to obtain the first fine-tuning model from the server, specifically including: The terminal device is used to send a first request to the server, the first request carrying the following information: program information of the first program and model information of the first basic model; the information carried in the first request is used by the server to match the first fine-tuning model from the plurality of fine-tuning models; The terminal device is used to receive the download address of the first fine-tuning model returned by the server; The terminal device is used to download the first fine-tuning model according to the download address.

3. The communication system as described in claim 1 or 2, characterized in that, The terminal device is also used to perform inference operations on the user input received by the first program using a first fusion model generated by fusing the first basic model and the first fine-tuning model, and to obtain a first inference result.

4. The communication system as described in any one of claims 1-3, characterized in that, The first program does not include the first fine-tuning model; the first program includes program code for calling a first interface, which is used to load and fuse the first base model and the first fine-tuning model.

5. The communication system as described in claim 4, characterized in that, The first program also includes program code for calling the second interface, which is used to call the fused model to perform inference operations on the user input and obtain the inference result.

6. The communication system as described in any one of claims 1-5, characterized in that, The terminal device is also used to return the first inference result to the server, and the returned first inference result is used as a training sample for the fine-tuning model corresponding to the first program.

7. A model update method, characterized in that, The method is applied to a terminal device in a first communication system, the first communication system comprising: the terminal device and a server, the server storing multiple fine-tuning models, the multiple fine-tuning models being divided into multiple groups, each group of fine-tuning models corresponding to a basic model; the terminal device storing a first basic model and installing a first program; the first basic model belonging to the multiple basic models corresponding to the multiple groups of fine-tuning models; The method includes: The terminal device obtains a first fine-tuning model from the server. The first fine-tuning model belongs to the plurality of fine-tuning models and is used in the first program. The terminal device integrates the first basic model and the first fine-tuning model.

8. The method as described in claim 7, characterized in that, The terminal device obtains the first fine-tuning model from the server, specifically including: The terminal device sends a first request to the server, the first request carrying the following information: program information of the first program and model information of the first basic model; the information carried in the first request is used by the server to match the first fine-tuning model from the plurality of fine-tuning models; The terminal device receives the download address of the first fine-tuning model returned by the server; The terminal device downloads the first fine-tuning model according to the download address.

9. The method as described in claim 7 or 8, characterized in that, Also includes: The terminal device uses a first fusion model, generated by fusing the first basic model and the first fine-tuning model, to perform inference operations on the user input received by the first program, and obtains a first inference result.

10. The method according to any one of claims 7-9, characterized in that, The first program does not include the first fine-tuning model; the first program includes program code for calling a first interface, which is used to load and fuse the first base model and the first fine-tuning model.

11. The method as described in claim 10, characterized in that, The information carried in the first request is used as the input parameter of the first interface.

12. The method as described in claim 10 or 11, characterized in that, The first program also includes program code for calling the second interface, which is used to call the fused model to perform inference operations on the user input and obtain the inference result.

13. The method as described in claim 12, characterized in that, The user input received by the first program is used as the input parameter for the second interface.

14. The method according to any one of claims 7-13, characterized in that, Also includes: The terminal device returns the first inference result to the server, and the returned first inference result is used as a training sample for the fine-tuning model corresponding to the first program.

15. The method as described in claim 14, characterized in that, The first request also carries: the fine-tuning model matching strategy of the first program; the first fine-tuning model is specifically the fine-tuning model found from multiple fine-tuning models stored on the server according to the fine-tuning model matching strategy.

16. The method as described in claim 15, characterized in that, The fine-tuning model matching strategy of the first program includes: Find the fine-tuning model that fits the first base model and the first program from the multiple fine-tuning models stored on the server; or, Find, from the multiple fine-tuning models stored on the server, a fine-tuning model that adapts to the first base model and the first program, and also adapts to the device capabilities of the terminal device; or, Find the minimum fine-tuning model that fits the first base model and the first program from among the multiple fine-tuning models stored on the server; or, Find the latest fine-tuned model that fits the first base model and the first program from among the multiple fine-tuned models stored on the server.

17. The method according to any one of claims 7-16, characterized in that, Before the terminal device sends the first request to the server, the method further includes: The terminal device detects the operation of running the first program; The terminal device finds a third fine-tuning model that is compatible with the first basic model and the first program on the terminal device, wherein the version of the third fine-tuning model is older than the version of the first fine-tuning model. The terminal device integrates the third fine-tuning model with the first basic model to obtain a third fusion model; The terminal device uses the third fusion model to perform inference operations on the user input received by the first program to obtain a third inference result.

18. The method according to any one of claims 7-17, characterized in that, The terminal device integrates the first basic model and the first fine-tuning model to obtain a first fusion model, which specifically includes: The terminal device converts the first basic model and the first fine-tuning model into a model structure and model weights; The terminal device loads the model structure and the model weights into memory; The terminal device performs a fusion process on the weights of the first basic model and the weights of the first fine-tuning model in the memory based on the model structure to obtain the weights of the first fused model.

19. The method as described in claim 18, characterized in that, Also includes: The terminal device quantizes the weights of the first fusion model and stores the quantized weights of the first fusion model in a cache queue; The cache queue also contains the quantization weights of the first basic model.

20. The method as described in claim 19, characterized in that, Also includes: The terminal device detects the operation of switching from the first program to the second program. The terminal device determines whether the cache queue contains the quantization weights of the second fusion model formed by fusing the second fine-tuning model and the first basic model. If so, it performs dequantization on the quantization weights of the second fusion model and replaces the weights of the first fusion model in memory with the weights of the second fusion model. The terminal device uses the second fusion model to perform inference operations on the user input received by the second program to obtain a second inference result; The second fine-tuning model is a fine-tuning model adapted to the second program and the first basic model.

21. The method as described in claim 20, characterized in that, The installation package of the second program does not include the second fine-tuning model; the second program includes program code for calling the first interface, which is used to load and merge the first base model and the first fine-tuning model.

22. The method as described in claim 21, characterized in that, The second program also includes program code for calling the second interface, which is used to call the fused model to perform inference operations on the user input and obtain the inference result.

23. A model update method, characterized in that, The method is applied to a server in a first communication system, the first communication system comprising: the server and a terminal device, the server storing multiple fine-tuning models, the multiple fine-tuning models being divided into multiple groups, each group of fine-tuning models corresponding to a base model; the terminal device storing a first base model and installing a first program; the first base model belongs to the multiple base models corresponding to the multiple groups of fine-tuning models; The method includes: The server selects a first fine-tuning model from the plurality of fine-tuning models, and the first fine-tuning model is used in the first program; The server sends the first fine-tuning model to the terminal device; the first fine-tuning model is used to fuse with the first base model.

24. The method as described in claim 23, characterized in that, The server sends the first fine-tuning model to the terminal device, specifically including: The server returns the download address of the first fine-tuning model to the terminal device; the download address is used by the terminal device to download the first fine-tuning model.

25. The method as described in claim 23 or 24, characterized in that, Also includes: The server trains and generates fine-tuned models that are adapted to different base models and different programs using training sample sets corresponding to different programs, based on different base models. The different base models include the first base model, and the different programs include the first program.

26. The method according to any one of claims 23-25, characterized in that, Also includes: The server receives the first inference result returned by the terminal device and uses the first inference result as a training sample corresponding to the first program.

27. The method according to any one of claims 23-26, characterized in that, Before the server matches the first fine-tuning model from the plurality of fine-tuning models, the method further includes: the server receiving a first request sent by the terminal device, the first request carrying the following information: program information of the first program and model information of the first base model; The server selects a first fine-tuning model from the plurality of fine-tuning models, specifically including: The server identifies the first fine-tuning model that is compatible with the first program and the first base model from among the plurality of fine-tuning models based on the information carried in the first request.

28. The method as described in claim 27, characterized in that, The first request also carries: the fine-tuning model matching strategy of the first program; The server selects a first fine-tuning model from the plurality of fine-tuning models, specifically including: The server selects the first fine-tuning model from the plurality of fine-tuning models according to the matching strategy.

29. The method as described in claim 28, characterized in that, The fine-tuning model matching strategy of the first program includes: Find the fine-tuning model that fits the first base model and the first program from the multiple fine-tuning models stored on the server; or, Find, from the multiple fine-tuning models stored on the server, a fine-tuning model that adapts to the first base model and the first program, and also adapts to the device capabilities of the terminal device; or, Find the minimum fine-tuning model that fits the first base model and the first program from among the multiple fine-tuning models stored on the server; or, Find the latest fine-tuned model that fits the first base model and the first program from among the multiple fine-tuned models stored on the server.

30. A terminal device, comprising a processor, a memory, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 7-22.

31. A server, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the method according to any one of claims 23-29.

32. A computer-readable storage medium having a computer program stored thereon; characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 7-29.

33. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 7-29.

Images