Federal learning-based training method and computing device

Abstract

The embodiment of the invention provides a training method based on federal learning and computing equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a training result of each round for a first parameter in a digital assistant model on each terminal device; based on the training result of each round corresponding to each terminal device, updating a first parameter of a digital assistant model on the cloud device until the digital assistant model on the cloud device meets a training termination condition, and obtaining a trained first parameter; and sending the trained first parameter to each terminal device, wherein the trained first parameter is used for the terminal device to update the first parameter in the digital assistant model of the terminal device. Therefore, training is carried out by combining a plurality of terminal devices, training and iteration of the large model are carried out by utilizing real use data of as many users as possible, it is ensured that the cloud large model can carry out continuous and integral learning according to data and preferences of a large number of users, and the overall intelligent degree is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method and computing device based on federated learning. Background Art

[0002] In the digital and intelligent age, people are increasingly inclined to use digital assistants. The emergence of large models has enabled the rapid development of digital assistants. Digital assistants predict user preferences, which is a key component for artificial intelligence (AI) companies to provide personalized services to users. However, since digital assistant user preferences are constantly changing, and different users have different preferences and usage scenarios, users cannot enjoy intelligent, personalized digital assistant services under a shared large model.

[0003] Currently, the intelligence of digital assistants is primarily based on a unified cloud-based brain, or a large cloud-based model. This unified model requires dynamic learning and updates to provide more intelligent and personalized services. However, this approach has two serious drawbacks:

[0004] 1. Different users require digital assistants to provide different and targeted intelligent services. The unified large model in the cloud cannot well support the diverse needs of thousands of people.

[0005] 2. The information of personal assistants involves a lot of private content and cannot be directly used for learning and iteration of large cloud models. Summary of the Invention

[0006] The embodiments of the present application provide a training method and computing device based on federated learning. By combining multiple terminal devices for training, the real usage data of as many users as possible are used to train and iterate the large model, ensuring that the cloud-based large model can perform continuous and holistic learning based on the data and preferences of a large number of users, thereby improving the overall intelligence level and better supporting the diverse needs of thousands of users.

[0007] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, a training method based on federated learning is provided, which is applied to a cloud device, wherein the cloud device is communicatively connected to multiple terminal devices, and a digital assistant model with the same network structure is deployed on the cloud device and the terminal devices. The method includes:

[0009] Obtaining a training result of each round of a first parameter in the digital assistant model on each of the terminal devices; the training result is obtained by training the digital assistant model based on interaction data between the user and the digital assistant model on the corresponding terminal device; the first parameter is a parameter to be trained;

[0010] Based on the training results of each round corresponding to each terminal device, updating the first parameters of the digital assistant model on the cloud device until the digital assistant model on the cloud device meets the training termination condition, thereby obtaining the trained first parameters;

[0011] The trained first parameters are sent to each of the terminal devices, and the trained first parameters are used by the terminal devices to update the first parameters in their own digital assistant models.

[0012] Through federated learning, we leverage real-world user data to train and iterate digital assistant models, ensuring the intelligence and real-time performance of enterprise models and continuously improving the competitiveness of enterprise intelligent services. Dynamically maintaining large cloud models ensures continuous, holistic learning based on the data and preferences of a large number of users, improving overall intelligence and better supporting the diverse needs of diverse users.

[0013] In addition, local users can set the degree of integration between the fine-tuning network of the local large model and the fine-tuning network of the latest large model in the cloud according to their preferences, taking into account the personalization of the user's intelligent service and the generalization of the large model.

[0014] In one possible implementation, the training result includes a parameter adjustment gradient of the first parameter; updating the first parameter of the digital assistant model on the cloud device based on the training result of each round corresponding to each of the terminal devices includes: weighting the parameter adjustment gradient of the first parameter corresponding to each of the terminal devices based on the weight corresponding to each of the terminal devices to obtain a target adjustment gradient; and using the target adjustment gradient to adjust the first parameter of the digital assistant model on the cloud device.

[0015] In one possible implementation, the weight corresponding to each of the terminal devices is related to the amount of interaction data used by the terminal device to train a local digital assistant model.

[0016] In one possible implementation, during the training process, parameters in the digital assistant model other than the first parameter remain unchanged.

[0017] In one possible implementation, before obtaining the training results of each round of the first parameters in the digital assistant model on each of the terminal devices, it also includes: adding a first network structure to the digital assistant model stored in the cloud, and synchronizing the digital assistant model with the changed network structure to each of the terminal devices; wherein the first parameter is the parameter contained in the first network structure.

[0018] In one possible implementation, the interaction data includes: input information of the digital assistant model and feedback information corresponding to the input information, wherein the feedback information represents the user's satisfaction with the digital assistant's response information; the input information includes at least one of the following: user portrait, prompt word template and user question.

[0019] In one possible implementation, the training result is data in an encrypted state; the first parameter of the digital assistant model on the cloud device is updated based on the training result of each round corresponding to each of the terminal devices, including: performing homomorphic encryption operation or semi-homomorphic encryption operation based on the training result of the encrypted state corresponding to each of the terminal devices to obtain a decrypted operation result; the decrypted operation result represents the weighted result of the parameter adjustment gradient of the first parameter corresponding to each of the terminal devices, or the weighted result of the first parameter corresponding to each of the terminals; and updating the first parameter of the digital assistant model on the cloud device according to the decrypted operation result.

[0020] In a second aspect, a training method is provided, which is applied to a terminal device, wherein the terminal device is in communication with a cloud device, and a digital assistant model with the same network structure is deployed on the cloud device and the terminal device, and the method includes:

[0021] Obtaining the training results of each round of the first parameter in the self-deployed digital assistant model; the training results are obtained by training the digital assistant model based on the interaction data between the user and the digital assistant model generated by the terminal device itself;

[0022] Sending the training results of each round of the first parameter to the cloud device; using the training results of each round of the first parameter to update the first parameter in the cloud device's own digital assistant model;

[0023] Receive the trained first parameter sent by the cloud device, and update the first parameter in its own digital assistant model according to the trained first parameter.

[0024] In one possible implementation, receiving the trained first parameter sent by the cloud device and updating the first parameter in the digital assistant model according to the trained first parameter includes:

[0025] Performing a weighted operation based on the trained first parameter and the first parameter in the digital assistant model of the digital assistant itself before the training to obtain an updated first parameter in the digital assistant model of the digital assistant itself;

[0026] Among them, the trained first parameter and the first parameter in the digital assistant model itself before the training correspond to the first weighted weight and the second weighted weight respectively.

[0027] In one possible implementation, the interaction data includes: input information of the digital assistant model and feedback information corresponding to the input information, wherein the feedback information represents the user's satisfaction with the digital assistant's response information; the input information includes at least one of the following: user portrait, prompt word template and user question.

[0028] In a third aspect, a communication device is provided, which includes: a functional unit for executing any one of the methods provided in the first aspect, and the actions performed by each functional unit are implemented by hardware or by hardware executing corresponding software. For example, the communication device may include: a first acquisition unit for acquiring the training results of each round of the first parameter in the digital assistant model on each of the terminal devices; the training results are obtained by training the digital assistant model based on the interaction data between the user and the digital assistant model on the corresponding terminal device; the first parameter is the parameter to be trained; an update unit for updating the first parameter of the digital assistant model on the cloud device based on the training results of each round corresponding to each of the terminal devices, until the digital assistant model on the cloud device meets the training termination condition and obtains the trained first parameter; a first sending unit for sending the trained first parameter to each of the terminal devices, and the trained first parameter is used by the terminal device to update the first parameter in its own digital assistant model.

[0029] In a fourth aspect, a communication device is provided, which includes: a functional unit for executing any one of the methods provided in the second aspect, and the actions performed by each functional unit are implemented by hardware or by hardware executing corresponding software. For example, the communication device may include: a second acquisition unit for acquiring the training results of each round of the first parameter in the digital assistant model deployed by itself; the training results are obtained by training the digital assistant model based on the interaction data between the user and the digital assistant model deployed by itself; the first parameter is the parameter to be trained; a second sending unit for sending the training results of each round of the first parameter to the cloud device; the training results of each round of the first parameter are used by the cloud device to update the first parameter in its own digital assistant model; a receiving unit for receiving the trained first parameter sent by the cloud device, and updating the first parameter in its own digital assistant model according to the trained first parameter.

[0030] In a fifth aspect, a computing device is provided, comprising: a controller and a memory; the controller is coupled to the memory; the memory is used for computer program instructions; and the controller is used to call the computer program instructions in the memory to execute any one of the methods provided in the first aspect or the second aspect.

[0031] In a sixth aspect, a computer-readable storage medium is provided, which stores computer execution instructions. When the computer execution instructions are executed on a computing device, the computing device executes any one of the methods provided in the first or second aspect above.

[0032] In a seventh aspect, a computer program product is provided, comprising: computer execution instructions, which, when executed on a computing device, cause the computing device to execute any one of the methods provided in the first or second aspect above.

[0033] Among them, the technical effects brought about by any implementation method in the second to seventh aspects can refer to the technical effects brought about by different implementation methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the structure of a system architecture provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0036] Figure 3 A flowchart of a training method for a cloud device provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of a training method provided in an embodiment of the present application;

[0038] Figure 5 Another schematic diagram of the training method provided in an embodiment of the present application;

[0039] Figure 6 A flowchart of a training method applied to a terminal device provided in an embodiment of the present application;

[0040] Figure 7 A schematic structural diagram of a first training device provided in an embodiment of the present application;

[0041] Figure 8 A structural diagram of the second training device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0043] In the description of this application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in this application is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural.

[0044] Furthermore, in the description of this application, unless otherwise specified, "plurality" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0045] In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit differences. At the same time, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0046] The following is a brief introduction to the relevant terms involved in the embodiments of this application.

[0047] Federated learning: A distributed machine learning approach that allows multiple parties to collaboratively train a shared model while protecting data privacy without directly sharing data.

[0048] Gradient update: It is an optimization method based on the gradient descent algorithm, which is used to minimize the loss function. It is a method used to optimize model parameters in machine learning and deep learning.

[0049] Semi-homomorphic encryption is a specialized encryption technique that allows specific computational operations to be performed directly on ciphertext without requiring decryption. This encryption method provides a degree of computational preservation, meaning that operations performed on ciphertext produce the same results as the original data after decryption. A key characteristic of semi-homomorphic encryption is that it only supports a limited set of computational operations, hence the term "semi" homomorphic.

[0050] The following is an illustrative introduction to the application scenarios of the embodiments of the present application.

[0051] Predicting user preferences is a crucial step for AI companies to provide personalized services through large models. However, because digital assistant preferences are constantly changing and differ across users and usage scenarios, users cannot enjoy intelligent, personalized digital assistant services using a shared large model.

[0052] In addition, since digital assistants often provide users with relatively private intelligent services, their data is often not suitable for uploading to AI companies for model training, otherwise it will cause privacy and compliance issues. The embodiment of the present application provides a training method, which is applied to cloud devices to obtain the training results of each round of the first parameter in the digital assistant model on each terminal device; the first parameter is the parameter to be trained; based on the training results of each round corresponding to each terminal device, the first parameter of the digital assistant model on the cloud device is updated until the digital assistant model on the cloud device meets the training termination condition, and the trained first parameter is obtained; the trained first parameter is sent to each terminal device, and the trained first parameter is used by the terminal device to update the first parameter in its own digital assistant model.

[0053] Therefore, by combining multiple terminal devices for training and utilizing the real usage data of as many users as possible to train and iterate the large model, it is ensured that the cloud-based large model can conduct continuous and holistic learning based on the data and preferences of a large number of users, thereby improving the overall intelligence level and better supporting the diverse needs of thousands of users.

[0054] In addition, the user can also set the degree of integration between the fine-tuning network of the local large model and the fine-tuning network of the latest large model in the cloud, taking into account the personalization of the user's intelligent service and the generalization of the large model.

[0055] Furthermore, the training method provided in the embodiment of the present application does not involve direct processing of the privacy data of each terminal device, and can ensure the security and confidentiality of user privacy data and historical usage data of each data source.

[0056] The following is an exemplary introduction to the system architecture of the embodiment of the present application.

[0057] See also Figure 1 In the embodiment of the present application, the system architecture includes a cloud device 100 and multiple terminal devices 101, and the cloud device 100 and the multiple terminal devices 101 are communicatively connected.

[0058] In the embodiments of the present application, the cloud device may be a computing device, such as a network device. The network device may include a server, etc. The server may be a single physical or logical server, or may be two or more physical or logical servers sharing different responsibilities and cooperating to implement various server functions.

[0059] For example, the server may be a blade server, a high-density server, a rack server or a tower server, an AI server, etc. The terminal device may include an augmented reality (AR) device, a virtual reality (VR) device, a personal digital assistant (PDA), an ultra-mobile personal computer (UMPC), a tablet computer, a laptop computer, a netbook, a desktop computer, an all-in-one computer, etc.

[0060] It should be noted that the embodiments of the present application do not limit the device forms of cloud devices and terminal devices, and the above is only an exemplary description.

[0061] Figure 2 A schematic diagram of the structure of a computing device provided in an embodiment of the present application.

[0062] Need to explain, Figure 2 The system architecture shown is merely an example and does not constitute a limitation on the system architecture of the computing device provided in the embodiments of the present application.

[0063] In the embodiments of the present application, the computing device may specifically be a network device. The network device may include a server, etc. The server may be a single physical server, or two or more physical servers sharing different responsibilities that collaborate to implement the various server functions. When the computing device comprises multiple servers, a server cluster with high availability may be formed.

[0064] The hardware of the computing device includes a processor, a basic input / output system (BIOS) chip, an out-of-band controller, and memory, while the software mainly includes the BIOS, an out-of-band management module, and an operating system (OS). Figure 2 shown.

[0065] The processor may include a central processing unit (CPU), which includes one or more CPU cores. The CPU's data processing operations are all performed by the CPU cores. The more CPU cores a CPU includes, the faster the data processing speed. In an embodiment of the present application, the processor in the computing device acts as a scheduler to execute the above-mentioned workflow scheduling method.

[0066] The BIOS chip is a chip installed on the motherboard that initializes and detects various hardware components during the computer's startup process. The BIOS chip includes a flash memory area.

[0067] The out-of-band management module is located in the out-of-band controller, and the operating system is located in the processor.

[0068] An out-of-band management module can be a management unit for non-business modules. For example, an out-of-band management module can remotely maintain and manage a computing device through a dedicated data channel. This out-of-band management module is completely independent of the computing device's operating system and can communicate with the BIOS and operating system through the computing device's out-of-band management interface.

[0069] Exemplarily, the out-of-band management module may include a management unit for the computing device's operating status, a management system in a management chip, a baseboard management controller (BMC) for the computing device, a system management module (SMM), etc. It should be noted that the embodiments of the present application do not limit the specific form of the out-of-band management module, and the above description is merely exemplary.

[0070] The operating system (OS) is a computer program that manages and controls the hardware and software resources of a computing device. Any other software must be supported by the OS to run. After a computing device is powered on, the BIOS first performs a series of operations, including self-tests and initialization, and then boots the OS, allowing the user to use the computing device normally.

[0071] BIOS is a set of programs embedded in the BIOS chip on the motherboard of a computing device. The main function of BIOS is to provide the lowest-level and most direct hardware settings and control for the computing device.

[0072] Memory, also known as internal storage or main memory, is installed in memory slots on the motherboard of a computing device.

[0073] It should be noted that the system architecture and application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0074] For ease of understanding, the training method provided in the embodiment of the present application is exemplarily introduced below in combination with the above system architecture and accompanying drawings.

[0075] Figure 3 FIG. 1 is a flow chart illustrating a training method according to an exemplary embodiment. For example, the training method can be applied to a cloud device, and the method steps include the following S301-S303.

[0076] S301: Obtain the training results of each round of the first parameter in the digital assistant model on each terminal device; the training results are obtained by training the digital assistant model based on the interaction data between the user and the digital assistant model on the corresponding terminal device.

[0077] In an embodiment of the present application, a digital assistant model with the same network structure is deployed on the terminal device and the cloud device. The terminal device can download the digital assistant model from the cloud device at a specific time. For example, when the terminal device needs to enable the digital assistant function for the first time, the digital assistant model is downloaded from the cloud device. Alternatively, when a user registers or logs in to the digital assistant on the terminal device for the first time, the digital assistant model is downloaded from the cloud device.

[0078] In the embodiments of the present application, after the digital assistant function is enabled, the terminal device uses its own digital assistant model to process the user's input information to the digital assistant, thereby accumulating sample data for subsequent training. Sample data can be understood as the interaction data between the user and the digital assistant model on the corresponding terminal device. This interaction data is generated by the user and the digital assistant model on the terminal device during interaction and can be stored on the terminal device.

[0079] It should be noted that the digital assistant is based on artificial intelligence and can understand the user's preferences and habits, help the user with scheduling, answer questions raised by the user, etc., and can essentially be regarded as a software program. Since the digital assistant model analyzes and processes the input information during the user's interaction with the digital assistant and outputs the processing results, in the embodiments of the present application, the interaction data between the user and the digital assistant can be equivalent to the interaction data between the user and the digital assistant model.

[0080] In an embodiment of the present application, the digital assistant model can be a large language model (LLM), which can also be called a large model, a natural language model or a large language model. A large language model refers to a natural language processing model with large-scale parameters and training data. The training process of a large language model usually adopts an unsupervised learning method, and the model is trained through a large-scale text corpus to learn the probability distribution and language laws of the language. During the training process, the large language model usually adopts a language model as the objective function, and optimizes the model parameters by maximizing the predicted probability of the next word. For example, the GPT series model based on the Transformer model structure is trained on a large-scale corpus and can generate high-quality natural language texts, such as articles, conversations, etc.

[0081] In the embodiment of the present application, a cloud device collaborates with multiple terminal devices to train a digital assistant model. Each terminal device corresponds to a different training user, but the parameters that need to be trained in the digital assistant model of each terminal device are the same. Therefore, this can be considered a federated learning task with different users but the same feature dimensions. This type of federated learning task is called horizontal federated learning. Therefore, in the embodiment of the present application, the cloud device collaborates with multiple terminal devices to train a digital assistant model, which can be considered horizontal federated learning.

[0082] In some embodiments of the present application, when the cloud device detects that the conditions for initiating federated learning are met, it instructs the terminal device to start federated learning.

[0083] As an example, the cloud device detects the number of online terminal devices and determines that the conditions for initiating federated learning are met if the number of online terminal devices reaches a first threshold. The first threshold can be set as needed, for example, to 100.

[0084] As another example, the cloud device detects the total amount of sample data accumulated by each terminal device that can be used for training. When the total amount reaches a second threshold, it determines that the conditions for initiating federated learning are met. Specifically, when the amount of sample data accumulated by a single terminal device reaches a third threshold, the sample data can be considered available for training. That is, for a terminal device whose sample data amount is less than the third threshold, the sample data amount of the terminal device can be excluded from the above-mentioned total amount of accumulated sample data. The above-mentioned second and third thresholds can be set as needed. For example, the second threshold is set to 1000 and the third threshold is set to 100.

[0085] As another example, the cloud device detects the current number of online terminal devices and the total amount of sample data accumulated by each terminal device that can be used for training. When the total number of online devices and the total amount of sample data reach the corresponding thresholds, it is determined that the conditions for initiating federated learning are met.

[0086] When it is determined that the conditions for initiating federated learning are met, the cloud device sends an enable instruction to the terminal device participating in the federated learning, instructing the terminal device to enable federated learning.

[0087] The terminal device that receives the power-on command converts the local sample data, performs fine-tuning training of the local digital assistant model, and uploads the training results to the cloud device.

[0088] The training results are obtained by training the digital assistant model based on the interaction data between the user and the digital assistant model on the corresponding terminal device. That is, the terminal device uses the interaction data generated by the user and the digital assistant model to train its own digital assistant model and obtain the training results.

[0089] In the embodiment of the present application, the cloud device can indicate the parameter to be trained during the federated learning process, which is recorded as the first parameter for ease of description, that is, the first parameter is the parameter to be trained. The cloud device notifies the terminal device to train the first parameter during the local training process.

[0090] As an example, the activation instruction for instructing the terminal device to activate federated learning carries the parameter name of the first parameter to be trained. Thus, during the local training process, the terminal device fixes all parameters in the local digital assistant model except the first parameter and only trains the first parameter.

[0091] As another example, the above-mentioned start instruction carries the identifier of a specific network layer of the digital assistant model, which is used to indicate that the parameters to be trained are the parameters contained in the specific network layer.

[0092] It should be noted that the above is only an exemplary description of indicating the first parameter, and in the embodiment of the present application, other methods may be used to indicate the first parameter to be trained. This is not limited to this.

[0093] After completing a round of training, the terminal device sends the training results to the cloud device. In the embodiment of the present application, a round of training is equivalent to one iteration, and the amount of sample data used by the terminal device for a round of training is not limited. The amount of sample data used by each terminal device for a round of training can also be different.

[0094] In some embodiments of the present application, the cloud device can pre-set the minimum number of epochs. Traditional deep learning model training can generally be set to 5-10 times. Generally speaking, the more training data, the more iterations are required. In the federated learning provided in the embodiments of the present application, the more sample data the terminal devices participating in the federated learning have, the more minimum iterations should be set. However, the more iterations there are, the greater the communication overhead between the cloud device and the terminal device.

[0095] In some embodiments of the present application, the minimum number of iterations epoch can be set according to the following formula:

[0096]

[0097] Among them, n represents the number of terminal devices participating in this federated learning. = represents rounding up, and us[u] represents the number of sample data from the terminal device labeled u (i.e., user sample). Furthermore, since fine-tuning training is based on a large model, for high learning depth, a minimum number of iterations β can be set, for example, to at least 20. This balances training scale and efficiency.

[0098] S302: Based on the training results of each round corresponding to each terminal device, update the first parameter of the digital assistant model on the cloud device until the digital assistant model on the cloud device meets the training termination condition, and obtains the trained first parameter.

[0099] In some embodiments of the present application, the training results sent by the terminal device to the cloud device may be the first parameters obtained in this round of training, that is, the changed parameter values of each first parameter.

[0100] Accordingly, the cloud device updates the first parameter of the digital assistant model on the cloud device in combination with the first parameter sent by each terminal device. As an example, for each first parameter, the parameter values sent by each terminal are averaged as the parameter value of the first parameter.

[0101] In other embodiments of the present application, the training result sent by the terminal device to the cloud device may be a parameter adjustment gradient of the first parameter.

[0102] Specifically, during the process of training the local digital assistant model, the terminal device will obtain the gradient data of the first parameter, that is, the parameter adjustment gradient, in each round of iterative training.

[0103] Accordingly, the cloud device summarizes the parameter adjustment gradients of each iteration round reported by the terminal device, and updates the first parameter according to the summarized parameter adjustment gradients.

[0104] When the digital assistant model on the cloud device meets the training termination condition, the cloud device determines the current first parameter as the trained first parameter.

[0105] The present embodiment does not specifically limit the training termination condition. It can be that the number of iterations reaches a threshold, such as 50, or the average rate of change of the parameter over multiple consecutive iterations is less than a specific threshold, such as 1 / 100.

[0106] As an example, after federated learning enters a certain number of iterations (for example, 20), the cloud device begins to calculate the parameter deviation between two adjacent iterations. This can be calculated based on the Euclidean distance, as shown in the following formula:

[0107]

[0108] Among them, ∥(θ all[i] -θ all[i-1] )∥2 function is used to calculate θ all[i] Matrix and θ all[i-1] The distance between matrices can quantify the degree of change of model parameters after two iterations. θn is θ all[i] and θ all[i-1] The number of network parameters. The result is a decimal, not a matrix. If this distance is small, it means that the model parameters have not changed much after two iterations; if this distance is large, it means that the model parameters have changed a lot after two iterations. all[i] Represents the set of summary values of the first parameter after the i-th iteration, which can be in the form of a matrix. all[i-1] Represents the set of summary values of the first parameter after the i-1th round of iteration. 偏差[i] If the value is less than a certain threshold for multiple consecutive times, for example, less than 0.0001 for two consecutive times, it can be determined that the digital assistant model meets the training termination conditions, that is, the model training of this federated learning is considered to have converged.

[0109] See also Figure 4 , is a schematic diagram of the training method provided in the embodiment of the present application, such as Figure 4 As shown, each terminal device uses the user portrait and the interaction data between the user and the digital assistant model generated by itself as input to train the local digital assistant model. During the local training process of the terminal device, part of the network structure is frozen and only the first parameter is trained.

[0110] Freezing a network structure means fixing the parameters contained in the network structure. During the training of the digital assistant model, only the parameters of the digital assistant model that do not belong to the frozen network structure are trained. These parameters are the first parameters. In other words, the first parameters can be understood as the parameters contained in the network structure other than the frozen network structure in the digital assistant model.

[0111] After completing a round of training, the terminal device uploads the training results to the cloud device. The cloud device combines the training results of each terminal device to update the cloud digital assistant model.

[0112] S303: Send the trained first parameters to each terminal device. The trained first parameters are used by the terminal device to update the first parameters in its own digital assistant model.

[0113] The cloud device sends the trained first parameters to each terminal device. Each terminal device receives the trained first parameters, determines that this federated learning is completed, and then updates the first parameters in its own digital assistant model based on the trained first parameters.

[0114] In some embodiments of the present application, when a terminal device updates its own digital assistant model, it can set the degree of integration between the fine-tuning network of the local large model and the fine-tuning network of the latest large model in the cloud. The fine-tuning network can be understood as the network structure involved in the first parameter.

[0115] As an example, after determining that this federated learning is complete, the cloud device determines the first parameter θ that has been trained. all_new ,θ all_new It can be regarded as a collection of summary values of the first parameter after training. all_new Sent to each terminal device participating in this federated learning.

[0116] The terminal device performs a weighted operation based on the trained first parameter and the first parameter in its own digital assistant model before this training to obtain the updated first parameter in its own digital assistant model; wherein the trained first parameter and the first parameter in its own digital assistant model before this training correspond to the first weighted weight and the second weighted weight respectively.

[0117] Exemplarily, the terminal device updates the first parameter of its own digital assistant model using the following formula:

[0118] θ 用户new[u] =a*θ 用户[u] +(1-a)*θ all_new

[0119] Among them, θ 用户new[u] represents the first parameter updated in the digital assistant model of the terminal device labeled u, θ 用户[u] represents the first parameter in the digital assistant model of the terminal device labeled u before this federated learning, θ all_new This is the first parameter of the federated learning training, that is, the first parameter of the training sent from the cloud device to the terminal device. a is a hyperparameter less than 1 and can be understood as a weight.

[0120] By setting the value of the hyperparameter, the terminal device can change the weighted weights corresponding to the trained first parameter and the first parameter in its own digital assistant model before this training, thereby setting the degree of integration between the fine-tuning network of the local large model and the fine-tuning network of the latest large model in the cloud.

[0121] As an example, the terminal device can set the degree of integration of the model parameters of the cloud device, such as setting three levels: high, medium, and low.

[0122] A high level means that users are more inclined to experience the latest and more popular intelligent services. In this case, a can take values such as 0.4, 0.5, etc.

[0123] A low level means that users are more inclined to experience the original, more personalized intelligent services. In this case, a can take values such as 0.8 and 0.9.

[0124] The mid-range represents that users prefer the latest and personalized intelligent services. In this case, a can take values such as 0.7, 0.6, etc.

[0125] See also Figure 5 , is another schematic diagram of the training method provided in the embodiment of the present application, such as Figure 5 As shown, after determining that this federated learning is completed, the cloud device sends the trained first parameter to each terminal device. The terminal device combines the received first parameter and the first parameter in its own digital assistant model before this training to update the first parameter in its own digital assistant model.

[0126] It can be seen that the training method provided in the embodiment of the present application is that the cloud device jointly trains the digital assistant model with multiple terminal devices, and uses the real usage data of the digital assistants of multiple users to train and iterate the model.

[0127] Through federated learning, we leverage real-world usage data from as many users as possible to train and iterate large models (digital assistant models), ensuring the intelligence and real-time performance of enterprise large models and continuously improving the competitiveness of enterprise intelligent services. When dynamically maintaining large cloud models, we ensure that they can continuously and holistically learn based on the data and preferences of a large number of users, improving their overall intelligence and better supporting the diverse needs of thousands of users.

[0128] In addition, the user's local terminal (terminal device) can set the degree of integration between the local large model's fine-tuning network and the latest large model's fine-tuning network in the cloud according to their preferences, taking into account the user's intelligent service personalization and the generalization of the large model.

[0129] In some embodiments of the present application, when the training results of each round reported by the terminal device include the parameter adjustment gradient of the first parameter, the cloud device updates the first parameter of the digital assistant model on the cloud device based on the training results of each round corresponding to each terminal device, specifically including:

[0130] Based on the weight corresponding to each terminal device, the parameter adjustment gradient of the first parameter corresponding to each terminal device is weighted to obtain a target adjustment gradient; and the target adjustment gradient is used to adjust the first parameter of the digital assistant model on the cloud device.

[0131] By weighting the training results of each terminal device, the rationality and refinement of model training can be improved, and it is also beneficial for large models to balance the training results of each terminal device according to needs.

[0132] Specifically, considering that the amount of interaction data used by different terminal devices when performing local training may be different, when updating the first parameter of the digital assistant model on the cloud device based on the parameter adjustment gradient reported by each terminal device, the parameter adjustment gradient of the first parameter corresponding to each terminal device can be weighted, and the weighted weight can be related to the amount of interaction data used by the terminal device when performing local training.

[0133] In some embodiments of the present application, the weighted weight may be positively correlated with the amount of interaction data used by the terminal device when performing local training. Specifically, for terminal devices that use a large amount of data for a single round of iterative training, the digital assistant model processes more training samples (interaction data between the user and the digital assistant model) during this round of iteration, and more training samples can reduce the impact of noise on the model. Noise can be understood as errors or inconsistencies generated during the data collection and annotation process. With more training samples, these noises can be better identified and filtered, thereby improving the accuracy of the model. Based on this, the weighted weight can be set to be positively correlated with the amount of interaction data used by the terminal device when performing local training.

[0134] In some embodiments of the present application, the weighted weight may be negatively correlated with the amount of interaction data used by the terminal device when performing local training. Specifically, in the application scenario of a digital assistant, the training samples of a single terminal device usually come from the same user. Therefore, in order to avoid over-considering the behavioral preferences of a single user and negatively affecting the overall training, for terminal devices with more training samples, the corresponding training results can be set with smaller weights. By appropriately reducing the weights of users with a large amount of training data, it is prevented that the large model is overly biased towards users with a large amount of training data.

[0135] As an example, the cloud device may adjust the first parameter of the digital assistant model based on the following formula:

[0136]

[0137] Among them, θ all [i] represents the set of parameter summary values of the first parameter after the i-th round of iterative learning, θ all [i-1] represents the set of parameter summary values of the first parameter after the i-1th round of iterative learning, represents the set of parameter adjustment gradients corresponding to the first parameter after the i-th round of iterative learning of the user labeled u; us[u] represents the number of sample data of the terminal device of the user labeled u (user sample).

[0138] For cloud devices, if i=1, θ all [i-1] is the set of parameter summary values of the cloud model trained by the last federated learning.

[0139] If this federated learning is the first federated learning performed by the cloud device, the initial value of the model training of the cloud device can be a random value (for example, a value of 0.1-0.2).

[0140] As can be seen, adjusting the first parameter of the cloud device's digital assistant model based on the above formula can appropriately downweight devices with larger amounts of training data, preventing the large model from being overly biased towards those devices with larger amounts of training data. This allows for better support for diverse user needs.

[0141] After the completion of this federated learning training iteration, the fine-tuning network parameters (first parameter) of the large model of the cloud device are θ all_new , according to the training logic, θ all_new It can be approximately expressed as:

[0142]

[0143] Among them, θ all [0] represents the aggregated values of the fine-tuned network parameters of the large model on the cloud device before this federated learning, θ 用户 [u] represents the set of summary values of the corresponding fine-tuning network parameters after the terminal device of the user labeled u performs local training.

[0144] Through federated learning, we can leverage real user data from as many users as possible to train and iterate large models, ensuring the intelligence and real-time performance of enterprise large models and continuously improving the competitiveness of enterprise intelligent services. Dynamically maintaining large cloud models ensures continuous and holistic learning based on the data and preferences of a large number of users, improving overall intelligence.

[0145] In some embodiments of the present application, the interaction data between the user and the digital assistant model includes: input information of the digital assistant model and feedback information corresponding to the input information, the feedback information represents the user's satisfaction with the response information of the digital assistant; the input information includes at least one of the following: user portrait, prompt word template and user question.

[0146] Specifically, the data used to train the digital assistant model consists of two parts. One part is user feedback on the digital assistant's generated content, such as likes and dislikes. This is equivalent to a label that evaluates the quality of the digital assistant's output. This label serves as the label data during training and is the content that the fine-tuning network needs to learn.

[0147] The other part is the user's interactive input, which may include user portraits, prompt templates, and questions asked by users. User portraits may include occupation, age, gender, interests and hobbies, and the city where they reside.

[0148] The following is an exemplary illustration of the interaction data between the user and the digital assistant model.

[0149] For example, the user profile for "Zhang San" includes: 30 years old, thin, a technology enthusiast, and a fan of the latest electronic gadgets. When using the digital assistant model, user "Zhang San" asks, "Hey, xx, turn on the living room lights." After processing, the digital assistant model responds, "OK, the living room lights are on." User "Zhang San" then likes this answer.

[0150] For example, the user "Zhang San" asked the digital assistant model, "I'd like some advice on a healthy diet." The digital assistant model processed the answer and said, "Establish a balanced diet and reduce carbohydrate intake." Since Zhang San is thin, he should increase his intake of meat and sugar. This answer was inappropriate, and the user "Zhang San" downvoted it.

[0151] For example, when using the digital assistant model, user Zhang San asked, "I'm planning a trip to xx next week. Do you have any recommendations?" The digital assistant model processed the answer and replied, "I recommend visiting the ancient temples and tasting the local matcha in xx." User Zhang San then liked the answer.

[0152] For example, the user profile for "Li Si" includes: 28 years old, music producer, and a lover of music creation and the latest music equipment. While using the digital assistant model, user "Li Si" asked, "I need some inspiration to create an electronic dance track." The digital assistant model processed the response, "I've generated some electronic dance music rhythm and melody samples for you. I hope you find them helpful." User "Li Si" then liked the response.

[0153] For example, when using the digital assistant model, user Li Si asked, "I'm looking to buy the latest music production software. Any recommendations?" The digital assistant model processed the answer and replied, "Based on your needs, I recommend xxx. It's powerful and highly rated." User Li Si then gave a thumbs-up to the answer.

[0154] For example, the user "Li Si" asked the digital assistant model, "Are there any music events worth attending recently?" The digital assistant model processed the answer and said, "Based on your preferences, I recommend attending the electronic music festival next month. There will be many exciting performances." Because this answer didn't specify a specific time or location, it was inappropriate. The user "Li Si" downvoted it.

[0155] For example, the user profile for "Wang Wu" includes: 45 years old, businessman, and interested in market trends and investment opportunities. When using the digital assistant model, user "Wang Wu" asked, "Give me an analysis of recent stock market trends." The digital assistant model processed the answer, "Currently, xx stock is performing best, but others require consideration of risks." User "Wang Wu" then liked the answer.

[0156] For example, the user Wang Wu, while using the digital assistant model, asked, "Please help me schedule a business meeting next week." The digital assistant model processed the response, "Okay, we've scheduled the meeting for you and sent the schedule to your email." Because the digital assistant model didn't ask about the meeting time, attendees, or purpose, this response was inappropriate. User Wang Wu downvoted the response.

[0157] For example, the user Wang Wu, while using the digital assistant model, asked, "I'm interested in the xx field. Are there any investment opportunities?" The digital assistant model processed the answer and replied, "Based on market analysis, we recommend you pay attention to xx, a leading company in the xx field." The user Wang Wu then liked the answer.

[0158] In the above example, the user's portrait, questions asked by the user, and feedback on the results generated by the digital assistant model can be used as interaction data to train the local digital assistant model of the terminal device.

[0159] As can be seen, in the embodiment of the present application, the input information of the digital assistant model and the feedback information corresponding to the input information are used as training data to train the digital assistant model. Since the input information and the corresponding feedback information of the digital assistant model are continuously accumulated, it is possible to ensure that the large model can be continuously updated. When the cloud-based large model is dynamically maintained, it is ensured that the cloud-based large model can be holistically learned based on the data and preferences of all users.

[0160] In addition, the interaction data between users and the digital assistant model is only stored on the terminal device. During the federated learning process, the terminal device only needs to perform local training based on the interaction data generated by itself and send the training results to the cloud device. There is no need to transmit the interaction data externally, which reduces privacy risks.

[0161] In some embodiments of the present application, the training results sent by the terminal device are encrypted data; accordingly, based on the training results of each round corresponding to each terminal device, updating the first parameter of the digital assistant model on the cloud device specifically includes:

[0162] A homomorphic encryption operation or a semi-homomorphic encryption operation is performed based on the training results of the encryption state corresponding to each terminal device, and the operation result is decrypted to obtain a decrypted operation result. The decrypted operation result represents a weighted result of the parameter adjustment gradient of the first parameter corresponding to each terminal device, or a weighted result of the first parameter corresponding to each terminal device. The first parameter of the digital assistant model on the cloud device is updated based on the decrypted operation result.

[0163] In the case where the training result sent by the terminal device is the first parameter after training, or in the case where the training result sent by the terminal device is the parameter adjustment gradient of the first parameter, the sent training results are all encrypted data.

[0164] Encryption can be based on homomorphic encryption algorithms or semi-homomorphic encryption algorithms. Cloud devices can calculate the ciphertext without decryption and obtain the correct result.

[0165] Specifically, the cloud device receives the encrypted training results sent by each terminal device, performs homomorphic encryption operation or semi-homomorphic encryption operation on the training results without decrypting the training results, and only decrypts the final operation result to update the first parameter.

[0166] In this way, the encrypted training results can be calculated and processed while protecting the data privacy of each terminal device.

[0167] As can be seen, in the embodiments of this application, the use of federated learning for joint fine-tuning of a large model among multiple users enables the pooling of collective wisdom, while maintaining legal and privacy compliance, to train and iterate the large model using the real usage data of multiple users' digital assistants. This enables the joint training of the large model by multiple parties while ensuring the security and confidentiality of user privacy data and historical usage data from each data source.

[0168] In some embodiments of the present application, before obtaining the training results of each round of the first parameter in the digital assistant model on each terminal device, it also includes: adding a first network structure to the digital assistant model stored in the cloud, and synchronizing the digital assistant model with the changed network structure to each terminal device; wherein the first parameter is the parameter contained in the first network structure.

[0169] It should be noted that the digital assistant model deployed on cloud devices can be understood as a large model that has been pre-trained by the system. Each time federated learning is enabled, it can be regarded as additional training for the current digital assistant model.

[0170] As an implementation method, the cloud device can add a first network structure to the digital assistant model stored in its own storage. The number of layers of the first network structure can be set as needed, for example, to a 10-20 layer structure.

[0171] Before enabling federated learning, the network structure of the digital assistant model stored in the cloud and the digital assistant model on the terminal device must be identical. Therefore, after the cloud device changes the digital assistant model, it must synchronize the modified digital assistant model with the terminal device. Accordingly, the first parameters to be trained in this federated learning are the parameters contained in the newly added first network structure.

[0172] It can be seen that in the embodiment of the present application, large model fine-tuning technology and federated learning are combined. Before starting federated learning, the cloud device adds a new network structure to the current digital assistant model and synchronizes it to each terminal device participating in federated learning. During the training process, each terminal device is instructed to freeze all parameters in the digital assistant model except for the parameters involved in the newly added network structure. By adding a new network structure, it is convenient to indicate the first parameter that needs to be trained, and the size of the newly added network structure can be set according to demand, which is suitable for different degrees of model iteration.

[0173] This demonstrates that, without significantly increasing model parameters, the digital assistant model can continuously and holistically learn based on the latest data and preferences of a large number of users, improving its overall intelligence and better supporting the diverse needs of thousands of users. Furthermore, since a single federated learning session requires fewer parameters to train, it also improves training efficiency and convergence speed.

[0174] See also Figure 6 The present application also provides a training method for a terminal device, wherein the terminal device is in communication with a cloud device, and digital assistant models with the same structure are deployed on the cloud device and the terminal device. The method includes the following steps:

[0175] S601: Obtain the training results of each round of the first parameter in the self-deployed digital assistant model; the training results are obtained by training the digital assistant model based on the interaction data between the user and the self-deployed digital assistant model; the first parameter is the parameter to be trained.

[0176] S602: Send the training results of each round of the first parameter to the cloud device; the training results of each round of the first parameter are used by the cloud device to update the first parameter in its own digital assistant model.

[0177] S603: Receive the trained first parameter sent by the cloud device, and update the first parameter in its own digital assistant model according to the trained first parameter.

[0178] The digital assistant model deployed on the terminal device is trained using the user interaction data generated by the terminal device itself. The terminal device obtains the training results of each round of the first parameter in the digital assistant model on the terminal device and sends the training results to the cloud device.

[0179] The cloud device updates the digital assistant model on the cloud based on the training results of each terminal device. The cloud device sends the trained first parameters to each terminal device. Each terminal device receives the trained first parameters, determines that this federated learning has been completed, and then updates the first parameters in its own digital assistant model based on the trained first parameters.

[0180] It can be seen that the training method provided in the embodiment of the present application is that the cloud device jointly trains the digital assistant model with multiple terminal devices, and uses the real usage data of the digital assistants of multiple users to train and iterate the model.

[0181] Through federated learning, we leverage real-world usage data from as many users as possible to train and iterate large models (digital assistant models), ensuring the intelligence and real-time performance of enterprise large models and continuously improving the competitiveness of enterprise intelligent services. When dynamically maintaining large cloud models, we ensure that they can continuously and holistically learn based on the data and preferences of a large number of users, improving their overall intelligence and better supporting the diverse needs of thousands of users.

[0182] In addition, the user's local terminal (terminal device) can set the degree of integration between the local large model's fine-tuning network and the latest large model's fine-tuning network in the cloud according to their preferences, taking into account the user's intelligent service personalization and the generalization of the large model.

[0183] In some embodiments of the present application, receiving a trained first parameter sent by a cloud device and updating the first parameter in the digital assistant model according to the trained first parameter includes:

[0184] Perform a weighted operation based on the trained first parameter and the first parameter in the digital assistant model of the digital assistant itself before the training to obtain an updated first parameter in the digital assistant model of the digital assistant itself;

[0185] Among them, the trained first parameter and the first parameter in the digital assistant model itself before this training correspond to the first weighted weight and the second weighted weight respectively.

[0186] Specifically, when the terminal device updates its own digital assistant model, it can set the degree of integration between the fine-tuning network of the local large model and the fine-tuning network of the latest large model in the cloud. The fine-tuning network can be understood as the network structure involved in the first parameter.

[0187] The terminal device performs a weighted operation based on the trained first parameter and the first parameter in its own digital assistant model before this training to obtain the updated first parameter in its own digital assistant model; wherein the trained first parameter and the first parameter in its own digital assistant model before this training correspond to the first weighted weight and the second weighted weight respectively.

[0188] Exemplarily, the terminal device updates the first parameter of its own digital assistant model using the following formula:

[0189] θ 用户new[u] =a*θ 用户[u] +(1-a)*θ all_new

[0190] Among them, θ 用户new[u] represents the first parameter updated in the digital assistant model of the terminal device labeled u, θ 用户[u] represents the first parameter in the digital assistant model of the terminal device labeled u before this federated learning, θ all_new The first parameter completed for this federated learning training, that is, the first parameter sent by the cloud device to the terminal device. a is a hyperparameter less than 1 and can be understood as a weight.

[0191] By setting the value of the hyperparameter, the terminal device can change the weighted weights corresponding to the trained first parameter and the first parameter in its own digital assistant model before this training, thereby setting the degree of integration between the fine-tuning network of the local large model and the fine-tuning network of the latest large model in the cloud.

[0192] As an example, the terminal device can set the degree of integration of the model parameters of the cloud device, such as setting three levels: high, medium, and low.

[0193] A high level means that users are more inclined to experience the latest and more popular intelligent services. In this case, a can take values such as 0.4, 0.5, etc.

[0194] A low level means that users are more inclined to experience the original, more personalized intelligent services. In this case, a can take values such as 0.8 and 0.9.

[0195] The mid-range represents that users prefer the latest and personalized intelligent services. In this case, a can take values such as 0.7, 0.6, etc.

[0196] It can be seen that the training method provided in the embodiment of the present application allows the user's local end (terminal device) to set the degree of integration of the fine-tuning network of the local large model and the fine-tuning network of the latest large model in the cloud according to his or her own preferences, taking into account the user's intelligent service personalization and the generalization of the large model.

[0197] In some embodiments of the present application, the interaction data includes: input information of the digital assistant model and feedback information corresponding to the input information, the feedback information represents the user's satisfaction with the digital assistant's response information; the input information includes at least one of the following: user portrait, prompt word template and user question.

[0198] Specifically, the data used to train the digital assistant model consists of two parts. One part is user feedback on the digital assistant's generated content, such as likes and dislikes. This is equivalent to a label that evaluates the quality of the digital assistant's output. This label serves as the label data during training and is the content that the fine-tuning network needs to learn.

[0199] The other part is the user's interactive input, which may include user portraits, prompt templates, and questions asked by users. User portraits may include occupation, age, gender, interests and hobbies, and the city where they reside.

[0200] As can be seen, in the embodiment of the present application, the input information of the digital assistant model and the feedback information corresponding to the input information are used as training data to train the digital assistant model. Since the input information and the corresponding feedback information of the digital assistant model are continuously accumulated, it is possible to ensure that the large model can be continuously updated. When the cloud-based large model is dynamically maintained, it is ensured that the cloud-based large model can be holistically learned based on the data and preferences of all users.

[0201] In addition, the interaction data between users and the digital assistant model is only stored on the terminal device. During the federated learning process, the terminal device only needs to perform local training based on the interaction data generated by itself and send the training results to the cloud device. There is no need to transmit the interaction data externally, which reduces privacy risks.

[0202] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, the training device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0203] In the embodiments of the present application, the functional modules of the training device can be divided according to the above method. For example, the training device can include functional modules corresponding to the functional divisions, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical functional division. In actual implementation, other division methods may be used.

[0204] For example, Figure 7A possible schematic diagram of the first training device involved in the above embodiment is shown. The first training device 700 may include a first acquisition unit 701, an update unit 702, and a first sending unit 703. The first acquisition unit 701 is used to obtain the training results of each round of the first parameter in the digital assistant model on each of the terminal devices; the training results are obtained by training the digital assistant model based on the interaction data between the user and the digital assistant model on the corresponding terminal device; the first parameter is the parameter to be trained; the update unit 702 is used to update the first parameter of the digital assistant model on the cloud device based on the training results of each round corresponding to each of the terminal devices, until the digital assistant model on the cloud device meets the training termination condition and obtains the trained first parameter; the first sending unit 703 is used to send the trained first parameter to each of the terminal devices, and the trained first parameter is used by the terminal device to update the first parameter in its own digital assistant model.

[0205] Optionally, the training result includes a parameter adjustment gradient of the first parameter; the update unit 702 is specifically used to: weight the parameter adjustment gradient of the first parameter corresponding to each terminal device based on the weight corresponding to each terminal device to obtain a target adjustment gradient; and use the target adjustment gradient to adjust the first parameter of the digital assistant model on the cloud device.

[0206] Optionally, the weight corresponding to each of the terminal devices is related to the amount of interaction data used by the terminal device to train the local digital assistant model.

[0207] Optionally, during the training process, parameters in the digital assistant model except the first parameter remain unchanged.

[0208] Optionally, a new module is also included for adding a first network structure to the digital assistant model stored in the cloud, and synchronizing the digital assistant model with the changed network structure to each of the terminal devices; wherein the first parameter is the parameter contained in the first network structure.

[0209] Optionally, the interaction data includes: input information of the digital assistant model and feedback information corresponding to the input information, the feedback information represents the user's satisfaction with the digital assistant's response information; the input information includes at least one of the following: user portrait, prompt word template and user question.

[0210] Optionally, the training result is encrypted data;

[0211] The update unit is specifically used to: perform homomorphic encryption operations or semi-homomorphic encryption operations based on the training results of the encryption states corresponding to each of the terminal devices, decrypt the operation results, and obtain decrypted operation results; the decrypted operation results represent the weighted results of the parameter adjustment gradients of the first parameters corresponding to each of the terminal devices, or the weighted results of the first parameters corresponding to each of the terminals; and update the first parameters of the digital assistant model on the cloud device according to the decrypted operation results.

[0212] For example, Figure 8 A possible schematic diagram of the second training device involved in the above embodiment is shown, and the second training device 800 may include a second acquisition unit 801, a second sending unit 802 and a receiving unit 803. Among them, the second acquisition unit 801 is used to obtain the training results of each round of the first parameter in the digital assistant model deployed by itself; the training results are obtained by training the digital assistant model based on the interaction data between the user and the digital assistant model deployed by itself; the first parameter is the parameter to be trained; the second sending unit 802 is used to send the training results of each round of the first parameter to the cloud device; the training results of each round of the first parameter are used by the cloud device to update the first parameter in its own digital assistant model; the receiving unit 803 is used to receive the trained first parameter sent by the cloud device, and update the first parameter in its own digital assistant model according to the trained first parameter.

[0213] Optionally, the receiving unit 803 is specifically used to: perform a weighted operation based on the trained first parameter and the first parameter in its own digital assistant model before this training to obtain the updated first parameter in its own digital assistant model; wherein the trained first parameter and the first parameter in its own digital assistant model before this training correspond to the first weighted weight and the second weighted weight, respectively.

[0214] Optionally, the interaction data includes: input information of the digital assistant model and feedback information corresponding to the input information, the feedback information represents the user's satisfaction with the digital assistant's response information; the input information includes at least one of the following: user portrait, prompt word template and user question.

[0215] An embodiment of the present application also provides a computing device, comprising: a controller and a memory; the controller is coupled to the memory; the memory is used for computer program instructions; and the controller is used to call the computer program instructions in the memory to execute any one of the methods in the above embodiments.

[0216] An embodiment of the present application further provides a computer-readable storage medium storing computer execution instructions. When the computer execution instructions are executed on a computing device, the computing device executes any one of the methods in the above embodiments.

[0217] For explanations of the relevant contents and descriptions of the beneficial effects of any of the computer-readable storage media provided above, reference may be made to the corresponding embodiments described above, and no further details will be given here.

[0218] The embodiment of the present application also provides a computer program product comprising instructions, which, when run on a computing device, causes the computing device to perform any one of the methods in the above embodiments. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. It should be noted that the above-mentioned devices for storing computer instructions or computer programs provided in the embodiment of the present application, such as but not limited to the above-mentioned memory, computer-readable storage medium and communication chip, etc., are all non-transitory.

[0219] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using a software program, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0220] Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0221] Although the present application has been described with reference to specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely illustrative of the present application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, the present application is intended to include such modifications and variations as fall within the scope of the claims of the present application and their equivalents.

Claims

1. A training method based on federated learning, characterized in that: Applied to a cloud device, the cloud device is communicatively connected to a plurality of terminal devices, and a digital assistant model with the same network structure is deployed on the cloud device and the terminal devices. The method includes: Obtaining a training result of each round of a first parameter in the digital assistant model on each of the terminal devices; the training result is obtained by training the digital assistant model based on interaction data between the user and the digital assistant model on the corresponding terminal device; the first parameter is a parameter to be trained; Based on the training results of each round corresponding to each terminal device, updating the first parameters of the digital assistant model on the cloud device until the digital assistant model on the cloud device meets the training termination condition, thereby obtaining the trained first parameters; The trained first parameters are sent to each of the terminal devices, and the trained first parameters are used by the terminal devices to update the first parameters in their own digital assistant models.

2. The method according to claim 1, characterized in that The training result includes a parameter adjustment gradient of the first parameter; The updating of the first parameter of the digital assistant model on the cloud device based on the training result of each round corresponding to each terminal device includes: Based on the weight corresponding to each of the terminal devices, weighting the parameter adjustment gradient of the first parameter corresponding to each of the terminal devices to obtain a target adjustment gradient; The target adjustment gradient is used to adjust the first parameter of the digital assistant model on the cloud device.

3. The method according to claim 2, characterized in that The weight corresponding to each of the terminal devices is related to the amount of interaction data used by the terminal device to train the local digital assistant model.

4. The method according to claim 1, wherein During the training process, the parameters in the digital assistant model except the first parameter remain unchanged.

5. The method according to claim 1, wherein Before obtaining the training results of each round of the first parameter in the digital assistant model on each terminal device, the method further includes: A first network structure is added to the digital assistant model stored in the cloud device, and the digital assistant model with the changed network structure is synchronized to each of the terminal devices; wherein the first parameter is a parameter contained in the first network structure.

6. The method according to claim 1, characterized in that The interaction data includes: Input information of the digital assistant model and feedback information corresponding to the input information, wherein the feedback information represents the user's satisfaction with the digital assistant's response information; the input information includes at least one of the following: user portrait, prompt word template and user question.

7. The method according to claim 2, characterized in that The training result is data in an encrypted state; The updating of the first parameter of the digital assistant model on the cloud device based on the training result of each round corresponding to each terminal device includes: Performing a homomorphic encryption operation or a semi-homomorphic encryption operation based on the training result of the encryption state corresponding to each of the terminal devices, decrypting the operation result, and obtaining a decrypted operation result; the decrypted operation result represents a weighted result of the parameter adjustment gradient of the first parameter corresponding to each of the terminal devices, or a weighted result of the first parameter corresponding to each of the terminals; Update the first parameter of the digital assistant model on the cloud device according to the decrypted operation result.

8. A training method, characterized in that: Applied to a terminal device, the terminal device is communicatively connected to a cloud device, and a digital assistant model with the same network structure is deployed on the cloud device and the terminal device, the method includes: Obtaining the training results of each round of a first parameter in the self-deployed digital assistant model; the training results are obtained by training the digital assistant model based on interaction data between the user and the self-deployed digital assistant model; the first parameter is the parameter to be trained; Sending the training results of each round of the first parameter to the cloud device; using the training results of each round of the first parameter to update the first parameter in the cloud device's own digital assistant model; Receive the trained first parameter sent by the cloud device, and update the first parameter in its own digital assistant model according to the trained first parameter.

9. The method according to claim 8, characterized in that The receiving the trained first parameter sent by the cloud device and updating the first parameter in the digital assistant model according to the trained first parameter includes: Performing a weighted operation based on the trained first parameter and the first parameter in the digital assistant model of the digital assistant itself before the training to obtain an updated first parameter in the digital assistant model of the digital assistant itself; Among them, the trained first parameter and the first parameter in the digital assistant model itself before the training correspond to the first weighted weight and the second weighted weight respectively.

10. A computing device, characterized in that comprising a controller and a memory; the controller is coupled to the memory; The memory is used for computer program instructions; The controller is configured to call the computer program instructions in the memory to execute the method according to any one of claims 1-7 or 8-9.

Images