Model updating method, resource recommendation method, device, equipment and system
By performing model training and updating on the terminal side, reducing user data backhaul, the problem of poor user data privacy in the prior art is solved, and higher data security and privacy protection are achieved.
Patent Information
- Application Number
- CN201911135124.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-11-19
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2039-11-19
AI Technical Summary
In the prior art, user data has poor privacy, resulting in security risks in the process of updating the recommendation model.
By training and updating the model on the terminal side and sending the trained model parameter information to the network device, model fusion is performed to update the server-side model, thereby reducing the return of user data and improving privacy.
It realizes improving the privacy of user data during the recommendation model update process, avoiding the direct exposure of user data on the server side, and enhancing data security.
Smart Images

Figure CN112907309B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a model updating method, a resource recommendation method, an apparatus, a device and a system. Background Art
[0002] In recent years, with the continuous development of smart terminals, there are more and more scenarios for making decisions from a candidate set. For example, one or more products are selected from a candidate set to recommend to users.
[0003] In the prior art, the server usually uses a recommendation model to make a decision from a candidate set. In addition, in order to enable the decision model on the server side to capture changes in user interests and changes in the environment, the terminal needs to send user data, such as the user's age, geographic location, whether to click, whether to purchase, etc., back to the server, and the server updates the recommendation model based on the user data.
[0004] However, the above recommendation methods have the problem of poor privacy of user data. Summary of the invention
[0005] The embodiments of the present application provide a model updating method, a resource recommendation method, an apparatus, a device and a system to solve the problem of poor privacy of user data in the prior art.
[0006] In a first aspect, an embodiment of the present application provides a model updating method, which is applied to a terminal, and the method includes:
[0007] According to the server-side model of the decision problem of the network device, the terminal-side model of the decision problem is updated locally; the terminal-side model is trained according to the user data of the user to obtain the trained terminal-side model; according to the trained terminal-side model, model information is sent to the network device, and the model information is used to indicate the model parameters of the trained terminal-side model, so that the network device performs model fusion according to the model information to update the server-side model of the decision problem.
[0008] In a second aspect, an embodiment of the present application provides a model updating method, which is applied to a network device, and the method includes:
[0009] Receive model information of a decision problem respectively sent by multiple terminals, the model information is used to indicate model parameters of a terminal-side model obtained by the terminal according to user data training; perform model fusion on the terminal-side model obtained by the terminal training according to the model information to update the server-side model of the decision problem; send the server-side model to a target device, so that the target device updates the terminal-side model of the decision problem according to the server-side model, and the target device includes the multiple terminals.
[0010] In a third aspect, an embodiment of the present application provides a resource recommendation method, which is applied to a terminal, including:
[0011] Obtain an access request from a user, wherein the access request is used to request resources corresponding to a decision problem; adopt a terminal-side model of the decision problem to determine a target resource corresponding to the access request; the terminal-side model is updated based on a server-side model of the decision problem, and the server-side model is obtained by fusing a terminal-side model obtained by a network device based on training of a server-side model with a terminal; and recommend the target resource to the user.
[0012] In a fourth aspect, an embodiment of the present application provides a model updating device, including:
[0013] An updating module, configured to update a local terminal-side model of a decision problem according to a server-side model of the decision problem of the network device;
[0014] A training module, used to train the terminal side model according to the user data of the user to obtain a trained terminal side model;
[0015] A sending module is used to send model information to the network device according to the trained terminal-side model, wherein the model information is used to indicate the model parameters of the trained terminal-side model, so that the network device performs model fusion according to the model information to update the server-side model of the decision problem.
[0016] In a fifth aspect, an embodiment of the present application provides a model updating device, including:
[0017] A receiving module, used to receive model information of decision problems respectively sent by multiple terminals, wherein the model information is used to indicate model parameters of a terminal-side model obtained by the terminal through training based on user data;
[0018] A fusion module, used to perform model fusion on the terminal side model obtained by the terminal training according to the model information, so as to update the server side model of the decision problem;
[0019] A sending module is used to send the server-side model to a target device so that the target device updates the terminal-side model of the decision problem according to the server-side model, and the target device includes the multiple terminals.
[0020] In a sixth aspect, an embodiment of the present application provides a resource recommendation device, applied to a terminal, including:
[0021] An acquisition module, used to acquire an access request from a user, where the access request is used to request a resource corresponding to a decision problem;
[0022] A determination module, configured to determine a target resource corresponding to the access request by adopting a terminal-side model of the decision problem; the terminal-side model is updated according to a server-side model of the decision problem, and the server-side model is obtained by fusing a terminal-side model obtained by a network device based on training of a server-side model;
[0023] A recommendation module is used to recommend the target resource to the user.
[0024] In a seventh aspect, an embodiment of the present application provides a terminal, comprising: a memory, a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in any one of the first aspects.
[0025] In an eighth aspect, an embodiment of the present application provides a network device, comprising: a memory, a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in any one of the second aspects.
[0026] In a ninth aspect, an embodiment of the present application provides a terminal, comprising: a memory, a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in any one of the third aspects.
[0027] In the tenth aspect, an embodiment of the present application provides a model recommendation system, comprising the terminal described in the seventh aspect and the network device described in the eighth aspect.
[0028] An embodiment of the present application also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, the computer program includes at least one section of code, and the at least one section of code can be executed by a computer to control the computer to execute the method as described in any one of the first aspects.
[0029] An embodiment of the present application also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, the computer program includes at least one section of code, and the at least one section of code can be executed by a computer to control the computer to execute the method as described in any one of the second aspects.
[0030] An embodiment of the present application also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, the computer program includes at least one section of code, and the at least one section of code can be executed by a computer to control the computer to execute the method as described in any one of the third aspects.
[0031] An embodiment of the present application further provides a computer program, characterized in that when the computer program is executed by a computer, it is used to implement the method as described in any one of the first aspects.
[0032] An embodiment of the present application further provides a computer program, characterized in that when the computer program is executed by a computer, it is used to implement the method as described in any one of the second aspects.
[0033] An embodiment of the present application further provides a computer program, characterized in that when the computer program is executed by a computer, it is used to implement the method as described in any one of the third aspects.
[0034] The model updating method, resource recommendation method, apparatus, device and system provided in the embodiments of the present application update the local terminal side model of the decision problem according to the server side model of the decision problem, train the terminal side model according to the user data of the user, obtain the trained terminal side model, and send model information indicating the model parameters of the trained terminal side model to the network device according to the trained terminal side model, so that the network device performs model fusion according to the model information to update the server side model of the decision problem. The model updating method realizes that the terminal trains the terminal side model obtained based on the server side model update according to its user data, and the network device fuses the terminal trained model to update the server side model, thereby improving the privacy of user data. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0036] Figure 1A schematic diagram of an application scenario of an embodiment of the present application;
[0037] Figure 2 A flowchart of a model updating method provided in one embodiment of the present application;
[0038] Figure 3 A flowchart of a model updating method provided by another embodiment of the present application;
[0039] Figure 4 A flowchart of a model updating method provided in yet another embodiment of the present application;
[0040] Figure 5 A flowchart of a resource recommendation method provided in an embodiment of the present application;
[0041] Figure 6 A schematic diagram of the structure of a model updating device provided in one embodiment of the present application;
[0042] Figure 7 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;
[0043] Figure 8 A schematic diagram of the structure of a model updating device provided in another embodiment of the present application;
[0044] Fig. 9 A schematic diagram of the structure of a network device provided in one embodiment of the present application;
[0045] Fig.10 A schematic diagram of the structure of a resource recommendation device provided in one embodiment of the present application;
[0046] Fig.11 A schematic diagram of the structure of a terminal provided in yet another embodiment of the present application. DETAILED DESCRIPTION
[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0048] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "said", and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.
[0049] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0050] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.
[0051] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such a product or system. In the absence of more restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the product or system including the element.
[0052] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.
[0053] The model updating method provided in the embodiment of the present application can be applied to Figure 1 In the application scenario shown, Figure 1 As shown, the application scenario may include a network device 11 and multiple terminals 12. The network device 11 is connected to the terminal 12 in communication. The specific method of the communication connection is not limited in this application. For example, a wireless communication connection may be realized based on Bluetooth technology, or a wired communication connection may be realized based on a communication interface such as RS-232.
[0054] The network device 11 is used to integrate the terminal-side model of the decision problem trained by the terminal 12 to update the server-side model of the decision problem. The terminal 12 is used to update the local terminal-side model of the decision problem according to the server-side model of the decision problem, and train the terminal-side model according to the user data of the user to obtain the trained terminal-side model. The network device 11 and the terminal 12 obtain a model for the decision problem by joint model training.
[0055] Exemplarily, the user data used by the terminal 12 for model training may include user data obtained from other terminals 13 corresponding to it, and the user of the terminal 12 and the other terminals 13 corresponding to it is the same, for example Figure 1 The user of one terminal 12 and another terminal 13 is user 1, and the user of another terminal 12 and another other terminal 13 is user 2. It should be noted that Figure 1 In the example, the number of other terminals 13 corresponding to one terminal 12 is one. It can be understood that the number of other terminals 13 corresponding to one terminal 12 can also be multiple.
[0056] Since the user of terminal 12 and its corresponding other terminals 13 are the same, terminal 12 performs model training based on user data obtained from other terminals 13, which not only ensures the privacy of user data but also increases training samples, which is beneficial to improving the training effect of the model.
[0057] Exemplarily, the terminal 12 may use the terminal-side model to recommend resources to the user, or the terminal 12 may be used only for model training without using the terminal-side model to recommend resources to the user.
[0058] Exemplarily, the terminal 12 may send the trained terminal-side model to other terminals 13, so that the other terminals 13 can recommend resources to the user according to the trained terminal-side model.
[0059] Exemplarily, the terminal 12 may send the terminal-side model updated according to the server-side model of the decision problem to other terminals 13, so that the other terminals 13 recommend resources to the user according to the updated terminal-side model.
[0060] Exemplarily, the network device may send the server-side model of the decision problem to other terminals 13, so that other terminals 13 update the terminal-side model according to the server-side model and recommend resources to users according to the updated terminal-side model.
[0061] It should be noted that the terminal 12 and the other terminal 13 are both terminal devices that can be used by the user and can be flexibly implemented according to the needs. For example, the terminal 12 can be a smart speaker, and the other terminal 13 can be a smart phone. For example, the terminal 12 can be a laptop, and the other terminal 13 can be a smart phone. For example, the terminal 12 can be a wearable device, and the other terminal 13 can be a tablet computer. For example, the terminal 12 can be a smart phone, and the other terminal 13 can be another smart phone.
[0062] The decision problem may be any problem involving selecting K elements from a set of candidate elements, where K is an integer greater than 0 and less than M, and M is the total number of elements in the set of candidate elements. The decision problem includes but is not limited to the following three specific problem scenarios:
[0063] The first is the material selection problem. For example, there are many videos in a video application. A video generally has many posters and screenshots of the video plot. These pictures can be presented to users as the cover pictures of the video. Since different users have different interests, they have different preferences for different types of cover pictures. Showing the cover pictures that users like can increase the click-through rate of the video itself, and even the viewing time of the video. Therefore, among M different pictures, which one to choose as the cover picture to present to the user is a decision-making problem.
[0064] The second is the problem of function and service selection. For example, almost all current mainstream smartphones provide a recommendation function for applications (APPs) and even services, such as the APP recommendation bar on the negative first screen of smartphones and the service recommendation bar on the negative first screen of smartphones. However, how to choose which services or APPs to put in the recommendation bar is a decision-making problem.
[0065] In addition, there are usually many ways to access the Internet on mobile phones, such as the third generation (3rd Generation, 3G), the fourth generation (4th Generation, 4G), Wireless Fidelity (Wireless Fidelity, WiFi), etc. At the same time, there are usually many ways to access the Internet for data transmission. So how to choose which way to connect to the network is also a decision-making problem.
[0066] The third is the problem of product selection. In shopping applications, there are usually tens of thousands of products in the backend database, but the display space for products is usually very limited, and only a few products can be displayed on one screen. The quality of displayed products will have a great impact on the user's transaction rate. So how to choose which products to display from tens of thousands of products is still a decision-making problem.
[0067] Specifically, multiple terminals 12 respectively obtain the server-side model M1 of the decision problem from the network device 11, update the terminal-side model N1 of the decision problem according to the server-side model M1, and train the terminal-side model N1 according to the user data to obtain the trained terminal-side model N1'. The network device performs model fusion based on the terminal-side models N1' respectively trained by the multiple terminals to update the server-side model M2 of the decision problem, and sends the server-side model M2 to the multiple terminals. The multiple terminals can again update the terminal-side model N2 of the decision problem according to the server-side model M2, and train the terminal-side model N2 according to the user data to obtain the trained terminal-side model N2'. The network device can again perform model fusion based on the terminal-side models N2' respectively trained by the multiple terminals, update the server-side model M3, and send the server-side model M3 to the multiple terminals. ..., by continuously iterating the training and fusion of the models, the terminal side model N3', the server side model M4, the terminal side model N4', the server side model M5, ... can be obtained in turn, until the server side model converges, so that the final server side model can capture changes in user interests, the environment, etc.
[0068] Optionally, the network device may include a server, and the server completes the fusion of the terminal side models of the multiple terminals. Alternatively, the network device may include a server and at least two intermediate devices, and the intermediate device may first perform a model fusion, and then the server further fuses the fusion results of the intermediate device, which is beneficial to reduce the amount of calculation of the server.
[0069] It should be noted that the above training and fusion process is described by taking a specific decision problem as an example. When there are multiple decision problems, multiple models can be obtained by joint training for multiple decision problems in the above manner.
[0070] The model updating method provided in this embodiment sends the server-side model of the decision problem to multiple terminals respectively through a network device, and the multiple terminals update the local terminal-side model according to the server-side model, train the updated terminal-side model according to user data, and send model information to the network device according to the trained terminal-side model. The network device updates the server-side model according to the terminal-side model trained by the terminal, and sends the updated server-side model to the terminal, thereby realizing a model updating method in which the terminal trains the terminal-side model updated based on the server-side model according to the user data, and the network device fuses the terminal-trained model to update the server-side model.
[0071] Some implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0072] Figure 2 This is a flow chart of a model updating method provided in an embodiment of the present application. The execution subject of this embodiment may be Figure 1 Terminal 12 in. Figure 2 As shown, the method of this embodiment may include:
[0073] Step 201 : updating a local terminal-side model of a decision problem according to a server-side model of the decision problem of a network device.
[0074] In this step, illustratively, the terminal may obtain the server-side model of the decision problem from the network device, for example, the terminal may receive the server-side model of the decision problem sent by the network device. Alternatively, illustratively, the terminal may obtain the server-side model of the decision problem from other terminals corresponding to it.
[0075] Exemplarily, step 201 may specifically include the terminal using the server-side model of the decision problem as the local terminal-side model of the decision problem, in which case the terminal-side models updated by different terminals are the same. Alternatively, exemplarily, the terminal obtains a terminal-side model that can reflect the user's personalization based on the user's characteristics and the server-side model of the decision problem, so that the terminal-side model can implement personalized recommendations, which is conducive to improving the accuracy of recommendations.
[0076] Step 202: Train the terminal side model according to the user data of the user to obtain a trained terminal side model.
[0077] In this step, the terminal further trains the local terminal-side model updated in step 201 according to the user data of its user. The user data includes the user's feedback on the decision problem. For example, the decision problem is to select a picture from 100 pictures. After a picture is selected from the 100 pictures and displayed to the user, the feedback of the decision problem may specifically be whether the user clicks on the picture. When the terminal uses the terminal-side model of the decision problem to recommend to the user, the user data may include local user data.
[0078] By training the terminal-side model based on user data through the terminal, rather than uploading the user data to the network device for model training, the problem of poor user data privacy caused by uploading the user data to the network device can be avoided.
[0079] Step 203, sending model information to the network device according to the trained terminal side model, wherein the model information is used to indicate model parameters of the trained terminal side model, so that the network device performs model fusion according to the model information to update the server side model of the decision problem.
[0080] In this step, the model information may be any type of information that can be used to indicate the trained terminal-side model. Exemplarily, the model information may include model parameters of the trained terminal-side model.
[0081] Exemplarily, the model information may include server-side model parameters and parameter difference information between the server-side model parameters and the terminal-side model parameters, wherein the server-side model parameters are model parameters of the trained terminal-side model, and the terminal-side model parameters are model parameters of the terminal-side model before training of the decision problem. The model information includes the server-side model parameters and the parameter difference information, which is beneficial to reducing the amount of server computation and facilitating the server to perform model fusion.
[0082] In this embodiment, the terminal side model of the local decision problem is updated according to the server side model of the decision problem, the terminal side model is trained according to the user data of the user to obtain the trained terminal side model, and model information indicating the model parameters of the trained terminal side model is sent to the network device according to the trained terminal side model, so that the network device performs model fusion according to the model information to update the server side model of the decision problem. This implements a model update method in which the terminal trains the terminal side model obtained based on the server side model update according to its user data, and the network device fuses the terminal trained model to update the server side model, thereby improving the privacy of user data.
[0083] Figure 3 This is a flow chart of a model updating method provided by another embodiment of the present application. The execution subject of this embodiment may be Figure 1 The network device 11 in. Figure 3 As shown, the method of this embodiment may include:
[0084] Step 301, receiving model information of decision problems respectively sent by multiple terminals, wherein the model information is used to indicate model parameters of a terminal-side model obtained by the terminal through training based on user data.
[0085] In this step, corresponding to step 203, exemplarily, the model information may include model parameters of the terminal side model obtained by training the terminal, or the model information may include server side model parameters and parameter difference information between the server side model parameters and the terminal side model parameters.
[0086] Step 302: Based on the model information, the terminal-side model obtained by the terminal training is subjected to model fusion to update the server-side model of the decision problem.
[0087] In this step, the network device updates the server-side model by fusing the terminal-side model. Specifically, the network device may use the server-side model obtained by model fusion as the current server-side model of the decision problem.
[0088] Exemplarily, the network device may integrate terminal-side models trained separately by multiple terminals, so that the terminal may use the server-side model of the decision problem as the terminal-side model of the local decision problem.
[0089] Alternatively, illustratively, the network device may classify and fuse terminal-side models trained separately for multiple terminals based on user characteristics, so that the terminal can obtain a terminal-side model that can reflect user personalization based on the user characteristics and the server-side model of the decision problem.
[0090] Since the network device updates the server-side model by fusing terminal-side models trained on multiple terminals respectively, and the multiple terminal-side models are trained by the terminals according to user data, the server-side model obtained by fusing the terminal-side models can reflect the user data corresponding to the multiple terminals respectively, that is, a large amount of sample data, so that the server-side model can converge quickly. Compared with model training on a single terminal, the convergence speed is improved and the load on the terminal is reduced.
[0091] Step 303: Send the server-side model to a target device, so that the target device updates the terminal-side model of the decision problem according to the server-side model, and the target device includes the multiple terminals.
[0092] In this step, the server-side model obtained by model fusion in step 302 is sent to the target device. The target device may also include other terminals corresponding to the terminal. Exemplarily, the target device may also include a terminal other than the terminal and the other terminals that needs to recommend resources to the user for the decision problem.
[0093] In this embodiment, by receiving model information of the decision problem sent by multiple terminals respectively, the model information is used to indicate the model parameters of the terminal side model obtained by the terminal according to the training of the user data, the terminal side model obtained by the terminal training is fused according to the model information to update the server side model of the decision problem, and the server side model is sent to the user device, so that the user device updates the terminal side model according to the server side model, thereby realizing the model update method in which the terminal trains the terminal side model obtained based on the server side model update according to the user data, and the network device fuses the terminal trained model to update the server side model.
[0094] Figure 4 This is a flow chart of a model updating method provided by another embodiment of the present application. Based on the above method embodiments, this embodiment mainly describes the interaction process between the network device and the terminal. Figure 4 As shown, the method of this embodiment may include:
[0095] Step 401, Y1 terminals respectively update the local terminal-side model of the decision problem according to the server-side model of the decision problem of the network device.
[0096] In this step, Y is an integer greater than 2. Exemplarily, different user features in the server-side model may correspond to different model parameters, and updating the local terminal-side model of the decision problem according to the server-side model of the decision problem may specifically include: determining the model parameters matching the user features of its user from the server-side model to obtain the local terminal-side model of the decision problem.
[0097] For example, assuming that user characteristics include gender and age, in the server-side model, gender "male" corresponds to parameter 1, gender "female" corresponds to parameter 2, age "20-30" corresponds to parameter 3, age "30-40" corresponds to parameter 4, age "40-50" corresponds to parameter 5, and age "over 50" corresponds to parameter 6. Then, when the gender of a terminal user is "male" and the age is "30-40", it can be determined that the model parameter corresponding to the gender in the local terminal-side model is parameter 1, and the model parameter corresponding to the age is parameter 4. Parameters 1 and 4 are the model parameters that match the user characteristics of their users.
[0098] Exemplarily, after obtaining the server-side model, the terminal can directly update the local terminal-side model according to the server-side model, or, after obtaining the server-side model, the terminal can first determine whether the local terminal-side model of the decision problem is converged, and if the local terminal-side model of the decision problem is not converged, the local terminal-side model is updated according to the server-side model. By updating the local terminal-side model according to the server-side model when the local terminal-side model of the decision problem is not converged, it is beneficial to improve the user personalization of the terminal-side model.
[0099] It should be noted that this application does not limit the specific method for determining whether the terminal side model has converged. Taking the online learning algorithm corresponding to the terminal side model as UCB as an example, whether the terminal side model has converged can be determined by the confidence interval radius and the average reward. For example, if a terminal side model has a candidate element whose average reward is significantly higher than other candidate elements and the confidence interval radius is small, it can be considered that the terminal side model has converged.
[0100] Step 402: The Y1 terminals respectively train the terminal side model according to the user data of the user to obtain the trained terminal side model.
[0101] In this step, exemplarily, the user data used by a terminal for terminal-side model training may include user data obtained from other corresponding terminals, where the user of the other terminal is the same user as the user.
[0102] Optionally, the user's usage enthusiasm for the terminal is lower than the user's usage enthusiasm for the other terminals. Usage enthusiasm can be flexibly implemented according to demand. For example, the usage enthusiasm can be determined according to the frequency of use and the duration of use. The higher the frequency of use and the longer the duration of use, the greater the usage enthusiasm. The lower the frequency of use and the shorter the duration of use, the smaller the usage enthusiasm. By making the usage enthusiasm of the terminal for model training on the terminal side model lower than the usage enthusiasm of other terminals, it is possible to implement terminal side model training by the terminal with lower usage enthusiasm, which can reduce the impact of model training on user use and is conducive to improving the user experience.
[0103] Among them, the terminal side model is a model implemented based on an online learning algorithm. Exemplarily, an instance of a decision problem can be called a machine, and the candidate elements therein are called arms. Users will give feedback to the display arms according to their own interests and other factors. For example, in the actual system, if a user clicks on one of the arms, it is called positive feedback, and if a user complains about an arm, it is called negative feedback. A set of the above display and feedback processes is called an experiment. The solution to a decision problem is usually an online learning algorithm, and the algorithm goal of the algorithm is to minimize the cumulative regret (Regret) that can be achieved after performing the above display and feedback process in an iterative order for n (where n is an integer greater than 1) rounds of experiments, that is, to minimize the gap between it and the cumulative optimization value (maximum positive feedback), so as to maximize positive feedback. The online learning algorithm will retrain the terminal side model according to the feedback of users in each round of experiments to achieve the algorithm goal, thereby obtaining a trained terminal side model.
[0104] It should be noted that the specific algorithm type of the online learning algorithm is not limited in the embodiments of the present application, for example, it can be an Upper Confidence Bounds (UCB) algorithm. In the embodiments of the present application, the terminal-side model preinstalled in the terminal can be a cold-start server-side model initiated by the server. Taking the algorithm of the model as UCB as an example, the average reward of the cold-start server-side model is 1, and the confidence radius is infinite.
[0105] Optionally, the Y1 terminals may be some or all of the terminals that are connected to the network device for communication. The Y1 terminals may activate their local training mode by the network device to train the terminal side model. Exemplarily, the following step 402' may be included before step 402.
[0106] Step 402': the network device sends training information to the Y1 terminals, where the training information is used to instruct training of the terminal side model of the decision problem.
[0107] Correspondingly, the terminal trains the terminal side model according to the user data of the user, which may specifically include: upon receiving the training information, training the terminal side model according to the user data of the user.
[0108] This application does not limit the specific manner in which the network device sends the training information to the Y1 terminals. For example, the network device may send the training information to the Y1 terminals respectively, or the network device may send the training information to the Y1 terminals by broadcasting. By sending the training information through the network device, the network device can control the training of the terminal side model, thereby improving the pertinence of the training.
[0109] Optionally, before step 402', the network device may select the Y1 terminals according to certain criteria. Exemplarily, the network device may select Y1 terminals according to random criteria, or the network device may select Y1 terminals according to certain activity criteria. Specifically, the network device may select the Y1 terminals whose target activity is greater than the activity threshold according to the target activity. The target activity may specifically be user activity or terminal activity. Taking the activity as user activity as an example, exemplarily, multiple users whose user activity is greater than the activity threshold may be selected, and the terminals used by the multiple users may be determined as the Y1 terminals. Taking the activity as terminal activity as an example, exemplarily, multiple terminals whose terminal activity is greater than the activity threshold may be selected as the Y1 terminals.
[0110] Among them, the terminal with an activity greater than the activity threshold can be considered as a terminal with more training samples. By selecting Y1 terminals with an activity greater than the activity threshold, it can be ensured that the terminal can obtain more training samples when performing terminal-side model training, thereby improving the accuracy of model training and saving terminal resources. It should be noted that the specific calculation method of activity may not be limited in this application. For example, the activity can be determined by weighted summation based on factors such as user usage time and user usage frequency.
[0111] Optionally, step 402' may specifically include: the network device determines whether the current server-side model of the decision problem has converged, and if the current server-side model of the decision problem has not converged, sending training information to the Y1 terminals. By sending training information to the Y1 terminals if the current server-side model has not converged, retraining of the unconverged server-side model can be achieved, thereby further improving the pertinence of the training, which is conducive to improving the convergence speed of the server-side model and saving terminal resources.
[0112] It should be noted that the present application does not limit the specific method for determining whether the server-side model is converged. Taking the online learning algorithm corresponding to the server-side model as UCB as an example, whether the server-side model is converged can be determined by the radius of the confidence interval.
[0113] Furthermore, considering that user interests may change over time, even when the current server-side model of the decision problem has converged, training information may be sent based on a certain trigger mechanism. For example, training information may be sent through a timing mechanism, for example, every 7 days.
[0114] Exemplarily, in order to improve the utilization rate of the terminal storage space, the terminal may not store the captured user feedback on the recommended resources for the decision problem before the network device activates the local training mode for the decision problem. After the network device activates the local training mode for the decision problem, the terminal may store the captured user feedback on the recommended resources for the decision problem.
[0115] Exemplarily, the user data of the user may be stored in a log.
[0116] Exemplarily, in order to improve the security of user data and prevent the user data stored in the log from being accessed by irrelevant programs, the method of this embodiment also includes: the terminal authenticates the target application, and the target application is the application to which the decision problem belongs; after identity authentication, the terminal reads the user data of the user from the log to train the terminal-side model based on the user data. For example, assuming that a decision problem is which picture to choose as the cover image of a video in a video application, it can be considered that the decision problem belongs to the video application. It should be noted that the specific method of identity authentication may not be limited in this application, for example, verification may be performed based on the application identifier.
[0117] Exemplarily, in order to improve the security of the terminal-side model, the terminal can store the terminal-side model in a security domain with a higher security level. Exemplarily, the terminal-side model can be stored in an operation and storage domain with a higher security level than the target application. For example, the terminal-side model can be stored in a trusted execution environment (TEE) or a Linux kernel. Taking the storage of the terminal-side model in TEE as an example, the terminal-side model can be stored using a TEE operating system (OS) instance. Further, it can be called through the TEE client application programming interface (API) to ensure the security of the terminal-side model.
[0118] Exemplarily, in order to improve the security of user data, after performing terminal-side model training based on the user data, the stored user data can be deleted.
[0119] Exemplarily, in order to improve the controllability of the terminal for model training, the terminal may also perform the following steps before model training: determine whether the training conditions are met; if the training conditions are met, perform the steps of training the terminal side model according to the user data of the user. The training conditions refer to the conditions that the terminal needs to meet to start training the terminal side model, which can be flexibly implemented according to the needs. Optionally, the training conditions include but are not limited to one or more of the following: the current power is greater than the power threshold, the battery is in standby mode, the load is less than the load threshold, and the current time is within a preset time period.
[0120] It should be noted that the relationship between the above-mentioned training information and training conditions can be: the terminal can first activate the local training mode of the decision problem by the training information sent by the network device, and then further determine when to start training the terminal-side model of the decision problem based on the training conditions.
[0121] Step 403: Y2 terminals send model information to the network device respectively.
[0122] In this step, Y2 is an integer greater than 2 and less than or equal to Y1, and the Y2 terminals may be some or all of the Y1 terminals. For example, when a terminal has trained a terminal-side model but is not connected to the network, it cannot send model information to the network device.
[0123] For example, assuming that Y2 terminals include terminal 1 and terminal 2, terminal 1 may send model information 1 to the network device, and terminal 2 may send model information 2 to the network device, wherein model information 1 may indicate a terminal-side model obtained by terminal 1 according to user data training of user 1, and model information 2 may be used to indicate a terminal-side model obtained by terminal 2 according to user data training of user 2. Further, model parameter 1 may include server-side model parameter 1 and parameter difference information 1, wherein server-side model parameter 1 is a model parameter of the terminal-side model before terminal 1 training, and parameter difference information 1 is parameter difference information between the model parameter of the terminal-side model before terminal 1 training and the model parameter of the terminal-side model after terminal 1 training; model parameter 2 may include server-side model parameter 2 and parameter difference information 2, wherein server-side model parameter 1 is a model parameter of the terminal-side model before terminal 2 training, and parameter difference information 2 is parameter difference information between the model parameter of the terminal-side model before terminal 2 training and the model parameter of the terminal-side model after terminal 2 training.
[0124] It should be noted that the above model parameters may be related to the corresponding online learning algorithm. Taking the online learning algorithm as UCB as an example, the model parameters may include the confidence interval radius and the average reward.
[0125] In the case where the network device includes a server, a first intermediate device, and a second intermediate device, multiple first terminals among the Y2 terminals can send model information to the first intermediate device, and multiple terminals among the Y2 terminals can send model information to the second intermediate device. It should be noted that the first terminal and the second terminal are different terminals.
[0126] In step 404, the network device fuses the terminal-side model obtained through terminal training according to the model information respectively sent by the Y2 terminals to update the server-side model of the decision problem.
[0127] In this step, exemplarily, step 404 may specifically include: according to the user characteristics of the terminal user, a fusion strategy of fusing model parameters corresponding to the same user characteristics is used to perform model fusion on the terminal side model obtained by the terminal training.
[0128] For example, assuming that the user characteristics include gender and age, the gender in the user characteristics of terminals 1 to 4 is "male", the gender in the user characteristics of terminals 5 to 8 is "female", and the age in the user characteristics of terminals 1 and 2 is "20-30", the age in the user characteristics of terminals 3 and 4 is "30-40", the age in the user characteristics of terminals 5 and 6 is "20-30", and the age in the user characteristics of terminals 7 and 8 is "30-40", then the parameters corresponding to the age and gender of terminal 1 and terminal 2 can be fused, the parameters corresponding to the age and gender of terminal 3 and terminal 4 can be fused, the parameters corresponding to the age and gender of terminal 5 and terminal 6 can be fused, and the parameters corresponding to the age and gender of terminal 7 and terminal 8 can be fused.
[0129] Exemplarily, the network device may use an aggregation function or algorithm to fuse the terminal side model obtained by terminal training. This application may not limit the specific type of aggregation function or algorithm. Taking the online learning algorithm as UCB as an example, the average reward of the fused server side model can be determined based on the average reward sent by the terminal by using an average aggregation function, and the confidence interval radius of the server side model can be determined based on the confidence interval radius sent by the terminal by using an aggregation function that selects the minimum value, thereby obtaining a new server side model.
[0130] Since the greater the parameter difference information in the model information of a terminal, the more user data accumulated on the terminal can be represented, so in order to reduce the amount of calculation of the network device, step 404 may specifically include: If the amount of the model information is greater than the first quantity threshold, the model information is sorted in descending order of parameter difference according to the parameter difference information; the terminal side models obtained by training respectively indicated by the first N model information are model fused, where N is an integer greater than 1 and less than Y2. Among them, the first N model information can correspond to the N terminals one by one.
[0131] If the amount of the model information is less than the first amount threshold, the model information respectively sent by Y2 terminals may be merged.
[0132] In order to further improve the controllability of network devices over fusion and reduce the amount of calculation, optionally, before step 404, it may also include: determining whether an ignoring condition is met, and if the ignoring condition is not met, executing step 404. Among them, the ignoring condition refers to the condition that the network device needs to meet to ignore the model information and not perform model fusion, which can be flexibly implemented according to needs. Optionally, the ignoring condition includes but is not limited to one or more of the following: the amount of the model information is less than the second quantity threshold, the current server-side model of the decision problem has converged, and the degree of parameter difference between the terminal-side model after training and the terminal-side model before training is less than the difference threshold. It can be understood that the second quantity threshold is less than the aforementioned first quantity threshold.
[0133] If the ignore condition is met, the process ends without executing step 404, that is, the server-side model is not updated according to the model information sent by Y2 terminals.
[0134] Optionally, when the network device includes a first intermediate device and a second intermediate device, step 404 may specifically include: the first intermediate device fuses the terminal side models obtained by training the multiple first terminals according to the model information sent by the multiple first terminals to obtain a first fusion model; the second intermediate device fuses the terminal side models obtained by training the multiple second terminals according to the model information sent by the multiple second terminals to obtain a second fusion model; the server fuses the first fusion model with the second fusion model.
[0135] Step 405: The network device sends the server-side model to Y terminals, where the Y terminals include the Y1 terminals.
[0136] In this step, optionally, before step 405, it may also include: judging whether the difference between the convergence degree of the server-side model and the historical server-side model of the decision problem before fusion is greater than the degree threshold; if the difference between the convergence degree of the server-side model and the historical server-side model is greater than the degree threshold, executing step 405. Thus, the network device can send the server-side model with significantly improved convergence degree to the terminal, avoiding invalid updates of the terminal-side model in the terminal, and reducing the occupation of transmission resources between the network device and the terminal.
[0137] If the difference in convergence degree between the server-side model and the historical server-side model is less than the degree threshold, the process ends without executing step 405, i.e., the new server-side model obtained by model fusion based on the model information sent by Y2 terminals is not sent to the terminal.
[0138] Optionally, the Y terminals may further include one or more terminals other than the Y1 terminals. After receiving the server-side model, the one or more terminals may update the local terminal-side model in the same manner as step 401 .
[0139] In this embodiment, Y1 terminals respectively train the terminal-side model of the local decision problem according to the user data of the user to obtain the trained terminal-side model, and Y2 terminals respectively send model information to the network device. The network device fuses the terminal-side models trained in the Y2 terminals according to the model information respectively sent by the Y2 terminals to update the server-side model of the decision problem. Further, the network device sends the server-side model of the decision problem to the Y terminals, so that the Y terminals update the local terminal-side model according to the server-side model, so that in the process of updating the model, the terminal does not need to send the user data back to the server, thereby improving the privacy of the user data, maintaining the privacy of the user, and reducing the load of the network device.
[0140] It should be noted that the above embodiment is described by taking the model update of one decision problem as an example. When there are multiple decision problems, the method described in the above embodiment can be used to update the model of each decision problem.
[0141] Figure 5 This is a flow chart of a resource recommendation method provided in an embodiment of the present application. The execution subject of this embodiment may be Figure 1 Terminal 12 or other terminal 13 in the Figure 5 As shown, the method of this embodiment may include:
[0142] Step 501: obtaining a user's access request, where the access request is used to request a resource corresponding to a decision problem.
[0143] In this step, the user's access request can trigger a decision problem, and it is necessary to recommend to the user the decision problem triggered by the access request. Taking the video application as an example, the access request may be a request to access the homepage of the video application, and the corresponding decision problem may be which video resource or resources to display in the recommendation column of the homepage of the video application.
[0144] Step 502, using the terminal side model of the decision problem to determine the target resource corresponding to the access request; the terminal side model is updated according to the server side model of the decision problem, and the server side model is obtained by integrating the terminal side model obtained by the network device based on the server side model training.
[0145] In this step, the terminal may exemplarily use the terminal-side model obtained through training to determine the target resource corresponding to the access request. Alternatively, the terminal may exemplarily use the terminal-side model obtained through updating the server-side model to determine the target resource corresponding to the access request.
[0146] Specifically, the terminal may obtain feature data of each candidate element in the candidate element set of the decision problem, and use the feature data of each candidate element as input to the terminal-side model of the decision problem, and the output of the terminal-side model may indicate the target resource, which is specifically at least one candidate element in the candidate element set. It should be noted that the specific type of feature data may not be limited in this application, for example, it may be product price, picture brightness, etc.
[0147] Optionally, the method of this embodiment may further include: the terminal determines whether the terminal-side model converges; if the terminal-side model converges, deleting some candidate elements in the candidate set of the decision problem, and the element scores of the some candidate elements are less than the score threshold, and the element scores can be used to indicate the user's interest level. The larger the element score of a candidate element, the greater the user's interest, that is, the more interested; the smaller the element score of a candidate element, the smaller the user's interest, that is, the less interested. This application does not limit the specific method of determining the element score of a candidate element. Taking the online learning algorithm as UCB as an example, the element score of a candidate element can be, for example, the sum of the average expectation of the candidate element and the radius of the confidence interval of the candidate selection.
[0148] By deleting some candidate elements in the candidate set of the decision problem, the amount of data of the candidate elements input by the terminal side model when determining the target resource can be reduced, which is conducive to reducing the amount of calculation of the terminal and saving the computing resources of the terminal. In addition, when the terminal needs to store the candidate set, it is also conducive to reducing the occupation of the terminal storage space.
[0149] For example, the number of candidate elements in the candidate set after deleting some candidate elements may be greater than the number of elements in the target resource, so that when the user's interests change, the necessity of downloading candidate elements from the server can be reduced, which is conducive to improving the real-time nature of the recommendation.
[0150] It should be noted that the updating process of the terminal side model of the decision problem in this embodiment can refer to the aforementioned Figure 2-Figure 4 The relevant description of the illustrated embodiment will not be repeated here.
[0151] Step 503: recommend the target resource to the user.
[0152] It should be noted that the specific method of recommending the target resource to the user can be flexibly implemented according to the needs, and the target resource can be recommended to the user by means of a pop-up window, SMS prompt, voice playback, etc. For example, the target resource can be displayed to the user in a user interface (UI).
[0153] In this embodiment, by obtaining the user's access request for requesting resources corresponding to the decision problem, the terminal-side model of the decision problem is used to determine the target resource corresponding to the access request, and recommend the target resource to the user, the terminal recommends resources to the user using the terminal-side model obtained by the model update method described in the previous embodiment.
[0154] Figure 6 This is a schematic diagram of the structure of a model updating device provided in one embodiment of the present application; Figure 6 As shown, this embodiment provides a model updating device, which can execute the method on the terminal side in the above-mentioned model updating method. Specifically, the model updating device may include:
[0155] An updating module 61, configured to update a local terminal-side model of a decision problem according to a server-side model of a decision problem of a network device;
[0156] A training module 62, configured to train the terminal side model according to the user data of the user to obtain a trained terminal side model;
[0157] The sending module 63 is used to send model information to the network device according to the trained terminal side model, and the model information is used to indicate the model parameters of the trained terminal side model, so that the network device performs model fusion according to the model information to update the server side model of the decision problem.
[0158] Optionally, different user characteristics in the server-side model correspond to different model parameters; the update module 61 is specifically used to determine the model parameters that match the user characteristics of the user from the server-side model to obtain a local terminal-side model of the decision problem.
[0159] Optionally, the user data includes user data obtained from other corresponding terminals, and the user of the other terminal is the same user as the user.
[0160] Optionally, the user's usage enthusiasm for the terminal is lower than the user's usage enthusiasm for the other terminals.
[0161] Optionally, the update module 61 is specifically used for:
[0162] Determine whether the local terminal side model of the current decision problem converges;
[0163] If the current local terminal-side model of the decision problem has not converged, the local terminal-side model of the decision problem is updated according to the server-side model of the decision problem of the network device.
[0164] Optionally, the training module 62 is specifically used for:
[0165] Determine whether the training conditions are met;
[0166] If the training conditions are met, the terminal side model is trained according to the user data.
[0167] Optionally, the training conditions include one or more of the following: the current power level is greater than a power threshold, the device is in standby mode, the load is less than a load threshold, or the current moment is within a preset time period.
[0168] Optionally, the model information includes: server-side model parameters and parameter difference information between the server-side model parameters and terminal-side model parameters, the server-side model parameters are the model parameters of the terminal-side model after training, and the terminal-side model parameters are the model parameters of the terminal-side model before training.
[0169] Optionally, the apparatus further includes a receiving module, configured to receive training information sent by the network device, wherein the training information is used to instruct training of a terminal side model of the decision problem;
[0170] The training module 62 is specifically used to train the terminal side model according to user data when the training information is received.
[0171] Figure 5 The device shown can perform Figure 2-Figure 4 The method on the terminal side of the embodiment shown in the figure, for the part not described in detail in this embodiment, please refer to Figure 2-Figure 4 The implementation process and technical effects of this technical solution refer to Figure 2-Figure 4 The description in the illustrated embodiment will not be repeated here.
[0172] In one possible implementation, Figure 6 The structure of the model updating device shown can be implemented as a terminal. Figure 7 As shown, the terminal may include: a processor 71 and a memory 72. The memory 72 is used to store the support terminal to execute the above Figure 2-Figure 4 The program on the terminal side of the model updating method provided in the illustrated embodiment, the processor 71 is configured to execute the program stored in the memory 72 .
[0173] The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 61, the following steps can be implemented:
[0174] Updating a local terminal-side model of the decision problem according to a server-side model of the decision problem of the network device;
[0175] Training the terminal-side model according to the user data of the user to obtain a trained terminal-side model;
[0176] Based on the trained terminal-side model, model information is sent to the network device, where the model information is used to indicate model parameters of the trained terminal-side model, so that the network device performs model fusion according to the model information to update the server-side model of the decision problem.
[0177] Optionally, the processor 71 is further configured to execute the aforementioned Figure 2-Figure 4 All or part of the steps on the terminal side in the illustrated embodiment.
[0178] The structure of the terminal may also include a communication interface 73 for the terminal to communicate with other devices or a communication network.
[0179] Figure 8A schematic diagram of the structure of a model updating device provided in another embodiment of the present application; with reference to FIG8 , this embodiment provides a model updating device, which can execute the method on the network device side in the above-mentioned model updating method. Specifically, the model updating device may include:
[0180] A receiving module 81, used to receive model information of decision problems respectively sent by multiple terminals, wherein the model information is used to indicate model parameters of a terminal-side model obtained by the terminal through training based on user data;
[0181] A fusion module 82, configured to perform model fusion on the terminal side model obtained by the terminal training according to the model information, so as to update the server side model of the decision problem;
[0182] The sending module 83 is used to send the server-side model to the target device so that the target device updates the terminal-side model of the decision problem according to the server-side model. The target device includes the multiple terminals.
[0183] Optionally, the fusion module 82 is specifically used to: perform model fusion on the terminal side model obtained by the terminal training according to the user characteristics of the terminal user and the fusion strategy of fusing model parameters corresponding to the same user characteristics.
[0184] Optionally, the model information includes: server-side model parameters and parameter difference information between the server-side model parameters and terminal-side model parameters, the server-side model parameters are model parameters of the terminal-side model after training, and the terminal-side model parameters are model parameters of the terminal-side model before training for the decision problem.
[0185] Optionally, the fusion module 82 is specifically used for:
[0186] If the amount of the model information is greater than a first amount threshold, sorting the model information in descending order of parameter differences according to the parameter difference information;
[0187] Model fusion is performed on the terminal-side models obtained through training respectively indicated by the first N model information.
[0188] Optionally, the fusion module 82 is specifically used for:
[0189] Determine whether the ignore condition is met;
[0190] If the ignoring condition is not met, the terminal side model obtained by the terminal training is subjected to model fusion according to the model information.
[0191] Optionally, the ignoring conditions include one or more of the following: the amount of model information is less than a second quantity threshold, the current server-side model of the decision problem has converged, and the parameter difference between the trained terminal-side model and the pre-trained terminal-side model is less than a difference threshold.
[0192] Optionally, the sending module is further used to send training information to the multiple terminals respectively, and the training information is used to instruct the training of the terminal side model of the decision problem.
[0193] Optionally, the device further includes a selection module, configured to select, based on the target activity, the plurality of terminals whose target activity is greater than an activity threshold.
[0194] Optionally, the sending module is specifically used to: determine whether the current server-side model of the decision problem has converged; if the current server-side model of the decision problem has not converged, send training information to the multiple terminals respectively.
[0195] Optionally, the sending module is specifically used to: determine whether the difference in convergence degree between the server-side model and the historical server-side model of the decision problem before fusion is greater than a degree threshold; if the difference in convergence degree between the server-side model and the historical server-side model is greater than the degree threshold, then send the server-side model to the target device.
[0196] Optionally, the receiving module 81 includes a first receiving module, a second receiving module, and the fusion module 82 includes a first fusion module, a second fusion module, and a third fusion module; the network device includes a first intermediate device, a second intermediate device, and a server, the first intermediate device includes a first fusion module and a first receiving module, the second intermediate device includes a second fusion module and a second receiving module, and the server includes a third fusion module
[0197] The first receiving module is used to receive server-side model information sent by multiple first terminals among the multiple terminals;
[0198] The second receiving module is used to receive terminal side model information sent by multiple second terminals among the multiple terminals;
[0199] The first fusion module is used to fuse the terminal side model trained by the first terminal according to the server side model information to obtain a first fusion model;
[0200] The second fusion module is used to fuse the terminal side model trained by the second terminal according to the terminal side model information to obtain a second fusion model;
[0201] The third fusion module is used to fuse the first fusion model and the second fusion model.
[0202] Figure 8 The device shown can perform Figure 2-Figure 4 The method of the network device side in the embodiment shown in the figure, for the part not described in detail in this embodiment, please refer to Figure 2-Figure 4 The implementation process and technical effects of this technical solution refer to Figure 2-Figure 4 The description in the illustrated embodiment will not be repeated here.
[0203] In one possible implementation, Figure 8 The structure of the model updating device shown can be implemented as a network device. Fig. 9 As shown, the network device may include: a processor 91 and a memory 92. The memory 92 is used to store the information that supports the network device to execute the above Figure 2-Figure 4 The model updating method provided in the illustrated embodiment is a program on the network device side, and the processor 91 is configured to execute the program stored in the memory 92 .
[0204] The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 91, the following steps can be implemented:
[0205] Receiving model information of decision problems respectively sent by a plurality of terminals, wherein the model information is used to indicate model parameters of a terminal-side model obtained by the terminal through training according to user data;
[0206] According to the model information, performing model fusion on the terminal side model obtained by the terminal training to update the server side model of the decision problem;
[0207] The server-side model is sent to a target device, so that the target device updates a terminal-side model of the decision problem according to the server-side model, and the target device includes the multiple terminals.
[0208] Optionally, the processor 91 is further configured to execute the aforementioned Figure 2-Figure 4 All or part of the steps on the network device side in the illustrated embodiment.
[0209] The structure of the network device may also include a communication interface 93 for the network device to communicate with other devices or a communication network.
[0210] Fig.10 This is a schematic diagram of the structure of a resource recommendation device provided in an embodiment of the present application; Fig.10 As shown, this embodiment provides a resource recommendation device, which can execute the above-mentioned resource recommendation method. Specifically, the resource recommendation device may include:
[0211] An acquisition module 101 is used to acquire an access request from a user, where the access request is used to request a resource corresponding to a decision problem;
[0212] A determination module 102 is used to determine a target resource corresponding to the access request using a terminal-side model of the decision problem; the terminal-side model is updated according to a server-side model of the decision problem, and the server-side model is obtained by fusing a terminal-side model obtained by a network device based on training of a server-side model by a terminal;
[0213] The recommendation module 103 is used to recommend the target resource to the user.
[0214] Fig.10 The device shown can perform Figure 5 For the method of the embodiment shown in the figure, the part not described in detail in this embodiment can be referred to Figure 5 The implementation process and technical effects of this technical solution refer to Figure 5 The description in the illustrated embodiment will not be repeated here.
[0215] In one possible implementation, Fig.10 The structure of the resource recommendation device shown in FIG. 11 can be implemented as a terminal. As shown in FIG. 11 , the terminal may include: a processor 111 and a memory 112. The memory 112 is used to store information that supports the terminal to execute the above Figure 5 The program of the resource recommendation method provided in the illustrated embodiment, the processor 111 is configured to execute the program stored in the memory 112 .
[0216] The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 111, the following steps can be implemented:
[0217] Obtaining an access request from a user, where the access request is used to request a resource corresponding to a decision problem;
[0218] The terminal side model of the decision problem is used to determine the target resource corresponding to the access request; the terminal side model is updated according to the server side model of the decision problem, and the server side model is obtained by fusing the terminal side model obtained by the network device based on the training of the server side model;
[0219] The target resource is recommended to the user.
[0220] Optionally, the processor 11 is further configured to execute the aforementioned Figure 5 All or part of the steps of the illustrated embodiment.
[0221] The structure of the terminal may also include a communication interface 113 for the terminal to communicate with other devices or a communication network.
[0222] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the terminal, which includes instructions for executing the above Figure 2-Figure 4 The program involved in the terminal side of the model updating method in the method embodiment shown.
[0223] The embodiment of the present invention provides a computer storage medium for storing computer software instructions used by a network device, which includes instructions for executing the above Figure 2-Figure 4 The program involved in the network device side of the model updating method in the method embodiment shown.
[0224] The embodiment of the present invention provides a computer storage medium for storing computer software instructions used by a network device, which includes instructions for executing the above Figure 5 The procedures involved in the illustrated method embodiment.
[0225] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0226] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0227] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processor of the computer or other programmable device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0228] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0229] These computer program instructions may also be loaded onto a computer or other programmable device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0230] In a typical configuration, a terminal includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0231] The memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0232] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a terminal. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0233] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A model updating method, applied to a terminal, characterized in that: The method comprises: Obtaining a server-side model for a decision problem of a network device, wherein different user characteristics in the server-side model correspond to different model parameters, and the user characteristics include user age and user gender; Determining model parameters that match the user characteristics of the user from the server-side model to obtain a local terminal-side model of the decision problem; Training the terminal-side model according to the user data of the user to obtain a trained terminal-side model; Based on the trained terminal-side model, model information is sent to the network device, and the model information is used to indicate the model parameters of the trained terminal-side model, so that the network device performs model fusion on the terminal-side model obtained by the terminal training according to the user characteristics of the user and the fusion strategy of the model parameters corresponding to the same user characteristics, so as to update the server-side model of the decision problem.
2. The method according to claim 1, characterized in that The user data includes user data obtained from other corresponding terminals, and the user of the other terminal is the same user as the user.
3. The method according to claim 2, characterized in that The user's usage enthusiasm for the terminal is lower than the user's usage enthusiasm for the other terminals.
4. The method according to claim 1, characterized in that: Determining the model parameters matching the user characteristics of the user from the server-side model to obtain a local terminal-side model of the decision problem includes: Determine whether the local terminal side model of the current decision problem converges; If the current local terminal-side model of the decision problem has not converged, model parameters matching the user characteristics of the user are determined from the server-side model to obtain the local terminal-side model of the decision problem.
5. The method according to claim 1, characterized in that The training of the terminal side model according to the user data of the user includes: Determine whether the training conditions are met; If the training conditions are met, the terminal side model is trained according to the user data.
6. The method according to claim 5, characterized in that The training conditions include one or more of the following: the current power level is greater than a power threshold, the battery is in a standby state, the load is less than a load threshold, or the current moment is within a preset time period.
7. The method according to any one of claims 1 to 6, characterized in that: The model information includes: server-side model parameters and parameter difference information between the server-side model parameters and the terminal-side model parameters. The server-side model parameters are the model parameters of the trained terminal-side model, and the terminal-side model parameters are the model parameters of the terminal-side model before training.
8. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Receiving training information sent by the network device, where the training information is used to instruct training of a terminal-side model of the decision problem; The user data is used to train the terminal side model, including: when the training information is received, the terminal side model is trained according to the user data.
9. A model updating method, applied to a network device, characterized in that: The method comprises: Receiving model information of decision problems respectively sent by a plurality of terminals, wherein the model information is used to indicate model parameters of a terminal-side model obtained by the terminal through training according to user data; According to the user characteristics of the terminal user, a fusion strategy for fusing model parameters corresponding to the same user characteristics is used to fuse the terminal side model obtained by the terminal training to update the server side model of the decision problem, wherein the user characteristics include user age and user gender; The server-side model is sent to a target device so that the target device determines model parameters matching user features from the server-side model to obtain a local terminal-side model of the decision problem, wherein the target device includes the multiple terminals.
10. The method according to claim 9, characterized in that The model information includes: server-side model parameters and parameter difference information between the server-side model parameters and the terminal-side model parameters, the server-side model parameters are the model parameters of the terminal-side model after training, and the terminal-side model parameters are the model parameters of the terminal-side model before training for the decision problem.
11. The method according to claim 9, wherein, according to the model information, performing model fusion on the terminal side model obtained by the terminal training comprises: If the amount of the model information is greater than a first amount threshold, sorting the model information in descending order of parameter difference according to parameter difference information between the server-side model parameters and the terminal-side model parameters; Model fusion is performed on the terminal-side models obtained through training respectively indicated by the first N model information.
12. The method according to claim 9, characterized in that: The performing model fusion on the terminal side model obtained by the terminal training according to the model information includes: Determine whether the ignore condition is met; If the ignoring condition is not met, the terminal side model obtained by the terminal training is subjected to model fusion according to the model information.
13. The method according to claim 12, characterized in that The ignoring conditions include one or more of the following: the amount of model information is less than a second quantity threshold, the current server-side model of the decision problem has converged, and the parameter difference between the trained terminal-side model and the pre-trained terminal-side model is less than a difference threshold.
14. The method according to any one of claims 9 to 13, characterized in that: Before receiving the model information of the decision problem respectively sent by multiple terminals, it also includes: sending training information to the multiple terminals respectively, and the training information is used to instruct the training of the terminal side model of the decision problem.
15. The method according to claim 14, characterized in that Before sending the training information to the multiple terminals respectively, the method further includes: selecting the multiple terminals whose target activities are greater than an activity threshold according to the target activities.
16. The method according to claim 14, characterized in that The sending the training information to the plurality of terminals respectively includes: Determining whether the current server-side model of the decision problem has converged; If the current server-side model of the decision problem has not converged, training information is sent to the multiple terminals respectively.
17. The method according to any one of claims 9 to 13, characterized in that: The sending the server-side model to the target device includes: Determine whether the difference between the convergence degree of the server-side model and the historical server-side model of the decision problem before fusion is greater than a degree threshold; If the difference between the convergence degree of the server-side model and the historical server-side model is greater than the degree threshold, the server-side model is sent to the target device.
18. The method according to any one of claims 9 to 13, characterized in that: The network device includes a first intermediate device, a second intermediate device and a server; The receiving model information of the decision problem respectively sent by the multiple terminals comprises: the first intermediate device receives the server-side model information sent by the multiple first terminals among the multiple terminals, and the second intermediate device receives the terminal-side model information sent by the multiple second terminals among the multiple terminals; The performing model fusion on the terminal-side model trained by the terminal according to the model information includes: The first intermediate device fuses the terminal-side model trained by the first terminal according to the server-side model information to obtain a first fused model; The second intermediate device fuses the terminal side model trained by the second terminal according to the terminal side model information to obtain a second fused model; The server fuses the first fusion model and the second fusion model.
19. A resource recommendation method, applied to a terminal, characterized in that: include: Obtaining an access request from a user, where the access request is used to request a resource corresponding to a decision problem; The terminal side model of the decision problem is used to determine the target resource corresponding to the access request; the terminal side model is obtained in the following manner: a server side model of the decision problem of the network device is obtained, different user features in the server side model correspond to different model parameters, and the user features include user age and user gender, and the model parameters matching the user features of the user are determined from the server side model to obtain the local terminal side model of the decision problem; the server side model is obtained by the network device according to the user features of the user and the fusion strategy of the model parameters corresponding to the same user features, and the terminal side model obtained by the terminal training is fused; The target resource is recommended to the user.
20. A model updating device, characterized in that: include: An updating module is used to obtain a server-side model of a decision problem of a network device, wherein different user characteristics in the server-side model correspond to different model parameters, and the user characteristics include user age and user gender; and determine the model parameters matching the user characteristics of the user from the server-side model to obtain a local terminal-side model of the decision problem; A training module, used to train the terminal side model according to the user data of the user to obtain a trained terminal side model; A sending module is used to send model information to the network device based on the trained terminal side model, wherein the model information is used to indicate the model parameters of the trained terminal side model, so that the network device performs model fusion on the terminal side model obtained by the terminal training according to the user characteristics of the user and the fusion strategy of the model parameters corresponding to the same user characteristics, so as to update the server side model of the decision problem.
21. A model updating device, characterized in that: include: A receiving module, used to receive model information of decision problems respectively sent by multiple terminals, wherein the model information is used to indicate model parameters of a terminal-side model obtained by the terminal through training based on user data; A fusion module, configured to perform model fusion on the terminal-side model obtained by terminal training according to the user characteristics of the terminal user and the fusion strategy of fusing model parameters corresponding to the same user characteristics, so as to update the server-side model of the decision problem, wherein the user characteristics include user age and user gender; A sending module is used to send the server-side model to a target device so that the target device determines model parameters that match the user characteristics of the user from the server-side model to obtain a local terminal-side model of the decision problem, and the target device includes the multiple terminals.
22. A resource recommendation device, applied to a terminal, characterized in that: include: An acquisition module, used to acquire an access request from a user, where the access request is used to request a resource corresponding to a decision problem; A determination module is used to determine the target resource corresponding to the access request by adopting the terminal side model of the decision problem; the terminal side model is obtained in the following manner: a server side model of the decision problem of the network device is obtained, different user features in the server side model correspond to different model parameters, and user features include user age and user gender, and model parameters matching the user features of the user are determined from the server side model to obtain a local terminal side model of the decision problem, the server side model is a fusion strategy in which the network device fuses the model parameters corresponding to the same user features according to the user features of the user, and the terminal side model obtained by terminal training is fused; A recommendation module is used to recommend the target resource to the user.
23. A terminal, comprising: A memory, a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in any one of claims 1 to 8.
24. A network device comprising: A memory, a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in any one of claims 9 to 18.
25. A terminal, comprising: A memory, a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as claimed in claim 19.
26. A model updating system, characterized in that: Includes the terminal described in claim 23 and the network device described in claim 24.
