Multimedia resource-based model training method and device, equipment and storage medium

Abstract

The application discloses a model training method and device based on multimedia resources, equipment and a storage medium. The method comprises the following steps: determining a first characteristic vector of a plurality of training samples through a first model, and then performing model optimization on the first model based on the first characteristic vector of the plurality of training samples; determining an object characteristic vector of a target training sample and a resource characteristic vector of a candidate multimedia resource in a database through the optimized first model; distilling one or more candidate multimedia resources from the database based on the matching degree between the resource characteristic vector of each candidate multimedia resource and the object characteristic vector of the target training sample; constructing one or more distilled samples by using the object characteristic information of the target training sample and the distilled candidate multimedia resources respectively; and performing model training on a second model by using each distilled sample and the plurality of training samples. The application can solve the underfitting problem of the model in a small data volume scene and improve the generalization ability of the model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to model training methods, apparatus, devices and storage media based on multimedia resources. Background Technology

[0002] In recommendation scenarios with limited data, insufficient data can lead to underfitting of the trained model, low generalization ability, and a tendency to get trapped in information cocoons. Currently, to address this problem, transfer learning and meta-learning algorithms are commonly used. These algorithms utilize large datasets with ample data to assist in training the current small dataset, while also requiring a high degree of similarity between the two datasets.

[0003] However, in real-world business scenarios, there are significant differences in performance between businesses with large amounts of data and those with insufficient data. Furthermore, the overlap in user base between these two types of businesses is small. Therefore, transfer learning and meta-learning algorithms have not effectively addressed the underfitting problem of models in scenarios with limited data. Thus, finding a solution to the underfitting problem in scenarios with limited data is an urgent issue to be addressed. Summary of the Invention

[0004] This application provides a model training method, apparatus, device, and storage medium based on multimedia resources, which can solve the underfitting problem of models in scenarios with small amounts of data and improve the generalization ability of models.

[0005] On one hand, embodiments of this application provide a model training method based on multimedia resources, the method comprising:

[0006] After determining the first representation vectors of multiple training samples through the first model, the first model is optimized based on the first representation vectors of the multiple training samples. A training sample includes an object feature information and a sample multimedia resource. The first representation vector of a training sample includes: an object representation vector obtained based on the corresponding object feature information, and a resource representation vector obtained based on the corresponding sample multimedia resource.

[0007] Select a target training sample from the multiple training samples, and determine the object representation vector of the target training sample and the resource representation vector of each candidate multimedia resource in the database through the optimized first model.

[0008] Based on the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample, one or more candidate multimedia resources are distilled from the database.

[0009] One or more distillation samples are constructed by using the object feature information in the target training sample and each distilled candidate multimedia resource, and each distillation sample contains a distilled candidate multimedia resource.

[0010] The second model is trained using each distillation sample and the multiple training samples; wherein the second model includes the first model or the optimized first model.

[0011] On one hand, embodiments of this application provide a model training apparatus based on multimedia resources, the apparatus comprising:

[0012] The training unit is used to optimize the first model based on the first representation vectors of the multiple training samples after the first model determines the first representation vectors of the multiple training samples. A training sample includes an object feature information and a sample multimedia resource. The first representation vector of a training sample includes: an object representation vector obtained based on the corresponding object feature information and a resource representation vector obtained based on the corresponding sample multimedia resource.

[0013] The training unit is also used to select a target training sample from the multiple training samples, and to determine the object representation vector of the target training sample and the resource representation vector of each candidate multimedia resource in the database through the optimized first model.

[0014] The distillation unit is used to distill one or more candidate multimedia resources from the database based on the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample.

[0015] The training unit is also used to construct one or more distillation samples by using the object feature information in the target training sample and each distilled candidate multimedia resource, and each distillation sample contains a distilled candidate multimedia resource.

[0016] The training unit is also used to train the second model using each distillation sample and the plurality of training samples; wherein the second model includes the first model or the optimized first model.

[0017] On one hand, embodiments of this application provide a computer device, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the aforementioned model training method based on multimedia resources.

[0018] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program. When the computer program is read and executed by the processor of a computer device, the computer device performs the aforementioned model training method based on multimedia resources.

[0019] On one hand, embodiments of this application provide a computer program product, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned multimedia resource-based model training method.

[0020] In this embodiment, after determining the first representation vectors of multiple training samples using a first model, the first model can be optimized based on these first representation vectors. The optimized first model then determines the object representation vector of the target training sample and the resource representation vectors of each candidate multimedia resource in the database. Next, based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vector of the target training sample, one or more candidate multimedia resources can be distilled from the database. Samples are then constructed using the object feature information from the target training sample and the distilled candidate multimedia resources. This allows for the construction of more distilled samples using existing training samples without relying on external datasets, thus expanding the original training sample set. Consequently, when the second model is trained using both distilled samples and multiple training samples, it can learn from more samples, avoiding underfitting issues in scenarios with limited data. Furthermore, by distilling the database to obtain distilled samples, the diversity of training samples for the second model can be effectively improved, so that the samples used to train the second model are not limited to one type of samples. This can improve the generalization ability of the second model during the training process, and enable the trained second model to break the information cocoon effect in practical applications. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1a This is a schematic diagram of the structure of a model training system based on multimedia resources provided in an embodiment of this application;

[0023] Figure 1b This is a schematic diagram of another model training system based on multimedia resources provided in an embodiment of this application;

[0024] Figure 2 This is a flowchart illustrating a model training method based on multimedia resources provided in an embodiment of this application.

[0025] Figure 3 This is a flowchart illustrating another model training method based on multimedia resources provided in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of a self-distillation training method provided in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of an information recommendation interface provided in an embodiment of this application;

[0028] Figure 6 This is a schematic diagram of another information recommendation interface provided in an embodiment of this application;

[0029] Figure 7 This is a schematic diagram of the structure of a model training device based on multimedia resources provided in an embodiment of this application;

[0030] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0032] This application relates to Artificial Intelligence (AI) technology. AI refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines capable of reacting in a manner similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. Specifically, AI technology covers a wide range of fields, including both hardware and software technologies. At the hardware level, AI technology generally includes technologies such as sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. At the software level, AI technology mainly includes computer vision, speech processing, natural language processing, machine learning / deep learning, autonomous driving, and intelligent transportation.

[0033] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning / deep learning typically includes techniques such as artificial neural networks, supervised learning, and self-distillation. Self-distillation is a method of knowledge distillation, which refers to transferring knowledge from a pre-trained teacher model to a student model through distillation. Self-distillation uses supervised learning for knowledge distillation, thereby guiding the training of its own model based on the distilled knowledge; that is, the teacher and student models involved in the self-distillation process are essentially the same model.

[0034] Based on the self-distillation technique in machine learning / deep learning mentioned above, this application proposes a model training method based on multimedia resources to address the underfitting problem of models in scenarios with small amounts of data and improve the generalization ability of the model. Specifically, the general principle of this model training method based on multimedia resources is as follows: First, a first representation vector of multiple training samples can be determined through a first model, and the first model is optimized based on the first representation vector of multiple training samples. After obtaining the optimized first model, the object representation vector of the target training sample in the multiple training samples and the resource representation vector of each candidate multimedia resource in the database can be determined through the optimized first model. Then, based on the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample, one or more candidate multimedia resources can be distilled from the database; and one or more distilled samples can be constructed using the object feature information of the target training sample and each distilled candidate multimedia resource. Finally, the second model can be trained using each distilled sample and multiple training samples. Here, the second model can be either the first model before optimization or the first model after optimization, without limitation.

[0035] In practical implementation, the aforementioned multimedia resource-based model training method can be executed by a computer device, which can be a terminal device or a server. The terminal device can be, for example, a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or smart vehicle, but is not limited to these. The server can be, for example, a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0036] Alternatively, the multimedia resource-based model training method mentioned above can be executed jointly by the terminal and the server. For example, see... Figure 1a As shown: The terminal device can first determine the first representation vectors of multiple training samples using a first model, and then send these first representation vectors to the server. Correspondingly, after receiving the first representation vectors of the multiple training samples, the server can optimize the first model based on these vectors, and then perform steps such as distillation processing of candidate multimedia resources, construction of distilled samples, and training of the second model based on the optimized first model. See also... Figure 1bAs shown, alternatively, the terminal device can first determine the first representation vectors of multiple training samples using the first model, and then perform model optimization based on these first representation vectors. In this case, the terminal device can send the optimized first model to the server, allowing the server to perform steps such as distillation of candidate multimedia resources, construction of distilled samples, and training of the second model based on the optimized first model.

[0037] Through the embodiments of this application, for model training scenarios with insufficient data, more distilled samples are constructed using existing training samples without relying on external datasets, thus expanding the original training sample set. This allows the second model to learn from more samples when trained using various distilled samples and multiple training samples, avoiding underfitting issues in scenarios with limited data. Furthermore, obtaining distilled samples through database distillation effectively improves the diversity of the second model's training samples, ensuring that the samples used for training are not limited to a single type. This enhances the generalization ability of the second model during training, enabling the trained second model to overcome the information cocoon effect in practical applications.

[0038] It is understood that the system architecture diagrams described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0039] Based on the above explanation, the following will be combined with Figure 2 The flowchart shown further illustrates the multimedia resource-based model training method proposed in this application. In this application embodiment, the method is primarily described using the aforementioned computer device. Please refer to [link to relevant documentation]. Figure 2 The model training method based on multimedia resources may specifically include steps S201 to S205:

[0040] S201. After determining the first representation vector of multiple training samples through the first model, the first model is optimized based on the first representation vector of multiple training samples.

[0041] The first model can refer to a deep neural network model, such as a Deep Structured Semantic Model (DSSM) or a Deep Relevance Matching Model (DRMM). Any training sample used to train the first model may include object feature information and a sample multimedia resource. The object feature information in any training sample refers to information used to characterize the object's features; the object mentioned here can refer to a user or other object, and object features may include, but are not limited to, behavioral features, interest features, social features, etc. The sample multimedia resource in any training sample refers to the multimedia resource used as a training sample for the first model; the multimedia resource mentioned in this embodiment can refer to resources in various media formats such as text, sound, and images.

[0042] Furthermore, for any training sample, the multimedia resources it includes can be multimedia resources for which a feedback action has been performed by the object corresponding to that training sample, or multimedia resources for which a feedback action has not been performed by the object corresponding to that training sample. When a multimedia resource in any training sample has had a feedback action performed by the corresponding object, that training sample can be called a positive training sample; when a multimedia resource in any training sample has not had a feedback action performed by the corresponding object, that training sample can be called a negative training sample. Feedback actions can include, but are not limited to: click actions, view actions, comment actions, like actions, etc.

[0043] It should be noted that when the object is a user, the object feature information, sample multimedia resources, and other related data involved in the embodiments of this application are all obtained with the user's authorization. Furthermore, when the embodiments of this application are applied to specific products or technologies, the data used must be authorized or agreed to by the user, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0044] In a specific implementation, the computer device can first acquire multiple training samples. For any given training sample, the first model can be invoked to represent the object feature information in that training sample as a vector, obtaining the object representation vector corresponding to that training sample. Additionally, the first model can be invoked to represent the multimedia resources in that training sample as a vector, obtaining the resource representation vector corresponding to that training sample. Then, the object representation vector and the resource representation vector corresponding to that training sample can be used to construct the first representation vector of that training sample. Therefore, the first representation vector of a training sample includes: the object representation vector obtained based on the corresponding object feature information, and the resource representation vector obtained based on the corresponding multimedia resources. It should be noted that when the first model is used to represent any information (such as object feature information, multimedia resources, etc.) as a vector, it can specifically employ methods such as the Bag of Words (BOW) model or Word Embedding.

[0045] Furthermore, after obtaining the first representation vectors of multiple training samples, the computer device can optimize the first model based on the first representation vectors of multiple training samples. Specifically, the computer device can calculate the model loss value of the first model based on the matching degree between the object representation vector and the resource representation vector in the first representation vector of each training sample, and then optimize the model parameters of the first model based on the model loss value of the first model.

[0046] S202. Select a target training sample from multiple training samples, and determine the object representation vector of the target training sample and the resource representation vector of each candidate multimedia resource in the database through the optimized first model.

[0047] In practical implementation, when selecting a target training sample from multiple training samples, the computer device may randomly select one training sample as the target training sample. For example, if the multiple training samples include training sample A, training sample B, and training sample C, training sample A, training sample B, or training sample C can be determined as the target training sample.

[0048] As mentioned above, multiple training samples can include one or more positive training samples and one or more negative training samples. Optionally, in this case, when selecting the target training sample from multiple training samples, the computer device can also randomly select one positive training sample from one or more positive training samples as the target training sample. Since the multimedia resources in the positive training samples have undergone feedback behavior by the corresponding objects, the multimedia resources in the positive training samples are related to the corresponding objects. Selecting positive training samples as target training samples is more conducive to improving the model's fitting degree and accuracy.

[0049] For example, multiple training samples include training sample A, training sample B, and training sample C, where training sample A and training sample B are positive training samples, and training sample C is a negative training sample. Therefore, training sample A or training sample B can be determined as the target training sample.

[0050] After selecting the target training samples using the above method, the computer device can determine the object representation vector of the target training samples using the optimized first model; specifically, the computer device can call the optimized first model to generate the object representation vector of the target training samples. In addition, the computer device can also retrieve multiple candidate multimedia resources from the database and call the optimized first model to generate the resource representation vector of each candidate multimedia resource. Here, candidate multimedia resources refer to multimedia resources stored in the database.

[0051] S203. Based on the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample, distill one or more candidate multimedia resources from the database.

[0052] In a specific implementation, the computer device can evaluate the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample by calculating the cosine of the angle between them, or by calculating the inner product of the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample. No limitation is made here.

[0053] When a computer device distills one or more candidate multimedia resources from a database based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vector of the target training sample, it can first arrange the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vector of the target training sample in descending order, and select the candidate multimedia resources corresponding to the top K matching degrees as one or more candidate multimedia resources distilled from the database, where K is a positive integer. It should be understood that the higher the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample, the closer the candidate multimedia resource corresponding to that matching degree is to the sample multimedia resource of the target training sample. Therefore, in this way, one or more candidate multimedia resources that are similar to the sample multimedia resource of the target training sample can be distilled from the database. Subsequent training of the model using the distilled candidate multimedia resources can improve the diversity of training samples and the model's fitting degree.

[0054] Alternatively, a matching threshold can be preset, and the matching degree that reaches the matching threshold can be determined from the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample. The candidate multimedia resources that reach the matching threshold are then used as one or more candidate multimedia resources distilled from the database.

[0055] S204. Construct one or more distillation samples by using the object feature information in the target training samples and each distilled candidate multimedia resource.

[0056] A distillation sample contains a distilled candidate multimedia resource. Accordingly, in a specific implementation, a computer device can construct a distillation sample for any distilled candidate multimedia resource using object feature information from the target training sample and that distilled candidate multimedia resource. For example, training sample A is the target training sample, and the candidate multimedia resources distilled from this database include candidate multimedia resource m and candidate multimedia resource n. Therefore, two distillation samples are constructed: distillation sample x and distillation sample y. Distillation sample x includes object feature information from training sample A and candidate multimedia resource m, and distillation sample y includes object feature information from training sample A and candidate multimedia resource n.

[0057] S205. Using various distillation samples and multiple training samples, train the second model; wherein the second model includes the first model or the optimized first model.

[0058] In its implementation, the training method of the second model is similar to that of the first model (optimization method). Specifically, the training principle of the second model is roughly as follows: For any training sample, the computer device can call the second model to perform vector representation of the object feature information in that training sample, obtaining the object representation vector corresponding to that training sample; and call the second model to perform vector representation of the sample multimedia resources in that training sample, obtaining the resource representation vector corresponding to that training sample. The second representation vector of a training sample includes: the object representation vector obtained based on the corresponding object feature information, and the resource representation vector obtained based on the corresponding sample multimedia resources. Similarly, for any distillation sample, the computer device can call the second model to perform vector representation of the object feature information in that distillation sample, obtaining the object representation vector corresponding to that distillation sample; and call the second model to perform vector representation of the candidate multimedia resources in that distillation sample, obtaining the resource representation vector corresponding to that training sample. The third representation vector of a distillation sample includes: the object representation vector obtained based on the corresponding object feature information, and the resource representation vector obtained based on the corresponding candidate multimedia resources.

[0059] Furthermore, after obtaining the second representation vectors of multiple training samples and the third representation vectors of each distillation sample, the computer device can optimize the second model based on the second representation vectors of multiple training samples and the third representation vectors of each distillation sample. Specifically, the computer device can calculate the model loss value of the second model based on the matching degree between the object representation vector and the resource representation vector in the second representation vector of each training sample and the matching degree between the object representation vector and the resource representation vector in the third representation vector of each distillation sample, and thus optimize the model parameters of the second model based on the model loss value of the second model.

[0060] In summary, in this embodiment, after determining the first representation vectors of multiple training samples using the first model, the first model is optimized based on these first representation vectors to obtain an optimized first model. Then, a target training sample selected from the multiple training samples is used to determine the object representation vector of the target training sample through the optimized first model, and the resource representation vectors of each candidate multimedia resource in the database are also determined through the optimized first model. Next, based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vectors of the target training sample, one or more candidate multimedia resources are distilled from the database, and the object feature information in the target training sample and each distilled candidate multimedia resource are used to construct samples. This allows for the construction of more distilled samples using existing training samples without relying on external datasets, thus expanding the original training sample set. Consequently, when the second model is trained using each distilled sample and multiple training samples, the second model can learn more samples, avoiding underfitting problems in scenarios with small amounts of data. Furthermore, by distilling the database to obtain distilled samples, the diversity of training samples for the second model can be effectively improved, so that the samples used to train the second model are not limited to one type of samples. This can improve the generalization ability of the second model during the training process, and enable the trained second model to break the information cocoon effect in practical applications.

[0061] The following is combined Figure 3 The flowchart shown further illustrates the multimedia resource-based model training method proposed in this application. In this application embodiment, the method is primarily described using the aforementioned computer device. Please refer to [link to relevant documentation]. Figure 3 The model training method based on multimedia resources may specifically include steps S301 to S306, where steps S303 and S304 are specific implementations of step S203. Specifically:

[0062] S301. After determining the first representation vector of multiple training samples through the first model, the first model is optimized based on the first representation vector of multiple training samples.

[0063] In this embodiment, the multiple training samples include one or more positive training samples and one or more negative training samples; the multimedia resources in the positive training samples have had feedback actions performed by the objects indicated by the corresponding object feature information, while the multimedia resources in the negative training samples have not had feedback actions performed by the objects indicated by the corresponding object feature information. For example, assuming the feedback action includes a click-to-view action, it can be understood that: the multimedia resources in the positive training samples have been clicked and viewed by the user, while the multimedia resources in the negative training samples have not been clicked and viewed by the user. In a specific implementation, step S301 may include the following steps s11-s14:

[0064] s11, calculate the first matching degree between the object representation vector in the first representation vector of each positive training sample and the corresponding first resource representation vector; and calculate the second matching degree between the object representation vector in the first representation vector of each negative training sample and the corresponding first resource representation vector.

[0065] Specifically, for any first representation vector, the computer device may evaluate the matching degree between the two vectors by calculating the cosine value of the angle between the object representation vector and the resource representation vector in the first representation vector, or by calculating the vector inner product, without limitation.

[0066] s12, traverse each positive training sample in the training sample set, and calculate the loss value generated by the first model through the current positive training sample based on the first matching degree corresponding to the current positive training sample and the second matching degree corresponding to each negative training sample.

[0067] The first matching degree can be converted into the posterior probability, i.e., the posterior probability corresponding to the positive training sample, by using the normalized exponential function softmax. The posterior probability corresponding to any positive training sample refers to the probability predicted based on the object feature information in any positive training sample that the multimedia resource in that positive training sample will have its feedback behavior executed. The posterior probability corresponding to any positive training sample can be calculated using formula (1), as shown below:

[0068]

[0069] Where Q is the object representation vector in the first representation vector of the training sample, and D is the training sample set. + D is the first resource representation vector corresponding to the object representation vector of the positive training sample. - R(Q,D) is the first resource representation vector corresponding to the object representation vector of the negative training sample. +R(Q,D) represents the first matching degree between the object representation vector and the corresponding first resource representation vector in the first representation vector of the positive training samples. - P(D) represents the second matching degree between the object representation vector and the corresponding first resource representation vector in the first representation vector of the negative training samples, γ is a trainable parameter, and P(D) represents the second matching degree between the object representation vector and the corresponding first resource representation vector in the first representation vector of the negative training samples. + |Q) represents the posterior probability of any positive training sample.

[0070] Furthermore, by iterating through each positive training sample in the training sample set, the loss function is minimized through maximum likelihood estimation, thereby calculating the loss value generated by the first model through the current positive training sample. Specifically, it can be calculated using formula (2), as shown below:

[0071]

[0072] Where Q is the object representation vector in the first representation vector of the training samples, and D + Let P(D) be the first resource representation vector corresponding to the object representation vector of the positive training sample. + |Q) represents the posterior probability corresponding to any positive training sample, Λ represents the trainable parameter, and L(Λ) represents the loss value generated by the first model through the current positive training sample.

[0073] s13, after obtaining the loss values ​​generated by the first model through each positive training sample, integrate the obtained loss values ​​to obtain the model loss value generated by the first model through the multiple training samples.

[0074] The method of integrating the loss values ​​generated by the first model through the positive training samples can be to calculate or sum the loss values, or to perform weighted summation of the loss values; no limitation is made here.

[0075] It should be noted that if a weighted summation or weighted average method is used to integrate the various loss values, the weights of each loss value can be pre-set, or determined based on the time it takes for the corresponding multimedia resource to receive feedback. In other words, the time it takes for the multimedia resource to receive feedback corresponds to the weight of the loss value. It should be understood that the shorter the time it takes for the multimedia resource to receive feedback, the higher the demand for that multimedia resource from the object receiving the feedback. Therefore, determining the weight of each loss value based on the time it takes for the corresponding multimedia resource to receive feedback helps improve the model's accuracy.

[0076] For example, the loss value 'a' generated by positive training sample A is 0.3, and the time for the multimedia resource corresponding to loss value 'a' to perform the feedback action is 40ms; the loss value 'b' generated by positive training sample B is 0.6, and the time for the multimedia resource corresponding to loss value 'a' to perform the feedback action is 60ms. Therefore, based on the time for the multimedia resource to perform the feedback action corresponding to loss values ​​'a' and 'b', the weight corresponding to loss value 'a' can be determined as 0.6, and the weight corresponding to loss value 'b' can be determined as 0.4. Therefore, after integrating loss values ​​'a' and 'b', the model loss value generated by the first model through training samples A and B is 0.42.

[0077] s14, update the model parameters of the first model in the direction of reducing the model loss value generated by the first model through the multiple training samples.

[0078] The method for updating the model parameters of the first model can be stochastic gradient descent, adaptive gradient algorithm, etc., and is not limited here.

[0079] S302. Select a target training sample from multiple training samples, and determine the object representation vector of the target training sample and the resource representation vector of each candidate multimedia resource in the database through the optimized first model.

[0080] In this embodiment, the computer device selects a target training sample from multiple training samples. Specifically, this can be achieved by: filtering out one or more positive training samples from the multiple training samples; and randomly selecting one positive training sample from the one or more positive training samples as the target training sample. It should be understood that the multiple training samples include positive and negative training samples. One or more positive training samples are filtered out from the multiple training samples, and then one positive training sample is randomly selected as the target training sample. For example, the multiple training samples include training sample A, training sample B, training sample C, and training sample D, and the filtered positive training samples include training sample A and training sample B. Randomly selecting a positive training sample as the target training sample can be done, for example, selecting training sample A as the target training sample, or selecting training sample B as the target training sample.

[0081] S303. In accordance with the recall order from high to low matching degree, one or more candidate multimedia resources are recalled from the database based on the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample.

[0082] S304. Distill the recalled candidate multimedia resources from the database.

[0083] In this embodiment, the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample is arranged in descending order of recall. The candidate multimedia resources corresponding to the top K matching degrees are selected as one or more candidate multimedia resources distilled from the database. Then, each recalled candidate multimedia resource is distilled from the database. Here, K is a positive integer.

[0084] For example, K is 2. The matching degree between the resource representation vector of candidate multimedia resource 1 and the object representation vector of the target training sample is 0.3, the matching degree between the resource representation vector of candidate multimedia resource 2 and the object representation vector of the target training sample is 0.62, and the matching degree between the resource representation vector of candidate multimedia resource 2 and the object representation vector of the target training sample is 0.82. These matching degrees are arranged in descending order of recall. The candidate multimedia resources corresponding to the first two matching degrees are selected as one or more candidate multimedia resources distilled from the database. That is, candidate multimedia resource 1 and candidate multimedia resource 2 are recalled from the database, and then the recalled candidate multimedia resource 1 and candidate multimedia resource 2 are distilled from the database.

[0085] S305. Construct one or more distillation samples by using the object feature information in the target training samples and each distilled candidate multimedia resource.

[0086] The specific implementation method of step S305 is the same as that of step S204 above, and will not be described in detail here.

[0087] S306. Using each distillation sample and the multiple training samples, train the second model; wherein the second model includes the first model or the optimized first model.

[0088] In a specific implementation, step S306 may include the following steps s21-s24:

[0089] s21, determine the second representation vector of multiple training samples through the second model, and determine the model loss value generated by the second model through the multiple training samples based on the second representation vector of each training sample.

[0090] Referring to the above implementation method of determining the first representation vector of multiple training samples through the first model, the implementation method of determining the second representation vector of multiple training samples through the second model by the computer device can be as follows: for any training sample, the computer device can call the second model to perform vector representation of the object feature information in the any training sample to obtain the object representation vector corresponding to the any training sample; and call the second model to perform vector representation of the sample multimedia resources in the any training sample to obtain the resource representation vector corresponding to the any training sample. The second representation vector of a training sample includes: the object representation vector obtained based on the corresponding object feature information, and the resource representation vector obtained based on the corresponding sample multimedia resources.

[0091] Referring to the above implementation method of determining the model loss value generated by the first model through multiple training samples based on the first representation vector of each training sample, the implementation method of determining the model loss value generated by the second model through multiple training samples based on the second representation vector of each training sample can be as follows: multiple training samples include positive training samples and negative training samples; calculate the third matching degree between the object representation vector in the second representation vector of each positive training sample and the corresponding second resource representation vector; and calculate the fourth matching degree between the object representation vector in the second representation vector of each negative training sample and the corresponding second resource representation vector; traverse each positive training sample in the training sample set, and calculate the loss value generated by the second model through the current positive training sample based on the third matching degree corresponding to the current positive training sample and the fourth matching degree corresponding to each negative training sample; after obtaining the loss value generated by the second model through each positive training sample, integrate the obtained loss values ​​to obtain the model loss value generated by the second model through multiple training samples.

[0092] s22, the third representation vector of each distillation sample is determined by the second model, and the loss value generated by the second model through each distillation sample is calculated based on the third representation vector of each distillation sample.

[0093] Referring to the above implementation method of determining the first representation vector of multiple training samples through the first model, the implementation method of determining the third representation vector of each distillation sample through the second model by the computer device can be as follows: for any distillation sample, the computer device can call the second model to perform vector representation of the object feature information in the any distillation sample to obtain the object representation vector corresponding to the any distillation sample; and call the second model to perform vector representation of the candidate multimedia resources in the any distillation sample to obtain the resource representation vector corresponding to the any training sample. The third representation vector of a distillation sample includes: the object representation vector obtained based on the corresponding object feature information, and the resource representation vector obtained based on the corresponding candidate multimedia resources.

[0094] Referring to the above implementation method of determining the model loss value generated by the first model through multiple training samples based on the first representation vector of each training sample, the implementation method of the computer device calculating the loss value generated by the second model through each distillation sample based on the third representation vector of each distillation sample can be as follows: multiple training samples include distillation samples and negative training samples; calculate the fifth matching degree between the object representation vector in the third representation vector of each distillation sample and the corresponding third resource representation vector; and calculate the sixth matching degree between the object representation vector in the third representation vector of each negative training sample and the corresponding third resource representation vector; traverse each distillation sample in the training sample set, and calculate the loss value generated by the second model through the current distillation sample based on the fifth matching degree corresponding to the current distillation sample and the sixth matching degree corresponding to each negative training sample; after obtaining the loss value generated by the second model through each distillation sample, integrate the obtained loss values ​​to obtain the model loss value generated by the second model through each distillation sample.

[0095] s23, determine the loss weight of each distillation sample, and use the loss weight of each distillation sample to perform a weighted summation of the loss values ​​generated by the second model through each distillation sample, so as to obtain the model loss value generated by the second model through all distillation samples.

[0096] In practice, by assigning appropriate weights to the loss values ​​generated by each distillation sample, model noise can be reduced and the accuracy of model training can be improved. For example, the loss weight of distillation sample x is 0.62, the loss weight of distillation sample y is 0.82, the loss value of distillation sample x is 0.6, and the loss value of distillation sample y is 0.3. Therefore, the model loss value generated by this second model through all distillation samples is 0.618.

[0097] In one possible implementation, the computer device determines the loss weights for each distillation sample. Specifically, this can be achieved by: for any distillation sample, determining the matching degree between the resource representation vector corresponding to the candidate multimedia resource in that distillation sample and the object representation vector of the target training sample; and calculating the loss weight for that distillation sample based on the determined matching degree. It should be understood that the matching degree between the resource representation vector corresponding to the candidate multimedia resource in any distillation sample determined by the first model and the object representation vector of the target training sample can be used as the loss weight for that distillation sample.

[0098] For example, the resource representation vector corresponding to the candidate multimedia resource in the distillation sample x determined by the first model is Q, and the object representation vector of the target training sample is P. The matching degree between the resource representation vector corresponding to the candidate multimedia resource in the distillation sample x and the object representation vector of the target training sample is 0.62. Therefore, the loss weight of the distillation sample x is determined to be 0.62. The resource representation vector corresponding to the candidate multimedia resource in the distillation sample y determined by the first model is R, and the object representation vector of the target training sample is T. The matching degree between the resource representation vector corresponding to the candidate multimedia resource in the distillation sample y and the object representation vector of the target training sample is 0.82. Therefore, the loss weight of the distillation sample x is determined to be 0.82.

[0099] Alternatively, the matching degree between the resource representation vector corresponding to the candidate multimedia resources in each distillation sample and the object representation vector of the target training sample can be normalized, and then the normalized matching degree of each distillation sample can be used as the loss weight of any distillation sample. The normalization process can employ linear function transformation, logarithmic function transformation, etc., and is not limited here.

[0100] For example, let Q be the resource representation vector corresponding to the candidate multimedia resource in distillation sample x determined by the first model, and P be the object representation vector of the target training sample. The matching degree between the resource representation vector corresponding to the candidate multimedia resource in distillation sample x and the object representation vector of the target training sample is 0.62. Let R be the resource representation vector corresponding to the candidate multimedia resource in distillation sample y determined by the first model, and T be the object representation vector of the target training sample. The matching degree between the resource representation vector corresponding to the candidate multimedia resource in distillation sample y and the object representation vector of the target training sample is 0.82. The matching degrees of distillation sample x and distillation sample y are processed using a linear function transformation normalization method, resulting in a normalized matching degree of 0 for distillation sample x and 1 for distillation sample y. The normalized matching degree of each distillation sample is used as the loss weight for any distillation sample, i.e., the loss weight of distillation sample x is 0, and the loss weight of distillation sample y is 1.

[0101] In one possible implementation, the computer device determines the loss weight of each distillation sample. Specifically, it can be implemented by: for any distillation sample, calculating the matching degree between the object representation vector and the resource representation vector in the third representation vector of that distillation sample; and calculating the loss weight of that distillation sample based on the calculated matching degree.

[0102] For example, the resource representation vector corresponding to the candidate multimedia resource in distillation sample x determined by the optimized first model is Q', and the object representation vector of the target training sample is P'. The matching degree between the resource representation vector corresponding to the candidate multimedia resource in distillation sample x and the object representation vector of the target training sample is 0.65. Therefore, the loss weight of distillation sample x is determined to be 0.65. The resource representation vector corresponding to the candidate multimedia resource in distillation sample y determined by the first model is R', and the object representation vector of the target training sample is T'. The matching degree between the resource representation vector corresponding to the candidate multimedia resource in distillation sample y and the object representation vector of the target training sample is 0.85. Therefore, the loss weight of distillation sample x is determined to be 0.85. Similarly, the matching degree between the resource representation vector corresponding to the candidate multimedia resource in each distillation sample determined by the optimized first model and the object representation vector of the target training sample can be normalized, and the normalized matching degree of each distillation sample can be used as the loss weight of any distillation sample. This will not be elaborated here.

[0103] In one possible implementation, the computer device determines the loss weights for each distillation sample. Specifically, this can be achieved by: for any distillation sample, determining the seventh matching degree between the candidate multimedia resources in that distillation sample and the object representation vector of the target training sample; calculating the eighth matching degree between the object representation vector and the resource representation vector in the third representation vector of that distillation sample; weighting and summing the seventh and eighth matching degrees based on their respective weights to obtain the target matching degree; and calculating the loss weight for that distillation sample based on the target matching degree. It should be understood that the matching degree between the resource representation vector corresponding to the candidate multimedia resources in any distillation sample determined by the optimized first model and the object representation vector of the target training sample can be used as the loss weight for any distillation sample. It should also be understood that the target matching degree is obtained by weighted summing of the matching degrees of any distillation sample determined through the above two methods, and this target matching degree is used as the loss weight for that distillation sample.

[0104] For example, the seventh matching degree between the resource representation vector corresponding to the candidate multimedia resource in the distillation sample x determined by the first model and the object representation vector of the target training sample is 0.62. The eighth matching degree between the resource representation vector corresponding to the candidate multimedia resource in the distillation sample x determined by the optimized first model and the object representation vector of the target training sample is 0.65. The weight corresponding to the seventh matching degree is 0.4, and the weight corresponding to the eighth matching degree is 0.6. Based on the weight of the seventh matching degree and the weight of the eighth matching degree, the seventh matching degree and the eighth matching degree are weighted and summed to obtain the target matching degree of the distillation sample x as 0.638. Therefore, the target matching degree is used as the loss weight of the distillation sample x, i.e., 0.638.

[0105] s24, sum the model loss values ​​generated by the second model through the multiple training samples and the model loss values ​​generated by the second model through all distilled samples to obtain the total model loss value of the second model.

[0106] For example, the model loss value generated by the second model through the multiple training samples is 18, the model loss value generated by the second model through all distilled samples is 10, and the total model loss value of the second model is the sum of the model loss value generated by the second model through the multiple training samples and the model loss value generated by the second model through all distilled samples, which is 28.

[0107] s25, update the model parameters of the second model in the direction of reducing the total model loss value.

[0108] Similarly, the model parameters of the second model can also be updated using stochastic gradient descent, adaptive gradient algorithm, etc., and are not limited here.

[0109] In summary, the model trained using the multimedia resource-based model training method proposed in this application can be used in information recommendation systems. Figure 4 As shown, Figure 4This is a schematic diagram of a self-distillation training method provided in an embodiment of this application. First, the Teacher model (which can be understood as the first model mentioned above) is trained using a training dataset. Then, the Teacher model distills the top k distilled samples. Finally, the original training dataset and the top k distilled samples are used together to train the Student model (which can be understood as the second model mentioned above). This self-distillation training method can supplement high-confidence distilled samples, solving the problem of insufficient data volume. It also alleviates the problems of weak model divergence and the tendency to fall into information cocoons, and mitigates the situation where multimedia resources of interest to users are sampled as negative samples: First, since the supplemented distilled samples are data that has not actually been exposed to users, the range of user interests is expanded during the training of the second model, alleviating the "information cocoon" phenomenon. Second, since the top K most relevant multimedia resources of interest to users are selected to construct the distilled samples, when the random negative sampling algorithm samples these multimedia resources of interest to users, since the distilled samples are all positive training samples, it can play a certain mitigating role.

[0110] Please refer to Table 1, which is a statistical table of online recall rate indicators provided in an embodiment of this application. The HR@200 indicator represents the proportion of the first test sample to all test samples. The first test sample is defined as follows: after recalling the top 200 candidate multimedia resources that match the object representation information in any test sample from the database, if the sample multimedia resource in any test sample is among the top 200 candidate multimedia resources, then that test sample is called the first test sample. The HR@500 indicator represents the proportion of the second test sample to all test samples. The second test sample is defined as follows: after recalling the top 500 candidate multimedia resources that match the object representation information in any test sample from the database, if the sample multimedia resource in any test sample is among the top 500 candidate multimedia resources, then that test sample is called the second test sample. As shown in Table 1, the model trained using only the original data achieved an HR@200 score of 26.00% and an HR@500 score of 38.23%; while the model trained using the self-distillation technique achieved an HR@200 score of 27.39% and an HR@500 score of 40.16%. Therefore, the multimedia resource-based model training method proposed in this paper solves the underfitting problem in scenarios with limited data and improves the model's fit.

[0111] Table 1

[0112] Plans / Indicators HR@200 HR@500 Use only raw data 26.00％ 38.23％ Combined with self-distillation technology 27.39％ 40.16％

[0113] Please refer to Table 2, which is a statistical table of online recall categories provided in this application embodiment. The number of categories appearing in the Top 100 recalls represents the number of categories of 100 candidate multimedia resources recalled in a test sample; the Top 3 category concentration represents the proportion of the three most frequently occurring candidate multimedia resources in a single recall among all candidate multimedia resources. These two indicators are used to measure the diversity of candidate multimedia resources recalled by the model. As shown in Table 2, the model trained using only the original data has 7.86 categories appearing in the Top 100 recalls and a Top 3 category concentration of 58.23%; the model trained using the self-distillation technique has 10.71 categories appearing in the Top 100 recalls and a Top 3 category concentration of 43.87%. Therefore, the multimedia resource-based model training method proposed in this solution improves the model's generalization ability, expands user interests, and breaks the information cocoon effect.

[0114] Table 2

[0115] Model / Indicator Number of items appearing in the Top 100 recalls Top 3 Category Concentration Model trained using only raw data 7.86 58.23％ Model trained using self-distillation technology 10.71 43.87％

[0116] Please see Figure 5 , Figure 5 This is a schematic diagram of an information recommendation interface provided in an embodiment of this application. For example... Figure 4 As shown, the information recommendation interface includes a search box, a "Audiobook Library" display box, a "What Everyone's Watching" display box, a "Friends' Thoughts" button, a "Today's Update" button, a "Friends' Likes" button, a "Claim for Free" button, and a "View All" button. Users can search for the information they want in the search box. The "Audiobook Library" display box shows recommended works; clicking the "Claim for Free" button allows users to view the content of that work, and clicking the "View All" button allows users to view more recommended works. The "What Everyone's Watching" display box shows recommended news articles; users can click on the relevant news articles to view their content. Additionally, the "Friends' Thoughts," "Today's Update," and "Friends' Likes" buttons allow users to view more recommended content.

[0117] Please see Figure 6 , Figure 6 This is a schematic diagram of another information recommendation interface provided in an embodiment of this application. For example... Figure 5 As shown, the information recommendation interface includes a "Hot Topics" display box, a "Watched by Film and Television Enthusiasts" display box, and a "Recommended" display box. The "Hot Topics" display box shows current trending topics, which users can view by clicking; the "Watched by Film and Television Enthusiasts" display box shows popular films and television shows, which users can watch by clicking; and the "Recommended" display box shows recommended news articles, which users can view by clicking.

[0118] In summary, in this embodiment, after determining the first representation vectors of multiple training samples using the first model, the first model is optimized based on these first representation vectors to obtain an optimized first model. Then, a target training sample is selected from these multiple training samples, and its object representation vector is determined using the optimized first model. The resource representation vectors of each candidate multimedia resource in the database are also determined using the optimized first model. Next, following a recall order from high to low matching degree, one or more candidate multimedia resources are recalled from the database based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vector of the target training sample. The recalled candidate multimedia resources are then distilled from the database. Finally, the object feature information in the target training samples and the distilled candidate multimedia resources are used to construct samples. This allows for the construction of more distilled samples using existing training samples without relying on external datasets, thus expanding the original training sample set. Consequently, when the second model is trained using both distilled samples and multiple training samples, it can learn more samples, avoiding underfitting issues in scenarios with limited data. Furthermore, by distilling the database to obtain distilled samples, the diversity of training samples for the second model can be effectively improved, so that the samples used to train the second model are not limited to one type of samples. This can improve the generalization ability of the second model during the training process, and enable the trained second model to break the information cocoon effect in practical applications.

[0119] Based on the aforementioned multimedia resource-based model training method, this application provides a multimedia resource-based model training apparatus. Please refer to... Figure 7 This is a schematic diagram of a model training device based on multimedia resources provided in an embodiment of this application. The model training device 700 based on multimedia resources can operate the following units:

[0120] The training unit 701 is used to optimize the first model based on the first representation vectors of the multiple training samples after determining the first representation vectors of the multiple training samples through the first model. A training sample includes an object feature information and a sample multimedia resource. The first representation vector of a training sample includes: an object representation vector obtained based on the corresponding object feature information and a resource representation vector obtained based on the corresponding sample multimedia resource.

[0121] The training unit 701 is also used to select a target training sample from the plurality of training samples, and to determine the object representation vector of the target training sample and the resource representation vector of each candidate multimedia resource in the database through the optimized first model.

[0122] Distillation unit 702 is used to distill one or more candidate multimedia resources from the database based on the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample.

[0123] The training unit 701 is also used to construct one or more distillation samples by using the object feature information in the target training sample and each distilled candidate multimedia resource, and each distillation sample contains a distilled candidate multimedia resource.

[0124] The training unit 701 is also used to train the second model using each distillation sample and the plurality of training samples; wherein the second model includes the first model or the optimized first model.

[0125] In one embodiment, the distillation unit 702, when distilling one or more candidate multimedia resources from the database based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vector of the target training sample, may specifically be used to:

[0126] Based on the matching degree from high to low, one or more candidate multimedia resources are recalled from the database according to the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample.

[0127] The various candidate multimedia resources to be recalled were distilled from the database.

[0128] In another embodiment, the plurality of training samples includes one or more positive training samples and one or more negative training samples; the multimedia resources in the positive training samples have been subjected to feedback behavior by the objects indicated by the corresponding object feature information, while the multimedia resources in the negative training samples have not been subjected to feedback behavior by the objects indicated by the corresponding object feature information; the training unit 701, when optimizing the first model based on the first representation vector of the plurality of training samples, may also be specifically used for:

[0129] Calculate the first matching degree between the object representation vector in the first representation vector of each positive training sample and the corresponding first resource representation vector; and calculate the second matching degree between the object representation vector in the first representation vector of each negative training sample and the corresponding first resource representation vector.

[0130] Iterate through each positive training sample in the training sample set, and calculate the loss value generated by the first model through the current positive training sample based on the first matching degree corresponding to the current positive training sample and the second matching degree corresponding to each negative training sample.

[0131] After obtaining the loss values ​​generated by the first model through each positive training sample, the obtained loss values ​​are integrated to obtain the model loss value generated by the first model through the multiple training samples.

[0132] The model parameters of the first model are updated in the direction of reducing the model loss value generated by the first model through the multiple training samples.

[0133] In another embodiment, the training unit 701, when selecting the target training sample from the plurality of training samples, may also be specifically used for:

[0134] From these multiple training samples, select one or more positive training samples;

[0135] From the one or more positive training samples, one positive training sample is randomly selected as the target training sample.

[0136] In another embodiment, when training the second model using the various distillation samples and the plurality of training samples, the training unit 701 may also be specifically used for:

[0137] The second model determines the second representation vector of the multiple training samples, and the model loss value generated by the second model through the multiple training samples is determined based on the second representation vector of each training sample.

[0138] The second model determines the third representation vector of each distillation sample, and calculates the loss value generated by the second model through each distillation sample based on the third representation vector of each distillation sample.

[0139] Determine the loss weight of each distillation sample, and use the loss weight of each distillation sample to perform a weighted sum of the loss values ​​generated by the second model through each distillation sample, so as to obtain the model loss value generated by the second model through all distillation samples.

[0140] The total model loss value of the second model is obtained by summing the model loss value generated by the second model through the multiple training samples and the model loss value generated by the second model through all distilled samples.

[0141] Update the model parameters of the second model in the direction of reducing the total model loss value.

[0142] In another implementation, the training unit 701, when determining the loss weights of each distillation sample, may also be specifically used for:

[0143] For any distillation sample, determine the matching degree between the resource representation vector corresponding to the candidate multimedia resource in the distillation sample and the object representation vector of the target training sample;

[0144] Based on the determined matching degree, the loss weight of any distilled sample is calculated.

[0145] In another implementation, the training unit 701, when determining the loss weights of each distillation sample, may also be specifically used for:

[0146] For any distillation sample, calculate the matching degree between the object representation vector and the resource representation vector in the third representation vector of that distillation sample;

[0147] Based on the calculated matching degree, the loss weight of any distilled sample is calculated.

[0148] According to another embodiment of this application, Figure 7 The various units in the multimedia resource-based model training device shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the multimedia resource-based model training device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0149] According to another embodiment of this application, the following can be achieved by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), a device capable of performing operations such as... Figure 2 or Figure 3 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 7 The diagram illustrates a multimedia resource-based model training apparatus and a multimedia resource-based model training method for implementing embodiments of this application. The computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the same medium, and run therein.

[0150] In this embodiment, after determining the first representation vectors of multiple training samples using a first model, the first model is optimized based on these first representation vectors. A training sample includes object feature information and a sample multimedia resource. The first representation vector of a training sample includes an object representation vector obtained based on the corresponding object feature information and a resource representation vector obtained based on the corresponding sample multimedia resource. A target training sample is selected from the multiple training samples, and the object representation vector of the target training sample and the resource representation vectors of each candidate multimedia resource in the database are determined using the optimized first model. Based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vector of the target training sample, one or more candidate multimedia resources are distilled from the database. Samples are constructed using the object feature information in the target training sample and the distilled candidate multimedia resources. This allows for the construction of more distilled samples using existing training samples without relying on external datasets, expanding the original training sample set. Consequently, when the second model is trained using various distilled samples and multiple training samples, the second model can learn more samples, avoiding underfitting problems in scenarios with small amounts of data. Furthermore, by distilling the database to obtain distilled samples, the diversity of training samples for the second model can be effectively improved, so that the samples used to train the second model are not limited to one type of samples. This can improve the generalization ability of the second model during the training process, and enable the trained second model to break the information cocoon effect in practical applications.

[0151] Based on the descriptions of the method and device embodiments above, this application also provides a terminal. Please refer to... Figure 8 The terminal includes at least a processor 801, an input interface 802, an output interface 803, and a computer storage medium 804. The processor 801, input interface 802, output interface 803, and computer storage medium 804 within the terminal can be connected via a bus or other means. The computer storage medium 804 can be stored in the terminal's memory. The computer storage medium 804 is used to store computer programs, which include program instructions. The processor 801 is used to execute the program instructions stored in the computer storage medium 804. The processor 801 (or CPU (Central Processing Unit)) is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve corresponding method flows or corresponding functions.

[0152] In one embodiment, the processor 801 described in this application embodiment can be used to perform a series of model training operations, specifically including: after determining the first representation vectors of multiple training samples through a first model, optimizing the first model based on the first representation vectors of the multiple training samples; a training sample includes object feature information and a sample multimedia resource, and the first representation vector of a training sample includes: an object representation vector obtained based on the corresponding object feature information, and a resource representation vector obtained based on the corresponding sample multimedia resource; selecting a target training sample from the multiple training samples, and determining the object representation vector of the target training sample and the resource representation vectors of each candidate multimedia resource in the database through the optimized first model; distilling one or more candidate multimedia resources from the database based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vector of the target training sample; constructing one or more distilled samples using the object feature information in the target training sample and each distilled candidate multimedia resource, each distilled sample containing one distilled candidate multimedia resource; training a second model using each distilled sample and the multiple training samples; wherein the second model includes the first model or the optimized first model, etc.

[0153] This application embodiment also provides a computer storage medium (memory), which is a memory device in a terminal used to store programs and data. It is understood that the computer storage medium here can include both the built-in storage medium in the terminal and extended storage media supported by the terminal. The computer storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by the processor 801. These instructions can be one or more computer programs (including program code). It should be noted that the computer storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer storage medium located remotely from the aforementioned processor.

[0154] In one embodiment, a processor may load and execute one or more instructions stored in a computer storage medium to achieve the aforementioned... Figure 2 or Figure 3 The corresponding steps of the method in the embodiment of the model training method based on multimedia resources are shown; in specific implementation, one or more instructions in the computer storage medium are loaded by the processor 801 and executed as follows:

[0155] After determining the first representation vectors of multiple training samples through the first model, the first model is optimized based on the first representation vectors of the multiple training samples. A training sample includes an object feature information and a sample multimedia resource. The first representation vector of a training sample includes: an object representation vector obtained based on the corresponding object feature information, and a resource representation vector obtained based on the corresponding sample multimedia resource.

[0156] Select a target training sample from the multiple training samples, and determine the object representation vector of the target training sample and the resource representation vector of each candidate multimedia resource in the database through the optimized first model.

[0157] Based on the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample, one or more candidate multimedia resources are distilled from the database.

[0158] One or more distillation samples are constructed by using the object feature information in the target training sample and each distilled candidate multimedia resource, and each distillation sample contains a distilled candidate multimedia resource.

[0159] The second model is trained using each distillation sample and the multiple training samples; wherein the second model includes the first model or the optimized first model.

[0160] In one implementation, when distilling one or more candidate multimedia resources from the database based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vector of the target training sample, the one or more instructions can be loaded and executed by the processor:

[0161] Based on the matching degree from high to low, one or more candidate multimedia resources are recalled from the database according to the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample; and the recalled candidate multimedia resources are distilled from the database.

[0162] In another implementation, the plurality of training samples includes one or more positive training samples and one or more negative training samples; the multimedia resources in the positive training samples have been subjected to feedback behavior by the objects indicated by the corresponding object feature information, while the multimedia resources in the negative training samples have not been subjected to feedback behavior by the objects indicated by the corresponding object feature information; when optimizing the first model based on the first representation vector of the plurality of training samples, the one or more instructions can be loaded and executed by the processor:

[0163] Calculate the first matching degree between the object representation vector in the first representation vector of each positive training sample and the corresponding first resource representation vector; and calculate the second matching degree between the object representation vector in the first representation vector of each negative training sample and the corresponding first resource representation vector.

[0164] Iterate through each positive training sample in the training sample set, and calculate the loss value generated by the first model through the current positive training sample based on the first matching degree corresponding to the current positive training sample and the second matching degree corresponding to each negative training sample.

[0165] After obtaining the loss values ​​generated by the first model through each positive training sample, the obtained loss values ​​are integrated to obtain the model loss value generated by the first model through the multiple training samples.

[0166] The model parameters of the first model are updated in the direction of reducing the model loss value generated by the first model through the multiple training samples.

[0167] In another implementation, when selecting a target training sample from the plurality of training samples, the one or more instructions may be loaded and executed by the processor:

[0168] From the multiple training samples, one or more positive training samples are selected; from the one or more positive training samples, one positive training sample is randomly selected as the target training sample.

[0169] In another implementation, when training the second model using the various distillation samples and the plurality of training samples, the one or more instructions can be loaded and executed by the processor:

[0170] The second model determines the second representation vector of the multiple training samples, and the model loss value generated by the second model through the multiple training samples is determined based on the second representation vector of each training sample.

[0171] The second model determines the third representation vector of each distillation sample, and calculates the loss value generated by the second model through each distillation sample based on the third representation vector of each distillation sample.

[0172] Determine the loss weight of each distillation sample, and use the loss weight of each distillation sample to perform a weighted sum of the loss values ​​generated by the second model through each distillation sample to obtain the model loss value generated by the second model through all distillation samples.

[0173] The total model loss value of the second model is obtained by summing the model loss value generated by the second model through the multiple training samples and the model loss value generated by the second model through all distilled samples.

[0174] Update the model parameters of the second model in the direction of reducing the total model loss value.

[0175] In another implementation, when determining the loss weight of each distillation sample, the one or more instructions can be loaded and executed by the processor:

[0176] For any distillation sample, determine the matching degree between the resource representation vector corresponding to the candidate multimedia resource in the distillation sample and the object representation vector of the target training sample;

[0177] Based on the determined matching degree, the loss weight of any distillation sample is calculated.

[0178] In another implementation, when determining the loss weight of each distillation sample, the one or more instructions can be loaded and executed by the processor:

[0179] For any distillation sample, calculate the matching degree between the object representation vector and the resource representation vector in the third representation vector of any distillation sample;

[0180] Based on the calculated matching degree, the loss weight of any distilled sample is calculated.

[0181] In this embodiment, after determining the first representation vectors of multiple training samples using a first model, the first model is optimized based on these first representation vectors to obtain an optimized first model. Then, a target training sample is selected from these multiple training samples, and its object representation vector is determined using the optimized first model. The resource representation vectors of each candidate multimedia resource in the database are also determined using the optimized first model. Next, following a recall order from high to low matching degree, one or more candidate multimedia resources are recalled from the database based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vector of the target training sample. The recalled candidate multimedia resources are then distilled from the database. Finally, the object feature information in the target training samples and the distilled candidate multimedia resources are used to construct samples. This allows for the construction of more distilled samples using existing training samples without relying on external datasets, thus expanding the original training sample set. Consequently, when the second model is trained using both distilled samples and multiple training samples, it can learn from more samples, avoiding underfitting issues in scenarios with limited data. Furthermore, by distilling the database to obtain distilled samples, the diversity of training samples for the second model can be effectively improved, so that the samples used to train the second model are not limited to one type of samples. This can improve the generalization ability of the second model during the training process, and enable the trained second model to break the information cocoon effect in practical applications.

[0182] It should be noted that, according to one aspect of this application, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the aforementioned... Figure 2 or Figure 3 The methods provided are presented in various alternative embodiments of the model training method based on multimedia resources. Furthermore, it should be understood that the above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, equivalent variations made in accordance with the claims of this application are still within the scope of this application.

Claims

1. A model training method based on multimedia resources, characterized in that, include: After determining the first representation vector of multiple training samples through the first model, the first model is optimized based on the first representation vector of the multiple training samples. A training sample includes an object feature information and a sample multimedia resource. The first representation vector of a training sample includes: an object representation vector obtained based on the corresponding object feature information, and a resource representation vector obtained based on the corresponding sample multimedia resource. Select a target training sample from the plurality of training samples, and determine the object representation vector of the target training sample and the resource representation vector of each candidate multimedia resource in the database through the optimized first model. Based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vectors of the target training sample, one or more candidate multimedia resources are distilled from the database. One or more distillation samples are constructed by using the object feature information in the target training samples and each distilled candidate multimedia resource, and each distillation sample contains a distilled candidate multimedia resource. The second model determines the second representation vector of the plurality of training samples, and the model loss value generated by the second model through the plurality of training samples is determined based on the second representation vector of each training sample. The second model determines the third representation vector of each distillation sample, and the loss value generated by the second model through each distillation sample is calculated based on the third representation vector of each distillation sample. Determine the loss weight of each distillation sample, and use the loss weight of each distillation sample to perform a weighted summation of the loss values ​​generated by the second model through each distillation sample, so as to obtain the model loss value generated by the second model through all distillation samples; The model loss value generated by the second model through the multiple training samples and the model loss value generated by the second model through all distilled samples are summed to obtain the total model loss value of the second model; the model parameters of the second model are updated in the direction of reducing the total model loss value; wherein, the second model includes the first model or the optimized first model.

2. The method as described in claim 1, characterized in that, Based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vectors of the target training samples, one or more candidate multimedia resources are distilled from the database, including: Based on the matching degree between the resource representation vector of each candidate multimedia resource and the object representation vector of the target training sample, one or more candidate multimedia resources are recalled from the database in descending order of matching degree. The recalled candidate multimedia resources are distilled from the database.

3. The method as described in claim 1 or 2, characterized in that, The plurality of training samples includes one or more positive training samples and one or more negative training samples; the multimedia resources in the positive training samples have been subjected to feedback behavior by the object indicated by the corresponding object feature information, while the multimedia resources in the negative training samples have not been subjected to feedback behavior by the object indicated by the corresponding object feature information. The optimization of the first model based on the first representation vector of the plurality of training samples includes: Calculate the first matching degree between the object representation vector in the first representation vector of each positive training sample and the corresponding first resource representation vector; and calculate the second matching degree between the object representation vector in the first representation vector of each negative training sample and the corresponding first resource representation vector. Iterate through each positive training sample in the training sample set, and calculate the loss value generated by the first model through the current positive training sample based on the first matching degree corresponding to the current positive training sample and the second matching degree corresponding to each negative training sample. After obtaining the loss values ​​generated by the first model through each positive training sample, the obtained loss values ​​are integrated to obtain the model loss value generated by the first model through the multiple training samples. The model parameters of the first model are updated in the direction of reducing the model loss value generated by the first model through the multiple training samples.

4. The method as described in claim 3, characterized in that, The step of selecting a target training sample from the plurality of training samples includes: From the plurality of training samples, one or more positive training samples are selected; From the one or more positive training samples, one positive training sample is randomly selected as the target training sample.

5. The method as described in claim 1, characterized in that, Determining the loss weight for each distillation sample includes: For any distillation sample, determine the matching degree between the resource representation vector corresponding to the candidate multimedia resource in the distillation sample and the object representation vector of the target training sample; Based on the determined matching degree, the loss weight of any distilled sample is calculated.

6. The method as described in claim 1, characterized in that, Determining the loss weight for each distillation sample includes: For any distillation sample, calculate the matching degree between the object representation vector and the resource representation vector in the third representation vector of the distillation sample; Based on the calculated matching degree, the loss weight of any distilled sample is calculated.

7. A model training device based on multimedia resources, characterized in that, The device includes: The training unit is used to optimize the first model based on the first representation vectors of the multiple training samples after determining the first representation vectors of the multiple training samples through the first model. A training sample includes an object feature information and a sample multimedia resource. The first representation vector of a training sample includes: an object representation vector obtained based on the corresponding object feature information and a resource representation vector obtained based on the corresponding sample multimedia resource. The training unit is also used to select a target training sample from the plurality of training samples, and to determine the object representation vector of the target training sample and the resource representation vector of each candidate multimedia resource in the database through the optimized first model. The distillation unit is used to distill one or more candidate multimedia resources from the database based on the matching degree between the resource representation vectors of each candidate multimedia resource and the object representation vectors of the target training sample. The training unit is also used to construct one or more distillation samples by using the object feature information in the target training sample and each distilled candidate multimedia resource, and each distillation sample contains a distilled candidate multimedia resource. The training unit is further configured to: determine a second representation vector for each of the plurality of training samples using a second model; determine a model loss value generated by the second model through the plurality of training samples based on the second representation vector of each training sample; determine a third representation vector for each distillation sample using the second model; calculate a loss value generated by the second model through each distillation sample based on the third representation vector of each distillation sample; determine a loss weight for each distillation sample; and perform a weighted summation of the loss values ​​generated by the second model through each distillation sample using the loss weights of each distillation sample to obtain a model loss value generated by the second model through all distillation samples; sum the model loss values ​​generated by the second model through the plurality of training samples and the model loss values ​​generated by the second model through all distillation samples to obtain a total model loss value of the second model; and update the model parameters of the second model in the direction of reducing the total model loss value; wherein the second model includes the first model or the optimized first model.

8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the model training method based on multimedia resources as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more computer programs, which are adapted to be loaded by a processor and executed as described in any one of claims 1 to 6, representing the model training method based on multimedia resources.

10. A computer program product, characterized in that, The computer program product includes a computer program adapted to be loaded by a processor and executed as described in any one of claims 1 to 6.

Images