Keyword and text matching method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to a keyword and text matching method and device, electronic equipment and storage medium. The method comprises: obtaining a keyword for retrieval; extracting feature information of the keyword using a third encoder; the third encoder is iteratively trained based on a loss value of distillation training of a third initial encoder and a first encoder, and a loss value of collaborative training with a second encoder; the first encoder and the second encoder are trained based on a double-tower model framework; obtaining feature information of a plurality of preset candidate texts; determining the matching degree between the keyword and each candidate multimedia resource according to the feature information of the keyword and the feature information of the plurality of candidate texts; and determining a target multimedia resource matched with the keyword from the plurality of candidate multimedia resources according to the matching degree. The present scheme realizes model compression and acceleration, improves prediction efficiency and accuracy, and guarantees online real-time application requirements.

Description

Technical Field

[0001] This disclosure relates to the field of natural language processing technology, and in particular to keyword and text matching methods, apparatus, electronic devices, and storage media. Background Technology

[0002] In the fields of search, recommendation, and advertising, efficiently calculating text similarity has always been an important issue.

[0003] For example, users might enter search information into a multimedia resource provider's website to find multimedia content they are interested in. Multimedia content typically includes a brief text description. To provide users with the most relevant content, the text descriptions of various multimedia resources can be compared with the user's search information to calculate text similarity, thus identifying the multimedia resources that best match the user's search and recommending them to the user.

[0004] Currently, the industry commonly uses the BERT (Bidirectional Encoder Representations from Transformer) model as the encoder to obtain text feature vectors, calculate text similarity, and then recommend related content based on similarity ranking.

[0005] However, the problem in practical applications is that if the model is too large, the training and computation time will be too long, resulting in excessively long time to match the search results in online applications, low prediction efficiency, and inability to meet the needs of real-time applications; if a small model is used, the accuracy will be too low, making it difficult to match multimedia resources that satisfy users. Summary of the Invention

[0006] This disclosure provides a keyword and text matching method, apparatus, electronic device, and storage medium to at least address the problem of unnatural skeletal rotation in body-driven systems in related technologies. The technical solution of this disclosure is as follows:

[0007] According to a first aspect of the present disclosure, a keyword and text matching method is provided, the method comprising:

[0008] Obtain the keywords used for retrieval;

[0009] The third encoder is used to extract the feature information of the keywords; the third encoder is obtained by iterative training based on the loss value of the third initial encoder and the first encoder during distillation training, and the loss value of the third encoder and the second encoder during co-training; the first encoder is used to extract the feature information of the keywords, and the second encoder is used to extract the feature information of the text; the first encoder and the second encoder are trained based on a dual-tower model framework.

[0010] The feature information of multiple preset candidate texts is obtained, and the feature information of the candidate texts is obtained in advance based on the second encoder; the multiple candidate texts are description information of multiple candidate multimedia resources.

[0011] Based on the feature information of the keyword and the feature information of the multiple candidate texts, the matching degree between the keyword and each of the candidate multimedia resources is determined;

[0012] Based on the matching degree, a target multimedia resource matching the keyword is determined from the plurality of candidate multimedia resources.

[0013] In one possible implementation, prior to obtaining the keywords entered online, the method further includes:

[0014] Obtain sample data, which includes keyword sample data, positive text sample data, and negative text sample data. The positive text sample data is text sample data whose matching degree with the keyword sample data is greater than or equal to a first threshold, and the negative text sample data is text sample data whose matching degree is less than the first threshold.

[0015] The keyword sample data is input into the first initial encoder, and the positive text sample data and negative text sample data corresponding to the keyword sample data are input into the second initial encoder respectively.

[0016] Obtain the predicted keyword feature information output by the first initial encoder, and obtain the predicted text feature information output by the second initial encoder and the predicted category of the predicted text feature information; the predicted text feature information includes the first predicted text feature information output based on the positive text sample data and the second predicted text feature information output based on the negative text sample data;

[0017] Based on the predicted keyword feature information, the first predicted text feature information, the second predicted text feature information, and the predicted category, the first loss value for this training is determined;

[0018] Based on the first loss value, the parameters of the first initial encoder and the second initial encoder are modified, and iterative training continues until the first loss value is less than a preset first loss value threshold, thus obtaining the first encoder and the second encoder.

[0019] In one possible implementation, determining the first loss value for this training based on the predicted keyword feature information, the first predicted text feature information, the second predicted text feature information, and the predicted category includes:

[0020] Based on the cosine distance between the predicted keyword feature information and the first predicted text feature information, and the cosine distance between the predicted keyword feature information and the second predicted text feature information, the first sub-loss value for this training is determined;

[0021] Based on the mean square error between the predicted text feature information and the corresponding pre-labeled standard text feature information, the second sub-loss value for this training is determined.

[0022] Based on the predicted category and the pre-labeled standard category label, a third sub-loss value is determined;

[0023] Based on the first sub-loss value, the second sub-loss value, the third sub-loss value, and the preset weights corresponding to the first sub-loss value, the second sub-loss value, and the third sub-loss value, the first loss value for this training is determined.

[0024] In one possible implementation, after obtaining the first encoder and the second encoder, the method further includes:

[0025] The keyword sample data is input into the first encoder and the third initial encoder respectively, and the positive text sample data and negative text sample data corresponding to the keyword sample data are input into the second encoder respectively.

[0026] Obtain the predicted keyword feature information output by the first encoder and the predicted keyword feature information output by the third encoder;

[0027] Obtain the third predicted text feature information output by the second encoder based on the positive text sample data and the fourth predicted text feature information output based on the negative text sample data;

[0028] Based on the predicted keyword feature information output by the first encoder, the predicted keyword feature information output by the third encoder, the third predicted text feature information, and the fourth predicted text feature information, a second loss value is determined.

[0029] Based on the second loss value, the parameters of the third initial encoder are modified, and iterative training of the third initial encoder continues until the second loss value is less than a preset second loss value threshold, thus obtaining the third encoder.

[0030] In one possible implementation, determining the second loss value includes:

[0031] Based on the difference between the predicted keyword feature information output by the first encoder and the predicted keyword feature information output by the third initial encoder, the fourth sub-loss value for this training is determined.

[0032] Based on the cosine distance between the predicted keyword feature information output by the third encoder and the third predicted text feature information, and the cosine distance between the predicted keyword feature information output by the first encoder and the fourth predicted text feature information, the fifth sub-loss value for this training is determined.

[0033] The second loss value is determined based on the fourth sub-loss value, the fifth sub-loss value, and the preset weights corresponding to the fourth and fifth sub-loss values.

[0034] In one possible implementation, after obtaining the third encoder, the method further includes:

[0035] Obtain text sample data and keyword sample data for retraining;

[0036] The text sample data used for retraining is input into the second encoder, and the keyword sample data used for retraining is input into the third encoder to obtain the first retrained text feature information and the retrained keyword feature information, respectively.

[0037] The first retrained text feature information is input into the initial fully connected layer to obtain the second retrained text feature information;

[0038] Based on the second retrained text feature information and the retrained keyword feature information, a third loss value is determined;

[0039] The parameters of the initial fully connected layer are modified based on the third loss value, and the initial fully connected layer is trained again until the third loss value is less than a preset third loss value threshold, thus obtaining the target fully connected layer.

[0040] In one possible implementation, after obtaining the target fully connected layer, the method further includes:

[0041] Based on the second encoder and the target fully connected layer, the feature information of the multiple candidate texts is extracted and stored in the database.

[0042] According to a second aspect of the present disclosure, a keyword and text matching apparatus is provided, the apparatus comprising:

[0043] The keyword retrieval module is configured to retrieve keywords used for retrieval.

[0044] The keyword feature extraction module is configured to extract feature information of the keywords using a third encoder; the third encoder is obtained through iterative training based on the loss value obtained by distilling the third initial encoder and the first encoder, and the loss value obtained by co-training with the second encoder; the first encoder is used to extract feature information of the keywords, and the second encoder is used to extract feature information of the text; the first encoder and the second encoder are trained based on a dual-tower model framework.

[0045] The text feature extraction module is configured to acquire feature information of a plurality of preset candidate texts, the feature information of which is obtained in advance based on the second encoder; the plurality of candidate texts are description information of a plurality of candidate multimedia resources.

[0046] The matching degree determination module is configured to determine the matching degree between the keyword and each of the candidate multimedia resources based on the feature information of the keyword and the feature information of the multiple candidate texts.

[0047] The target multimedia resource determination module is configured to determine, based on the matching degree, the target multimedia resource that matches the keyword from the plurality of candidate multimedia resources.

[0048] In one possible implementation, the device further includes:

[0049] The sample data acquisition module is configured to acquire sample data, which includes keyword sample data, positive text sample data, and negative text sample data. The positive text sample data is text sample data whose matching degree with the keyword sample data is greater than or equal to a first threshold, and the negative text sample data is text sample data whose matching degree is less than the first threshold.

[0050] The first input module is configured to input the keyword sample data into the first initial encoder, and input the positive text sample data and negative text sample data corresponding to the keyword sample data into the second initial encoder respectively.

[0051] The first feature information acquisition module is configured to acquire the predicted keyword feature information output by the first initial encoder, and acquire the predicted text feature information output by the second initial encoder and the predicted category of the predicted text feature information; the predicted text feature information includes the first predicted text feature information output based on the positive text sample data and the second predicted text feature information output based on the negative text sample data.

[0052] The first loss value determination module is configured to determine the first loss value for this training based on the predicted keyword feature information, the first predicted text feature information, the second predicted text feature information, and the predicted category.

[0053] The first training module is configured to modify the parameters of the first initial encoder and the second initial encoder based on the first loss value, and continue iterative training until the first loss value is less than a preset first loss value threshold, thereby obtaining the first encoder and the second encoder.

[0054] In one possible implementation, the first loss value determination module is specifically configured to perform:

[0055] Based on the cosine distance between the predicted keyword feature information and the first predicted text feature information, and the cosine distance between the predicted keyword feature information and the second predicted text feature information, the first sub-loss value for this training is determined;

[0056] Based on the mean square error between the predicted text feature information and the corresponding pre-labeled standard text feature information, the second sub-loss value for this training is determined.

[0057] Based on the predicted category and the pre-labeled standard category label, a third sub-loss value is determined;

[0058] Based on the first sub-loss value, the second sub-loss value, the third sub-loss value, and the preset weights corresponding to the first sub-loss value, the second sub-loss value, and the third sub-loss value, the first loss value for this training is determined.

[0059] In one possible implementation, the device further includes:

[0060] The second input module is configured to input the keyword sample data into the first encoder and the third initial encoder respectively, and input the positive text sample data and negative text sample data corresponding to the keyword sample data into the second encoder respectively.

[0061] The second feature information acquisition module is configured to acquire the predicted keyword feature information output by the first encoder and the predicted keyword feature information output by the third encoder.

[0062] The third feature information acquisition module is configured to acquire the third predicted text feature information output by the second encoder based on the positive text sample data and the fourth predicted text feature information output based on the negative text sample data.

[0063] The second loss value determination module is configured to determine a second loss value based on the predicted keyword feature information output by the first encoder, the predicted keyword feature information output by the third encoder, the third predicted text feature information, and the fourth predicted text feature information.

[0064] The second training module is configured to modify the parameters of the third initial encoder based on the second loss value and continue to iteratively train the third initial encoder until the second loss value is less than a preset second loss value threshold, thereby obtaining the third encoder.

[0065] In one possible implementation, the second loss value determination module is specifically configured to perform:

[0066] Based on the difference between the predicted keyword feature information output by the first encoder and the predicted keyword feature information output by the third initial encoder, the fourth sub-loss value for this training is determined.

[0067] Based on the cosine distance between the predicted keyword feature information output by the third encoder and the third predicted text feature information, and the cosine distance between the predicted keyword feature information output by the first encoder and the fourth predicted text feature information, the fifth sub-loss value for this training is determined.

[0068] The second loss value is determined based on the fourth sub-loss value, the fifth sub-loss value, and the preset weights corresponding to the fourth and fifth sub-loss values.

[0069] In one possible implementation, the device further includes:

[0070] The retraining data acquisition module is configured to acquire text sample data and keyword sample data for retraining.

[0071] The third input module is configured to input the text sample data used for retraining into the second encoder and the keyword sample data used for retraining into the third encoder, so as to obtain the first retrained text feature information and the retrained keyword feature information respectively.

[0072] The fourth feature information acquisition module is configured to input the first retrained text feature information into the initial fully connected layer to obtain the second retrained text feature information;

[0073] The third loss value determination module is configured to determine a third loss value based on the second retrained text feature information and the retrained keyword feature information.

[0074] The third training module is configured to modify the parameters of the initial fully connected layer based on the third loss value, and continue training the initial fully connected layer until the third loss value is less than a preset third loss value threshold, thereby obtaining the target fully connected layer.

[0075] In one possible implementation, the device further includes:

[0076] Based on the second encoder and the target fully connected layer, the feature information of the multiple candidate texts is extracted and stored in the database.

[0077] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the keyword and text matching method as described in the first aspect.

[0078] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of a server, enables the server to perform the keyword and text matching method as described in the first aspect.

[0079] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program / instructions, characterized in that, when the computer program / instructions are executed by a processor, they implement the keyword and text matching method described in the first aspect.

[0080] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0081] In the embodiments of this disclosure, keywords for retrieval are obtained; feature information of the keywords is extracted using a third encoder; the third encoder is iteratively trained based on the loss values ​​obtained from distillation training with a third initial encoder and a first encoder, and the loss values ​​obtained from co-training with a second encoder; the first encoder is used to extract feature information of the keywords, and the second encoder is used to extract feature information of the text; the first encoder and the second encoder are trained based on a dual-tower model framework; feature information of multiple preset candidate texts is obtained, the feature information of the candidate texts being pre-extracted by the second encoder; the multiple candidate texts are description information of multiple candidate multimedia resources; the matching degree between the keywords and each candidate multimedia resource is determined based on the feature information of the keywords and the feature information of the multiple candidate texts; based on the matching degree, the target multimedia resource matching the keywords is determined from the multiple candidate multimedia resources. In the above method, the third initial encoder is trained together based on the loss values ​​from distillation training and the loss values ​​from co-training, so that the third initial encoder has performance comparable to the first encoder, while also achieving model compression and acceleration, thereby improving prediction efficiency and accuracy, ensuring the needs of online real-time applications, and accurately matching multimedia resources that satisfy the user.

[0082] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0083] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0084] Figure 1 This is a flowchart illustrating the steps of a first keyword and text matching method according to an exemplary embodiment;

[0085] Figure 2 This is a flowchart illustrating the steps of a training method for a first encoder and a second encoder according to an exemplary embodiment.

[0086] Figure 3 This is a flowchart illustrating the steps of a training method for a third encoder according to an exemplary embodiment;

[0087] Figure 4 This is a flowchart illustrating the steps of a method for training a target fully connected layer according to an exemplary embodiment;

[0088] Figure 5 This is a schematic diagram illustrating a training process for a first encoder, a second encoder, and a third encoder according to an exemplary embodiment;

[0089] Figure 6 This is a structural block diagram illustrating a keyword and text matching device according to an exemplary embodiment;

[0090] Figure 7 This is a block diagram illustrating an electronic device for keyword and text matching according to an exemplary embodiment. Detailed Implementation

[0091] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0092] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0093] Figure 1 This is a flowchart illustrating the steps of a first keyword and text matching method according to an exemplary embodiment, such as... Figure 1 As shown, the method includes the following steps.

[0094] In step S11, keywords for retrieval are obtained.

[0095] In this embodiment of the invention, users can obtain multimedia resource content of interest by entering query information on the website of a multimedia resource provider. This multimedia resource content may include videos, images, music, etc.

[0096] The entity implementing this solution (such as a multimedia resource provider's server) can obtain the query information entered by the user on the terminal device remotely or locally via wired or wireless network connection, and determine the keywords corresponding to the query information.

[0097] The query information can be obtained by the terminal device through voice input, text input, or gesture input. The aforementioned execution entity obtains the query information from the terminal device via the network, and then extracts keywords from the query information using a keyword extraction tool to obtain the keywords.

[0098] It is understood that a suitable keyword extraction tool can be selected based on the language category and content of the text, and this disclosure does not impose any specific limitations on this. For example, the keyword extraction tool can be KeyBert, a keyword extraction tool based on the BERT model.

[0099] In step S12, the feature information of the keywords is extracted using a third encoder; the third encoder is obtained through iterative training based on the loss value obtained from distillation training with the first encoder and the loss value obtained from co-training with the second encoder; the first encoder is used to extract the feature information of the keywords, and the second encoder is used to extract the feature information of the text; the first encoder and the second encoder are trained based on a dual-tower model framework.

[0100] In this embodiment of the disclosure, based on the dual-tower model framework, a first initial encoder for extracting keyword feature information and a second initial encoder for extracting text feature information are trained to obtain a first encoder and a second encoder.

[0101] Then, the third initial encoder is trained based on the first encoder and the second encoder.

[0102] Specifically, there are two methods: distillation training of the third initial encoder and the first initial encoder, and collaborative training of the third initial encoder and the second initial encoder. The parameters of the third initial encoder are modified based on the loss values ​​of the two training methods, and iterative training is performed to obtain the third encoder.

[0103] In the distillation training, the first encoder serves as the teacher model, and the third initial encoder serves as the student model. The first encoder is a single complex network or a collection of networks, possessing good performance and generalization ability, while the third initial model network is smaller in scale and has limited expressive power. The knowledge learned by the first encoder is used to guide the training of the third initial encoder, enabling it to achieve performance comparable to the first encoder, but with a significantly reduced number of parameters, thus achieving model compression and acceleration.

[0104] In collaborative training, the third initial encoder predicts the keyword feature information corresponding to the input keywords, while the second encoder predicts the positive text feature information corresponding to the input positive sample text and the negative text feature information corresponding to the negative sample text. Further, the loss values ​​between keyword feature information and positive text feature information, and between keyword feature information and negative text feature information, are determined. Based on the principle that keyword feature information is closer to positive text feature information and farther from negative text feature information, the parameters of the third initial encoder are modified, and iterative training is performed to obtain the third encoder.

[0105] This solution is equivalent to obtaining a third encoder through a three-tower model framework and a first encoder and a second encoder through a two-tower model framework. It can achieve model compression and acceleration while ensuring high accuracy, thereby improving prediction and matching efficiency and ensuring online application and rapid iteration.

[0106] In step S13, feature information of a plurality of preset candidate texts is obtained, the feature information of the candidate texts being pre-extracted by the second encoder; the plurality of candidate texts are description information of a plurality of candidate multimedia resources.

[0107] Specifically, feature extraction models can be used to obtain feature information of multiple candidate texts in real time, or the feature information of candidate texts can be pre-stored in a database and retrieved directly from the database.

[0108] To ensure the efficiency of online applications, the feature information of candidate multimedia resources is pre-stored in the database. Retrieving the feature information of multimedia resources directly from the database improves information retrieval efficiency compared to real-time extraction.

[0109] Candidate multimedia resources can be resources that are relevant to the keywords after coarse ranking, or resources with content on any topic. Candidate multimedia resources can include videos, music, images, documents, etc.

[0110] Each candidate multimedia resource has a similar content summary description. This description is used as candidate text. Based on the second encoder, feature information of multiple candidate texts corresponding to multiple multimedia resources is extracted, thereby obtaining the feature information of the candidate multimedia resources.

[0111] The second encoder can be obtained by feature learning training on the BERT model. The BERT model is a language model used for pre-training, possessing bidirectional encoding capabilities and powerful feature extraction capabilities. The second encoder obtained through learning and training can perform better text feature extraction.

[0112] It is understandable that the second encoder can also be any other model capable of encoding text into vector form. The feature information of candidate multimedia resources can also be feature information extracted from resources retrieved in real-time from online media.

[0113] In step S14, the matching degree between the keyword and each of the candidate multimedia resources is determined based on the feature information of the keyword and the feature information of the multiple candidate texts.

[0114] The similarity between the feature information of keywords and the feature information of candidate texts can characterize the matching degree between candidate multimedia resources corresponding to candidate texts and keywords.

[0115] Specifically, methods such as cosine similarity and DSSM (Deep Structured Semantic Models) similarity can be used to calculate the similarity between the feature information of keywords and the feature information of candidate texts.

[0116] In step S15, a target multimedia resource matching the keyword is determined from the plurality of candidate multimedia resources based on the matching degree.

[0117] Matching score indicates the degree of fit between candidate multimedia resources and keywords. The higher the matching score, the better the candidate multimedia resource and keywords are matched. Therefore, from the matching scores of each candidate multimedia resource, the top N candidate multimedia resources can be selected as the target multimedia resources that match the keywords, and the target multimedia resources can be displayed on the front-end page in descending order of matching score.

[0118] In summary, in this embodiment, keywords for retrieval are obtained; feature information of the keywords is extracted using a third encoder; the third encoder is iteratively trained based on the loss values ​​obtained from distillation training with the third initial encoder and the first encoder, and the loss values ​​obtained from co-training with the second encoder; the first encoder is used to extract feature information of the keywords, and the second encoder is used to extract feature information of the text; the first encoder and the second encoder are trained based on a dual-tower model framework; feature information of multiple preset candidate texts is obtained, and the feature information of the candidate texts is obtained based on the pre-extracted feature information of the second encoder; the multiple candidate texts are description information of multiple candidate multimedia resources; the matching degree between the keywords and each candidate multimedia resource is determined based on the feature information of the keywords and the feature information of the multiple candidate texts; based on the matching degree, the target multimedia resource matching the keywords is determined from the multiple candidate multimedia resources. In the above method, the third initial encoder is trained based on the loss values ​​from distillation training and the loss values ​​from co-training, so that the third initial encoder has performance comparable to the first encoder, while also achieving model compression and acceleration, thereby improving prediction efficiency and accuracy, ensuring the needs of online real-time applications, and accurately matching multimedia resources that satisfy users.

[0119] Figure 2 This is a flowchart illustrating the steps of a training method for a first encoder and a second encoder according to an exemplary embodiment. Figure 2 As shown, the method includes the following steps:

[0120] In step S21, sample data is obtained, which includes keyword sample data, positive text sample data, and negative text sample data. The positive text sample data is text sample data whose matching degree with the keyword sample data is greater than or equal to a first threshold, and the negative text sample data is text sample data whose matching degree is less than the first threshold.

[0121] Sample data can be obtained from publicly available online datasets or from self-labeled samples.

[0122] The sample data includes paired keyword samples and text samples. Keyword and text pairs with a matching degree greater than or equal to the first threshold are positive samples, and keyword and text pairs with a matching degree less than the first threshold are negative samples.

[0123] In step S22, the keyword sample data is input into the first initial encoder, and the positive text sample data and negative text sample data corresponding to the keyword sample data are input into the second initial encoder.

[0124] The first and second initial encoders are untrained, raw encoder models.

[0125] The first initial encoder is used to extract feature information from keywords, and the second initial encoder is used to extract feature information from text. Therefore, keyword sample data is input into the first initial encoder, and the positive text sample data and negative text sample data corresponding to the keyword sample data are input into the second initial encoder, respectively.

[0126] In step S23, the predicted keyword feature information output by the first initial encoder is obtained, and the predicted text feature information output by the second initial encoder and the predicted category of the predicted text feature information are obtained; the predicted text feature information includes the first predicted text feature information output based on the positive text sample data and the second predicted text feature information output based on the negative text sample data.

[0127] Obtain the feature information output by the first initial encoder and the second initial encoder, wherein the second initial encoder outputs predicted text feature information and the content category corresponding to the predicted text feature information, i.e., the predicted category.

[0128] For example, if the predicted text feature information is a celebrity named Zhang San, then the predicted category is celebrity.

[0129] Corresponding to the input positive and negative samples, the output predicted text feature information includes the first predicted text feature information and the negative predicted feature information.

[0130] For example, a positive sample is: keyword - pet, text - Corgi running on the grass, and a negative sample is: keyword - pet, text - tiger walking in the mountains. If the keyword input to the first initial encoder is "pet", and the text input to the second initial encoder is "Corgi running on the grass" and "tiger walking in the mountains", then the first predicted text feature information output is "Corgi, grass, running", and the second predicted text feature information output is "tiger, mountain, walking".

[0131] In step S24, a first loss value for this training is determined based on the predicted keyword feature information, the first predicted text feature information, the second predicted text feature information, and the predicted category.

[0132] Specifically, the loss value of the feature information can be determined based on the difference between the predicted keyword feature information and the first predicted text feature information, and the difference between the predicted keyword feature information and the second predicted text feature information. The loss value of the content category can be determined based on the difference between the standard category label and the predicted category of the predicted text feature information.

[0133] Then, based on the loss values ​​of feature information and content category, the first loss value for this training is determined.

[0134] In one possible implementation, step S24 includes the following steps:

[0135] Step S241: Based on the cosine distance between the predicted keyword feature information and the first predicted text feature information, and the cosine distance between the predicted keyword feature information and the second predicted text feature information, determine the first sub-loss value for this training.

[0136] Specifically, the formula for calculating the first sub-loss value is as follows:

[0137]

[0138] Where Loss1' represents the first sub-loss value, q represents the predicted keyword feature information, and d + d represents the first predicted text feature information. - This represents the second predicted text feature information, where τ is a hyperparameter. cos() represents the cosine distance.

[0139] The first sub-loss value is a loss value based on feature information, which makes the distance between keyword feature information and positive text feature information get closer and closer during training, and the distance between keyword feature information and negative text feature information get farther and farther, thereby ensuring that the feature information obtained by the dual-tower model can accurately measure the relevance.

[0140] Step S242: Based on the mean square error between the predicted text feature information and the corresponding pre-labeled standard text feature information, determine the second sub-loss value for this training.

[0141] Specifically, the formula for calculating the second sub-loss value is as follows:

[0142]

[0143] Where Loss2' represents the second sub-loss value, x i This represents pre-annotated standard text feature information, y i This represents the predicted text feature information.

[0144] The second sub-loss value is a loss value based on text feature information, ensuring the accuracy of the text feature information output by the second encoder.

[0145] Step S243: Determine the third sub-loss value based on the predicted category and the pre-labeled standard category label.

[0146] Specifically, the formula for calculating the third sub-loss value is as follows:

[0147]

[0148] Where Loss3' represents the third sub-loss value, y i Indicates standard category labels, Indicates the prediction category.

[0149] The third sub-loss value uses a multi-task learning approach to ensure that the obtained feature information has good discriminative power.

[0150] Step S244: Determine the first loss value for this training based on the first sub-loss value, the second sub-loss value, the third sub-loss value, and the preset weights corresponding to the first sub-loss value, the second sub-loss value, and the third sub-loss value.

[0151] Specifically, the formula for calculating the first loss value is as follows:

[0152] Loss1=w1*Loss1'+w2*Loss2'+w3*Loss3' (4)

[0153] Where Loss1 represents the first loss value, and w1, w2, and w3 represent the preset weights corresponding to the first, second, and third sub-loss values, respectively.

[0154] In step S25, the parameters of the first initial encoder and the second initial encoder are modified based on the first loss value, and iterative training continues until the first loss value is less than a preset first loss value threshold, thus obtaining the first encoder and the second encoder.

[0155] The parameters of the first and second initial encoders are modified in reverse based on the first loss value, so that the first loss value becomes smaller in the next training. This process continues until the first loss value is less than the first loss value threshold after multiple training sessions, at which point the training ends and the first and second encoders are obtained.

[0156] In summary, during the training of the first and second encoders, the first sub-loss value is based on the feature information loss value, which makes the distance between keyword feature information and positive text feature information increasingly closer, and the distance between keyword and negative text feature information increasingly farther, thus ensuring that the feature information obtained by the dual-tower model can accurately measure relevance. The second sub-loss value is based on the text feature information loss value, ensuring the accuracy of the text feature information output by the second encoder. The third sub-loss value uses a multi-task learning approach to ensure that the obtained feature information has good discriminative power. In this way, the feature information output by the first and second encoders has high accuracy and discriminative power.

[0157] Figure 3 This is a flowchart illustrating the steps of a training method for a third encoder according to an exemplary embodiment, as follows: Figure 3 As shown, the method includes the following steps:

[0158] In step S31, the keyword sample data is input into the first encoder and the third initial encoder respectively, and the positive text sample data and negative text sample data corresponding to the keyword sample data are input into the second encoder respectively.

[0159] This embodiment employs a three-tower model to train the third encoder. The first and second encoders are pre-trained; both the first and third initial encoders are used to extract keyword feature information, while the second encoder is used to extract text feature information.

[0160] The first encoder serves as the teacher model, and the third initial encoder serves as the student model. The first encoder is a single complex network or a collection of networks, possessing good performance and generalization ability, while the third initial model network is smaller in scale and has limited expressive power. The knowledge learned by the first encoder is used to guide the training of the third initial encoder, enabling it to achieve performance comparable to the first encoder while significantly reducing the number of parameters, thus achieving model compression and acceleration.

[0161] Specifically, during training, the keyword sample data is input into the first encoder and the third initial encoder, respectively, and the positive text sample data and negative text sample data corresponding to the keyword sample data are input into the second encoder.

[0162] In step S32, the predicted keyword feature information output by the first encoder and the predicted keyword feature information output by the third encoder are obtained.

[0163] The keyword sample data is input into the first encoder and the third initial encoder respectively to obtain the predicted keyword feature information output by the first encoder and the predicted keyword feature information output by the third encoder.

[0164] In step S33, the third predicted text feature information output by the second encoder based on the positive text sample data and the fourth predicted text feature information output based on the negative text sample data are obtained.

[0165] The positive text sample data and negative text sample data corresponding to the keyword sample data are input into the second encoder to obtain the third predicted text feature information output based on the positive text sample data and the fourth predicted text feature information output based on the negative text sample data.

[0166] In step S34, a second loss value is determined based on the predicted keyword feature information output by the first encoder, the predicted keyword feature information output by the third encoder, the third predicted text feature information, and the fourth predicted text feature information.

[0167] Specifically, the loss value between the student model and the teacher model can be determined based on the difference between the predicted keyword feature information output by the first encoder and the predicted keyword feature information output by the third encoder; the loss value in terms of the correlation between keyword feature information and text feature information can be determined based on the difference between the predicted keyword feature information output by the third encoder and the third predicted text feature information, as well as the difference between the predicted keyword feature information output by the third encoder and the fourth predicted text feature information.

[0168] In one possible implementation, step S34 includes:

[0169] Step S341: Based on the difference between the predicted keyword feature information output by the first encoder and the predicted keyword feature information output by the third initial encoder, determine the fourth sub-loss value for this training.

[0170] Specifically, the formula for calculating the fourth sub-loss value is as follows:

[0171] Loss4'=KL(p||q)=E[log(p(x))-log(q(x))] (5)

[0172] Where Loss4' represents the fourth sub-loss value, p represents the predicted keyword feature information output by the first encoder, and q represents the predicted keyword feature information output by the third initial encoder. KL() represents the KL divergence loss function.

[0173] KL divergence can be understood as the difference between the cross-entropy H(p,q) of P and q and the entropy H(p) of the true distribution p. Minimizing the KL divergence can bring H(p,q) closer to H(p), thus making the distributions p and q more similar.

[0174] In this way, the knowledge learned by the first encoder is used to guide the training of the third initial encoder, so that the third initial encoder has performance comparable to that of the first encoder. Since the third encoder network is smaller, model compression and acceleration can be achieved.

[0175] Step S342: Based on the cosine distance between the predicted keyword feature information output by the third encoder and the third predicted text feature information, and the cosine distance between the predicted keyword feature information output by the first encoder and the fourth predicted text feature information, determine the fifth sub-loss value for this training.

[0176] Specifically, the formula for calculating the fifth sub-loss value is as follows:

[0177]

[0178] Where Loss5' represents the fifth sub-loss value, q represents the predicted keyword feature information output by the third encoder, and d + d represents the third predicted text feature information. - This represents the fourth predicted text feature information, where τ is a hyperparameter. cos() represents the cosine distance.

[0179] It can be observed that the fifth sub-loss value uses the same algorithm as the first sub-loss value, but the inputs are different. The fifth sub-loss value is a loss value based on feature information, which makes the distance between keyword feature information and positive text feature information become closer and closer and the distance between keyword feature information and negative text feature information become farther and farther during training. This ensures that the feature information obtained by the dual-tower model composed of the third encoder and the second encoder can accurately measure relevance.

[0180] Step S343: Determine the second loss value based on the fourth sub-loss value, the fifth sub-loss value, and the preset weights corresponding to the fourth sub-loss value and the fifth sub-loss value.

[0181] Specifically, the formula for calculating the second loss value is as follows:

[0182] Loss2=w4*Loss4'+w5*Loss5' (7)

[0183] Here, Loss2 represents the second loss value, and w4 and w5 represent the weights corresponding to the fourth and fifth sub-loss values, respectively.

[0184] In step S35, the parameters of the third initial encoder are modified based on the second loss value, and iterative training of the third initial encoder continues until the second loss value is less than a preset second loss value threshold, thus obtaining the third encoder.

[0185] The parameters of the third initial encoder are modified in reverse based on the first loss value, so that the second loss value becomes smaller in the next training. This process continues until the second loss value is less than the preset second loss value threshold after multiple training sessions. At this point, the training ends and the third encoder is obtained.

[0186] Figure 4 This is a schematic diagram illustrating a training process for a first encoder, a second encoder, and a third encoder according to an exemplary embodiment.

[0187] like Figure 4 As shown, Encoder1, Encoder2, and Encoder3 represent the first, second, and third initial encoders, respectively. `query text` represents keyword sample data, and `doc text` represents text sample data, including positive and negative text samples. `emb1` represents the predicted keyword feature information output by the first initial encoder, `emb2` represents the predicted text feature information output by the second initial encoder, and `emb3` represents the predicted keyword feature information output by the third initial encoder. `Loss1'` represents the first sub-loss value, `Loss2'` represents the second sub-loss value, `Loss3'` represents the third sub-loss value, `Loss4'` represents the fourth sub-loss value, and `Loss5'` represents the fifth sub-loss value. `Loss1` represents the first loss value, and `Loss2` represents the second loss value.

[0188] Among them, the first initial encoder and the second initial encoder are complex models, the third initial encoder is a simple model, the first initial encoder is the teacher model of the third initial encoder, and the third initial encoder is the student model.

[0189] The query text is input into the first initial encoder, outputting emb1. The document text is input into the second initial encoder, outputting emb2. Loss1' is determined based on the cosine distance between emb1 and emb2. Furthermore, a third sub-loss value, Loss3', is determined based on the predicted class of emb1 and the pre-labeled standard class labels. A second sub-loss value, Loss2', is determined based on the difference between the standard keyword feature information labeled the query text and emb1. Finally, the first loss value, Loss1, is determined based on Loss1', Loss2', and Loss3'.

[0190] Based on Loss1, the parameters of the first and second initial encoders are modified, and iterative training is performed to obtain the first and second encoders.

[0191] The doc text is input into the trained second encoder, and the query text is input into the third initial encoder. Loss 5' is determined based on emb2 and emb3. Simultaneously, the query text is input into the trained first encoder, and the query text is input into the third initial encoder. Loss 4' is determined based on the output emb1 and emb3. Then, the second loss value, Loss 2, is determined based on Loss 4' and Loss 5'.

[0192] Based on Loss2, the parameters of the third initial encoder are modified, and the third initial encoder is iteratively trained to obtain the third encoder.

[0193] In summary, during the training of the third encoder, the knowledge learned by the first encoder is used to guide the training of the third initial encoder, enabling it to achieve performance comparable to the first encoder. Due to the smaller network size of the third encoder, model compression and acceleration can be achieved. Furthermore, the fifth sub-loss value, based on feature information, ensures that the distance between keyword features and third text features decreases during training, while the distance between keyword features and fourth text features increases. This guarantees that the feature information obtained by the dual-tower model composed of the third and second encoders can accurately measure relevance.

[0194] Figure 5 This is a flowchart illustrating the steps of a method for training a target fully connected layer according to an exemplary embodiment.

[0195] like Figure 5 As shown, the method includes:

[0196] In step S41, text sample data and keyword sample data for retraining are obtained.

[0197] After training to obtain the third and second encoders, the second encoder is directly used to calculate the predicted text feature information of all candidate multimedia resources and stored in the database as an offline index file. During the initial online testing, the third encoder is used to calculate the predicted keyword feature information of online keywords, and the predicted text feature information in the offline index file is directly used to match the predicted keyword feature information to calculate the matching degree between candidate multimedia resources and keywords.

[0198] However, after the initial online testing, the model still needs to be retrained for practical applications. In traditional methods, each retraining requires modifications to the second encoder, which is a large model with a complex network structure and a long iteration cycle. This solution adds a fully connected layer (MLP, Fully Connected Neural Network) after the second encoder, enabling rapid updates and iterations.

[0199] Specifically, in practical applications, text sample data and keyword sample data for retraining are obtained, and these sample data are used for subsequent training of the fully connected layer.

[0200] In step S42, the text sample data used for retraining is input into the second encoder, and the keyword sample data used for retraining is input into the third encoder to obtain the first retraining text feature information and the retraining keyword feature information, respectively.

[0201] Specifically, the second encoder outputs the first retrained text feature information, and the third encoder outputs the retrained keyword feature information.

[0202] In step S43, the first retrained text feature information is input into the initial fully connected layer to obtain the second retrained text feature information.

[0203] The initial fully connected layer is fully connected to all neurons of the second encoder. The initial fully connected layer can integrate local information with class discrimination in the convolutional or pooling layers to improve the overall network performance.

[0204] The first training text feature information is processed by the initial fully connected layer to obtain the second training text feature information.

[0205] In step S44, a third loss value is determined based on the second retrained text feature information and the retrained keyword feature information.

[0206] Similarly, the third loss value can be determined based on the calculation method of the first sub-loss value.

[0207] In step S45, the parameters of the initial fully connected layer are modified based on the third loss value, and the initial fully connected layer is trained again until the third loss value is less than a preset third loss value threshold, thus obtaining the target fully connected layer.

[0208] When updating the model, the parameters of the initial fully connected layer can be modified based on the third loss value, without modifying the parameters of the second encoder or the multimedia resource database index corresponding to the second encoder, thus improving the iteration efficiency of retraining.

[0209] Updates to the initial fully connected layer are almost at the microsecond level and will not affect users' online experience.

[0210] In step S46, feature information of the multiple candidate texts is extracted based on the second encoder and the target fully connected layer, and the feature information of the multiple candidate texts is stored in the database.

[0211] After obtaining the target fully connected layer, the second encoder and the target fully connected layer are used to extract the feature information of the candidate text, and the feature information is stored in the database for later use in step S13.

[0212] Reference Figure 4 During retraining, a fully connected Multi-Level Processing (MLP) layer is added after the second encoder to enable rapid update iterations. Specifically, `emb2` is input into the MLP to predict `emb2_new`. Based on the cosine distance between `emb2_new` and `emb3`, the third loss value, `Loss3`, is determined. When updating the model, the parameters of the MLP can be modified based on `Loss3`, without needing to modify the parameters of the second encoder or the corresponding multimedia resource database index, thus improving the iterative efficiency of retraining.

[0213] In one possible implementation, obtaining feature information of a preset plurality of candidate texts includes:

[0214] Retrieve feature information of multiple preset candidate texts from the database.

[0215] To ensure the efficiency of online applications, the feature information of candidate multimedia resources is pre-stored in the database. Retrieving the feature information of multimedia resources directly from the database improves information retrieval efficiency compared to real-time extraction.

[0216] In summary, this embodiment connects the initial fully connected layer to the second encoder. During online retraining, a third loss value is determined based on the second retrained text feature information and the retrained keyword feature information. The parameters of the initial fully connected layer are then modified based on the third loss value, and the initial fully connected layer is trained again to obtain the target fully connected layer. This allows for rapid update iterations using only the initial fully connected layer, without needing to update the second encoder, which has a complex network structure, during retraining.

[0217] Figure 6 This is a structural block diagram illustrating a keyword and text matching device according to an exemplary embodiment. Figure 6 As shown, the keyword and text matching device 50 includes:

[0218] Keyword acquisition module 51 is configured to acquire keywords for retrieval;

[0219] The keyword feature extraction module 52 is configured to extract feature information of the keywords using a third encoder; the third encoder is obtained through iterative training based on the loss value obtained by distilling training with the third initial encoder and the first encoder, and the loss value obtained by co-training with the second encoder; the first encoder is used to extract feature information of the keywords, and the second encoder is used to extract feature information of the text; the first encoder and the second encoder are trained based on a dual-tower model framework.

[0220] The text feature extraction module 53 is configured to acquire feature information of a plurality of preset candidate texts, the feature information of which is obtained in advance based on the second encoder; the plurality of candidate texts are description information of a plurality of candidate multimedia resources.

[0221] The matching degree determination module 54 is configured to determine the matching degree between the keyword and each of the candidate multimedia resources based on the feature information of the keyword and the feature information of the plurality of candidate texts.

[0222] The target multimedia resource determination module 55 is configured to determine, based on the matching degree, the target multimedia resource that matches the keyword from the plurality of candidate multimedia resources.

[0223] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0224] Figure 7 This is a block diagram illustrating an electronic device for keyword and text matching according to an exemplary embodiment. Its internal structure diagram can be shown as follows: Figure 7As shown, the server or electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the server or electronic device is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a keyword and text matching method.

[0225] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the server or electronic device to which the present disclosure is applied. A specific server or electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0226] In an exemplary embodiment, a server or electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the keyword and text matching method as described in the embodiments of this disclosure.

[0227] In an exemplary embodiment, a computer-readable storage medium is also provided, wherein instructions in the computer-readable storage medium, when executed by a processor of a server or electronic device, enable the server or electronic device to perform the keyword and text matching method of the embodiments of this disclosure. The computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0228] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the keyword and text matching method in the embodiments of this disclosure.

[0229] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0230] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0231] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A keyword and text matching method, characterized in that, The method includes: Obtain the keywords used for retrieval; The third encoder is used to extract the feature information of the keywords; the third encoder is obtained by iterative training based on the loss value of the third initial encoder and the first encoder during distillation training, and the loss value of the third encoder and the second encoder during co-training; the first encoder is used to extract the feature information of the keywords, and the second encoder is used to extract the feature information of the text; the first encoder and the second encoder are trained based on a dual-tower model framework. The feature information of multiple preset candidate texts is obtained, and the feature information of the candidate texts is obtained in advance based on the second encoder; the multiple candidate texts are description information of multiple candidate multimedia resources. Based on the feature information of the keyword and the feature information of the multiple candidate texts, the matching degree between the keyword and each of the candidate multimedia resources is determined; Based on the matching degree, a target multimedia resource matching the keyword is determined from the plurality of candidate multimedia resources; The third encoder is trained using the following method: The keyword sample data is input into the first encoder and the third initial encoder respectively, and the positive text sample data and negative text sample data corresponding to the keyword sample data are input into the second encoder respectively. Obtain the predicted keyword feature information output by the first encoder and the predicted keyword feature information output by the third encoder; Obtain the third predicted text feature information output by the second encoder based on the positive text sample data and the fourth predicted text feature information output based on the negative text sample data; Based on the predicted keyword feature information output by the first encoder, the predicted keyword feature information output by the third encoder, the third predicted text feature information, and the fourth predicted text feature information, a second loss value is determined. Based on the second loss value, the parameters of the third initial encoder are modified, and iterative training of the third initial encoder continues until the second loss value is less than a preset second loss value threshold, thus obtaining the third encoder.

2. The method according to claim 1, characterized in that, Before obtaining keywords entered online, the process also includes: Obtain sample data, which includes keyword sample data, positive text sample data, and negative text sample data. The positive text sample data is text sample data whose matching degree with the keyword sample data is greater than or equal to a first threshold, and the negative text sample data is text sample data whose matching degree is less than the first threshold. The keyword sample data is input into the first initial encoder, and the positive text sample data and negative text sample data corresponding to the keyword sample data are input into the second initial encoder respectively. Obtain the predicted keyword feature information output by the first initial encoder, and obtain the predicted text feature information output by the second initial encoder and the predicted category of the predicted text feature information; the predicted text feature information includes the first predicted text feature information output based on the positive text sample data and the second predicted text feature information output based on the negative text sample data; Based on the predicted keyword feature information, the first predicted text feature information, the second predicted text feature information, and the predicted category, the first loss value for this training is determined; Based on the first loss value, the parameters of the first initial encoder and the second initial encoder are modified, and iterative training continues until the first loss value is less than a preset first loss value threshold, thus obtaining the first encoder and the second encoder.

3. The method according to claim 2, characterized in that, The step of determining the first loss value for this training based on the predicted keyword feature information, the first predicted text feature information, the second predicted text feature information, and the predicted category includes: Based on the cosine distance between the predicted keyword feature information and the first predicted text feature information, and the cosine distance between the predicted keyword feature information and the second predicted text feature information, the first sub-loss value for this training is determined; Based on the mean square error between the predicted text feature information and the corresponding pre-labeled standard text feature information, the second sub-loss value for this training is determined. Based on the predicted category and the pre-labeled standard category label, a third sub-loss value is determined; Based on the first sub-loss value, the second sub-loss value, the third sub-loss value, and the preset weights corresponding to the first sub-loss value, the second sub-loss value, and the third sub-loss value, the first loss value for this training is determined.

4. The method according to claim 1, characterized in that, Determining the second loss value includes: Based on the difference between the predicted keyword feature information output by the first encoder and the predicted keyword feature information output by the third initial encoder, the fourth sub-loss value for this training is determined. Based on the cosine distance between the predicted keyword feature information output by the third initial encoder and the third predicted text feature information, and the cosine distance between the predicted keyword feature information output by the first encoder and the fourth predicted text feature information, the fifth sub-loss value for this training is determined. The second loss value is determined based on the fourth sub-loss value, the fifth sub-loss value, and the preset weights corresponding to the fourth and fifth sub-loss values.

5. The method according to claim 1, characterized in that, After obtaining the third encoder, it also includes: Obtain text sample data and keyword sample data for retraining; The text sample data used for retraining is input into the second encoder, and the keyword sample data used for retraining is input into the third encoder to obtain the first retrained text feature information and the retrained keyword feature information, respectively. The first retrained text feature information is input into the initial fully connected layer to obtain the second retrained text feature information; Based on the second retrained text feature information and the retrained keyword feature information, a third loss value is determined; The parameters of the initial fully connected layer are modified based on the third loss value, and the initial fully connected layer is trained again until the third loss value is less than a preset third loss value threshold, thus obtaining the target fully connected layer.

6. The method according to claim 5, characterized in that, After obtaining the target fully connected layer, the following is also included: Based on the second encoder and the target fully connected layer, feature information of the multiple candidate texts is extracted and stored in the database; The acquisition of feature information from multiple preset candidate texts includes: Retrieve feature information of multiple preset candidate texts from the database.

7. A keyword and text matching device, characterized in that, The device includes: The keyword retrieval module is configured to retrieve keywords used for retrieval. The keyword feature extraction module is configured to extract feature information of the keywords using a third encoder; the third encoder is obtained through iterative training based on the loss value obtained by distilling the third initial encoder and the first encoder, and the loss value obtained by co-training with the second encoder; the first encoder is used to extract feature information of the keywords, and the second encoder is used to extract feature information of the text; the first encoder and the second encoder are trained based on a dual-tower model framework. The text feature extraction module is configured to acquire feature information of a plurality of preset candidate texts, the feature information of which is obtained in advance based on the second encoder; the plurality of candidate texts are description information of a plurality of candidate multimedia resources. The matching degree determination module is configured to determine the matching degree between the keyword and each of the candidate multimedia resources based on the feature information of the keyword and the feature information of the multiple candidate texts. The target multimedia resource determination module is configured to determine, based on the matching degree, the target multimedia resource that matches the keyword from the plurality of candidate multimedia resources; The device further includes: The second input module is configured to input the keyword sample data into the first encoder and the third encoder respectively. The initial encoder inputs the positive text sample data and negative text sample data corresponding to the keyword sample data respectively. Enter the second encoder; The second feature information acquisition module is configured to acquire the predicted keyword features output by the first encoder. Information and the predicted keyword feature information output by the third encoder; The third feature information acquisition module is configured to acquire the information obtained by the second encoder based on the number of text samples. Based on the output of the third predicted text feature information and the output of the fourth predicted text feature information based on the negative text sample data; The second loss value determination module is configured to perform prediction of keyword feature information based on the output of the first encoder. Information, the predicted keyword feature information output by the third encoder, the third predicted text feature information, and the fourth Predict textual features and determine the second loss value; The second training module is configured to modify the parameters of the third initial encoder based on the second loss value. The third initial encoder is then iteratively trained until the second loss value is less than the preset second loss. The threshold is used to obtain the third encoder.

8. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the keyword and text matching method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the server, the server is able to perform the keyword and text matching method as described in any one of claims 1 to 7.

Images