Recall model training method and recommendation method, device, and electronic equipment

Abstract

The recall model training method, recommendation method, device, and electronic device provided in this invention are applied in the field of information technology. They involve: acquiring the feature similarity between at least one search term sample and multiple first recall results corresponding to each search term sample; acquiring first selection information for each first recall result within a first preset historical time range; selecting one or more first negative samples from multiple first recall results based on the first selection information; inputting at least one search term sample and one or more first negative samples into the recall model to be trained to calculate the corresponding calculation result; training based on the calculation result until a preset iteration stopping condition is reached or the current loss is less than a preset loss threshold, thus obtaining a trained recall model. This allows for the selection of training samples without manual intervention, improving the efficiency of training sample selection and consequently improving the training efficiency of the model.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to recall model training methods and recommendation methods, devices, and electronic devices. Background Technology

[0002] Currently, in the process of users using some applications, information is often recalled and recommended based on user requests. For example, when a user searches for videos on a certain topic, a recall model can be used to recall relevant information based on that topic, and then all or part of the recalled information can be selected to recommend to the user.

[0003] However, training recall models often requires a large number of training samples. Currently, training samples are typically selected manually, which is not only inefficient but also often negatively impacts the efficiency of model training. Summary of the Invention

[0004] The purpose of this invention is to provide a recall model training method, recommendation method, apparatus, and electronic device to improve the training efficiency of the recall model. The specific technical solution is as follows:

[0005] In a first aspect of this invention, a recall model training method is provided, the method comprising:

[0006] Obtain the feature similarity between at least one search term sample and multiple first recall results corresponding to each search term sample, wherein each first recall result corresponds to a feature similarity, and the feature similarity represents the similarity between the first recall result and the at least one search term sample;

[0007] Obtain first selection information for each of the first recall results within a first preset historical time range, and select one or more as first negative samples from the plurality of first recall results based on the first selection information;

[0008] The at least one search term sample and the one or more first negative samples are input into the recall model to be trained, and the corresponding calculation result is calculated according to the recall model to be trained. The current loss of the recall model to be trained is calculated according to the calculation result. The parameters of the recall model to be trained are adjusted according to the current loss. The process is repeated until the preset iteration stopping condition is reached or the current loss is less than the preset loss threshold, and the trained recall model is obtained.

[0009] In one possible implementation, obtaining first selection information for each of the first recall results within a first preset historical time range, and selecting one or more as first negative samples from the plurality of first recall results based on the first selection information, includes:

[0010] Identify the first recall results whose feature similarity is not within a first preset range among the plurality of first recall results, and obtain one or more second recall results;

[0011] Obtain first selection information for each of the first recall results within a first preset historical time range, and select one or more as first negative samples from the one or more second recall results based on the first selection information.

[0012] In one possible implementation, the step of obtaining selection information for each of the first recall results within a first preset historical time range, and selecting one or more as first negative samples from the plurality of first recall results based on the first selection information, includes:

[0013] Obtain selection information for each of the first recall results within a first preset historical time range, wherein the selection information for each of the first recall results represents the information of the recall results selected by the user in each of the first recall results;

[0014] Based on the first selection information, one or more samples are selected as first negative samples from the plurality of first recall results according to a first preset probability.

[0015] In one possible implementation, the first selection information includes second selection information corresponding to multiple preset time ranges, and the step of selecting one or more as first negative samples from the multiple first recall results according to the selection information and a first preset probability includes:

[0016] Obtain the selection probability corresponding to each of the plurality of preset time ranges, wherein the selection probability corresponding to the preset time range that is closer to the current time is higher.

[0017] Based on the second selection information corresponding to each of the multiple preset time ranges and the selection probability corresponding to each of the multiple preset time ranges, one or more are selected as the first negative samples from the multiple first recall results.

[0018] In one possible implementation, the step of selecting one or more as first negative samples from the plurality of first recall results according to the first selection information and a first preset probability includes:

[0019] A third recall result is obtained by identifying the recall results other than the first negative sample among the multiple first recall results;

[0020] The recall results of the third recall results are identified as having a preset target type, and one or more of the identified recall results are selected as the first negative samples according to a second preset probability, wherein the second preset probability is greater than the first preset probability.

[0021] In a second aspect of the invention, a recommended method is also provided, the method comprising:

[0022] Get the search terms to be searched;

[0023] The search term is input into a pre-trained recall model so that the pre-trained recall model can identify one or more recall results that match the search term and the matching degree corresponding to each recall result. The pre-trained recall model is a network model trained by any of the recall model training methods described above.

[0024] Based on the matching degree corresponding to each recall result, the one or more recall results are sorted in descending order, and the top N results are selected for feedback.

[0025] In a third aspect of the invention, a recall model training apparatus is also provided, the apparatus comprising:

[0026] The similarity acquisition module is used to acquire the feature similarity between at least one search term sample and multiple first recall results corresponding to each search term sample, wherein each first recall result corresponds to a feature similarity, and the feature similarity represents the similarity between the first recall result and the at least one search term sample;

[0027] The sample selection module is used to obtain first selection information for each of the first recall results within a first preset historical time range, and select one or more as first negative samples from the plurality of first recall results according to the first selection information.

[0028] The model training module is used to input the at least one search term sample and the one or more first negative samples into the recall model to be trained, calculate the corresponding calculation result based on the recall model to be trained, calculate the current loss of the recall model to be trained based on the calculation result, adjust the parameters of the recall model to be trained based on the current loss, and return to the step of inputting the at least one search term sample and the one or more first negative samples into the recall model to be trained to continue training until a preset iteration stopping condition is reached or the current loss is less than a preset loss threshold, thereby obtaining a trained recall model.

[0029] In one possible implementation, the sample selection module includes:

[0030] The recall result identification submodule is used to identify the first recall result whose corresponding feature similarity is not within a first preset range among the plurality of first recall results, and obtain one or more second recall results;

[0031] The first negative sample acquisition submodule is used to acquire first selection information for each of the first recall results within a first preset historical time range, and select one or more as the first negative sample from the one or more second recall results according to the first selection information.

[0032] In one possible implementation, the sample selection module includes:

[0033] The selection information acquisition submodule is used to acquire selection information for each of the first recall results within a first preset historical time range, wherein the selection information for each of the first recall results represents the information of the recall results selected by the user in each of the first recall results;

[0034] The first negative sample selection submodule is used to select one or more as first negative samples from the plurality of first recall results according to the first selection information and a first preset probability.

[0035] In one possible implementation, the first negative sample selection submodule includes:

[0036] A selection probability acquisition unit is used to acquire the selection probability corresponding to each of the plurality of preset time ranges, wherein the selection probability corresponding to the preset time range that is closer to the current time is higher.

[0037] The probability selection unit is used to select one or more as first negative samples from the plurality of first recall results based on the second selection information corresponding to each of the plurality of preset time ranges and the selection probability corresponding to each of the plurality of preset time ranges.

[0038] In one possible implementation, the first negative sample selection submodule includes:

[0039] The third recall result identification unit is used to identify the recall results other than the first negative sample among the plurality of first recall results, and obtain the third recall result;

[0040] The probability identification unit is used to identify the recall results of the preset target type in the third recall results, and select one or more from the identified recall results as the first negative sample according to the second preset probability, wherein the second preset probability is greater than the first preset probability.

[0041] In a fourth aspect of the invention, a recommended apparatus is also provided, the apparatus comprising:

[0042] The search term acquisition module is used to acquire the search terms to be searched.

[0043] The matching degree recognition module is used to input the search term into the pre-trained recall model so that the pre-trained recall model can identify one or more recall results that match the search term and the matching degree corresponding to each recall result. The pre-trained recall model is a network model trained by any of the recall model training methods described above.

[0044] The result feedback module is used to sort the one or more recall results in descending order based on the matching degree corresponding to each recall result, and select the top N results for feedback.

[0045] In another aspect of the present invention, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0046] Memory, used to store computer programs;

[0047] When a processor executes a program stored in memory, it implements any of the recall model training methods described above.

[0048] In another aspect of the present invention, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0049] Memory, used to store computer programs;

[0050] When a processor executes a program stored in memory, it implements any of the recommended methods described above.

[0051] In another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the recall model training methods described above.

[0052] In another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program, when executed by a processor, implements any of the recommended methods described above.

[0053] In another aspect of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the recall model training methods described above.

[0054] In another aspect of the invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the recommended methods described above.

[0055] The recall model training method, recommendation method, apparatus, and electronic device provided in this invention involve: acquiring feature similarity between at least one search term sample and multiple first recall results corresponding to each search term sample, wherein each first recall result corresponds to a feature similarity, and the feature similarity represents the similarity between the first recall result and the at least one search term sample; acquiring first selection information for each first recall result within a first preset historical time range, and selecting one or more first negative samples from the multiple first recall results according to the first selection information; and inputting the at least one search term sample and the one or more first negative samples into the training module. The system trains a recall model and calculates the corresponding results based on the model. It then calculates the current loss of the recall model based on the results and adjusts the parameters of the model based on the current loss. Finally, it returns to the step of inputting at least one search term sample and one or more first negative samples into the recall model to continue training until a preset iteration stopping condition is met or the current loss is less than a preset loss threshold, resulting in a well-trained recall model. This allows for the selection of training samples without manual intervention, improving the efficiency of training sample selection and thus enhancing the model's training efficiency. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0057] Figure 1 This is a flowchart illustrating a recall model training method in an embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of a process for selecting the first negative sample in an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of another process for selecting the first negative sample in an embodiment of the present invention;

[0060] Figure 4 This is another schematic diagram of the process for selecting the first negative sample in an embodiment of the present invention;

[0061] Figure 5 This is another flowchart illustrating the recall model training method in an embodiment of the present invention;

[0062] Figure 6 This is a flowchart illustrating a recommended method in an embodiment of the present invention;

[0063] Figure 7 This is a schematic diagram of a recall model training device in an embodiment of the present invention;

[0064] Figure 8 This is a schematic diagram of a recommended device in an embodiment of the present invention;

[0065] Figure 9 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention;

[0066] Figure 10 This is a schematic diagram of the structure of another electronic device in an embodiment of the present invention. Detailed Implementation

[0067] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0068] In a first aspect of this invention, a recall model training method is provided, see [link to relevant documentation]. Figure 1 The above methods include:

[0069] Step S11: Obtain the feature similarity of at least one search term sample and multiple first recall results corresponding to each search term sample.

[0070] In this application, the method is applied to the training of a recall model. Specifically, the recall results can be related to the target recalled by the recall model, and can be videos, images, music, etc. For example, when the recall model to be trained is used for video recall and recommendation, the multiple first recall results corresponding to each search term sample can be videos. Each first recall result corresponds to a feature similarity, which represents the similarity between the first recall result and at least one search term sample. For example, after obtaining the search term, the similarity between the search term and the name, characters, and other information of each video in a preset video library is calculated, and then the N videos with the highest similarity, or those with similarity greater than a preset threshold, are taken as the first recall results.

[0071] The method described in this application is applied to a smart terminal, which can be used to train a recall model. Specifically, the smart terminal can be a computer or a server, etc.

[0072] Step S12: Obtain first selection information for each first recall result within a first preset historical time range, and select one or more as first negative samples from multiple first recall results based on the first selection information.

[0073] In this embodiment, the first selection information for each first recall result within a first preset historical time range refers to which of the multiple first recall results the user selected within that first preset historical time range. The first preset historical time range can be set according to actual conditions, such as 7 days, 14 days, etc. For example, the first selection information for each first recall result within the first preset historical time range might be: which of the multiple first recall results the user selected within 7 days. When selecting one or more as first negative samples from multiple first recall results based on the first selection information, one or more of the user's selections can be selected from the multiple first recall results and used as first negative samples. For example, if the first selection information is: the user selected a, b, and c within 7 days, then when selecting one or more as first negative samples from multiple first recall results based on the first selection information, if the multiple first recall results are a, e, and f, then since a was selected by the user within the first preset historical time range among the multiple first recall results, a will be used as the first negative sample.

[0074] Step S13: Input at least one search term sample and one or more first negative samples into the recall model to be trained, and calculate the corresponding calculation result according to the recall model to be trained. Calculate the current loss of the recall model to be trained according to the calculation result, adjust the parameters of the recall model to be trained according to the current loss, and return to the step of inputting at least one search term sample and one or more first negative samples into the recall model to be trained to continue training until the preset iteration stopping condition is reached or the current loss is less than the preset loss threshold, and the trained recall model is obtained.

[0075] In practical use, the calculation result obtained from the recall model to be trained can be the similarity between each sample in the input samples and the search term. The current loss is then calculated based on this result, which can be done in various ways, such as using the 0-1 loss function, the Log loss function, or the Hinge loss function. When adjusting the parameters of the recall model to be trained based on the current loss, methods such as backpropagation optimization can be used to adjust the model parameters. Preset iteration stopping conditions and preset loss thresholds can be set according to actual conditions. For example, the preset iteration stopping condition is when the number of iterations reaches a preset number, such as 100,000, and the preset loss threshold is 0.003.

[0076] As can be seen, the method of this application embodiment can obtain the feature similarity of at least one search term sample and multiple first recall results corresponding to each search term sample, and obtain and select one or more first negative samples from the multiple first recall results according to the first selection information, thereby inputting the at least one search term sample and the one or more first negative samples into the recall model to be trained for training, thereby obtaining a trained recall model. Thus, the selection of training samples can be carried out without manual intervention, improving the efficiency of training sample selection and thus improving the training efficiency of the model.

[0077] In one possible implementation, see Figure 2 Step S12 obtains first selection information for each first recall result within a first preset historical time range, and selects one or more as first negative samples from multiple first recall results based on the first selection information, including:

[0078] Step S121: Identify the first recall results whose feature similarity is not within a first preset range among multiple first recall results, and obtain one or more second recall results;

[0079] Step S122: Obtain first selection information for each first recall result within a first preset historical time range, and select one or more from one or more second recall results as first negative samples based on the first selection information.

[0080] After obtaining the feature similarity between at least one search term sample and multiple first recall results corresponding to each search term sample, the obtained similarity is typically between 0 and 1. Then, based on a first preset range, the first recall results whose corresponding feature similarity is not within the first preset range are identified, resulting in one or more second recall results. The first preset range can be set according to actual conditions; for example, it can be 0.2–0.8, 0.3–0.6, etc. In one example, after obtaining the feature similarity between at least one search term sample and multiple first recall results corresponding to each search term sample, the resulting multiple first recall results are A, B, C, D, and E, with corresponding feature similarities of 0.1, 0.3, 0.5, 0.6, and 0.9, respectively. If the first preset range is 0.3–0.6, the identified second recall results are B, C, and D.

[0081] Specifically, by acquiring and selecting one or more second recall results as first negative samples based on the first selection information, the first recall results whose corresponding feature similarity is not within a first preset range can be recalled. Therefore, when the first preset range limits the corresponding feature similarity in the first recall results to a relatively large value, by identifying first recall results whose corresponding feature similarity is not within the first preset range, the corresponding recall results with lower similarity can be selected. Then, based on the first selection information, one or more of these can be selected as first negative samples. This ensures that the final selected one or more first negative samples are recall results with low similarity to the search term samples, thus guaranteeing the effectiveness of the negative samples and preventing the situation where recall results with high similarity, which should be positive samples, are used as negative samples.

[0082] As can be seen, by using the method of this application embodiment, the feature similarity corresponding to multiple first recall results can be identified, and one or more of them can be selected as first negative samples. This ensures that the first negative sample is finally selected as the recall result with low similarity to the search term sample, thereby ensuring the effectiveness of the negative sample.

[0083] In one possible implementation, see Figure 3 Step S12 obtains selection information for each first recall result within a first preset historical time range, and selects one or more as first negative samples from multiple first recall results based on the first selection information, including:

[0084] Step S123: Obtain selection information for each first recall result within a first preset historical time range, wherein the selection information for each first recall result represents the information of the recall result selected by the user in each first recall result;

[0085] Step S124: Based on the first selection information, select one or more as the first negative samples from multiple first recall results according to the first preset probability.

[0086] In this embodiment, different preset probabilities can be set for different preset time ranges. Specifically, the closer the first preset time range is to the current time, the higher the corresponding first preset probability can be. For example, when the first preset time range is 14 days, the corresponding first preset probability is 5%, and when the first preset time range is 7 days, the corresponding first preset probability is 15%.

[0087] In one possible implementation, the first selection information includes second selection information corresponding to each of multiple preset time ranges. Based on the selection information, one or more samples are selected as first negative samples from multiple first recall results according to a first preset probability. This includes: obtaining the selection probability corresponding to each of the multiple preset time ranges, wherein the selection probability corresponding to a preset time range closer to the current time is higher; and selecting one or more samples as first negative samples from multiple first recall results based on the second selection information corresponding to each of the multiple preset time ranges and the selection probability corresponding to each of the multiple preset time ranges.

[0088] By setting the selection probability of each of the multiple preset time ranges, the selection probability of the preset time range closer to the current time is higher. This can make the selected first negative sample more of the negative samples that are closer to the current time, thereby improving the timeliness of the selected negative samples and ensuring that the selected negative samples better meet the requirements of the current time.

[0089] As can be seen, the method of this application embodiment can select one or more as first negative samples from multiple first recall results according to a first preset probability. When the first preset time range is closer to the current time, the corresponding first preset probability can be greater, which can make more negative samples closer to the current time among the selected first negative samples, thereby improving the timeliness of the selected negative samples and ensuring that the selected negative samples better meet the requirements of the current time.

[0090] In one possible implementation, see Figure 4 Based on the first selection information, one or more samples are selected as the first negative samples from multiple first recall results according to a first preset probability, including:

[0091] Step S41: Identify the recall results other than the first negative sample among multiple first recall results to obtain the third recall result;

[0092] Step S42: Identify the recall results of the preset target type in the third recall results, and select one or more from the identified recall results as the first negative samples according to the second preset probability, wherein the second preset probability is greater than the first preset probability.

[0093] In this embodiment, the recall results for the preset target type can be popular samples. Specifically, popular samples can be pre-collected to form a popular sample set. When identifying the recall results for the preset target type in the third recall results, the third recall results can be compared with each sample in the popular sample set to determine which recall results in the third recall results are popular samples. Since the second preset probability is greater than the first preset probability (for example, when the first preset probability is 5%, the second preset probability is 15%), after identifying the recall results other than the first negative samples in multiple first recall results and obtaining the third recall results, when selecting the first negative samples, the samples that were not selected as first negative samples can be reselected for negative samples with a higher probability targeting the popular samples. This can increase the number of popular samples among the selected negative samples, thereby meeting the current popularity requirements and improving the practicality of the negative samples.

[0094] As can be seen, the method of this application embodiment can identify recall results other than the first negative sample among multiple first recall results, obtain a third recall result, identify the recall results of the preset target type in the third recall result, and select one or more as the first negative sample from the identified recall results according to the second preset probability. This can increase the number of popular samples in the selected negative samples, thereby meeting the real-time requirements, improving the practicality of the negative samples and the practicality of the finally trained model.

[0095] To illustrate the methods of the embodiments of this application, the following description is provided in conjunction with specific examples. (See attached examples.) Figure 5 ,include:

[0096] 1. Extract simple samples and determine whether the recall score corresponding to the simple sample is greater than alpha and less than beta. Here, the recall score represents the similarity between the sample and the preset recommended words, and alpha and beta are two preset values, with alpha being less than beta. If so, select them as negative samples. For example, if the recall score of the item recalled by the recall model is between [0, 1], randomly select items whose recall score ranks between the top 30% and top 60% as negative samples, that is, the score is between [alpha, beta], where alpha = score * 0.3, beta = score * 0.6, and score is the maximum recall score;

[0097] 2. Select hard sample 1, and then recall items that users have clicked in the past 14 days. For example, if the recalled items have been clicked in the past 14 days, they can be selected as negative samples according to a certain probability theta.

[0098] 3. If the recalled item has not been clicked in the past 14 days, it will be selected as hard sample 2. Then, it will be determined whether there are popular samples in the recalled items that have not been clicked this time. For example, if there are popular samples in the recalled items that have not been clicked this time, the probability of it being selected as a negative sample can be increased.

[0099] As can be seen, the method of this application embodiment can select training samples without manual intervention, thereby improving the efficiency of training sample selection and thus improving the training efficiency of the model.

[0100] In a second aspect of this invention, a recommended method is also provided, see [link to relevant documentation]. Figure 6 The above methods include:

[0101] Step S61: Obtain the search term to be searched;

[0102] Step S62: Input the search term into the pre-trained recall model so that the pre-trained recall model can identify one or more recall results that match the search term and the matching degree corresponding to each recall result. The pre-trained recall model is a network model trained by any of the recall model training methods mentioned above.

[0103] Step S63: Sort one or more recall results in descending order of matching degree corresponding to each recall result, and select the top N results for feedback.

[0104] The search term to be searched can be a search term entered by the user during actual use. The pre-trained recall model in this embodiment can be a model trained according to any of the recall model training methods described above. In actual use, after obtaining the trained recall model according to the above training methods, the search term to be searched is input so that the trained recall model can match the recall results from a pre-created database based on the search term. All or part of the matched results are then matched. When sorting one or more recall results in descending order based on the matching degree corresponding to each recall result and selecting the top N for feedback, the results can also be sorted and fed back according to the matching degree, so that those with higher matching degrees are ranked higher, thereby improving the user experience when the user clicks on the top results first.

[0105] As can be seen, the method of this application embodiment can input search terms into a pre-trained recall model, so that the pre-trained recall model can identify one or more recall results that match the search terms and the matching degree corresponding to each recall result. Based on the matching degree corresponding to each recall result, one or more recall results are sorted in descending order, and the top N are selected for feedback, thereby realizing the calculation and feedback of recall results based on search terms by the trained recall model.

[0106] In a third aspect of this invention, a recall model training apparatus is also provided, see [link to relevant documentation]. Figure 7 The aforementioned device includes:

[0107] The similarity acquisition module 701 is used to acquire the feature similarity between at least one search term sample and multiple first recall results corresponding to each search term sample, wherein each first recall result corresponds to a feature similarity, and the feature similarity represents the similarity between the first recall result and at least one search term sample;

[0108] The sample selection module 702 is used to obtain first selection information for each first recall result within a first preset historical time range, and select one or more as first negative samples from multiple first recall results according to the first selection information.

[0109] The model training module 703 is used to input at least one search term sample and one or more first negative samples into the recall model to be trained, calculate the corresponding calculation result based on the recall model to be trained, calculate the current loss of the recall model to be trained based on the calculation result, adjust the parameters of the recall model to be trained based on the current loss, return to the step of inputting at least one search term sample and one or more first negative samples into the recall model to be trained and continue training until the preset iteration stopping condition is reached or the current loss is less than the preset loss threshold, and obtain the trained recall model.

[0110] In one possible implementation, the sample selection module includes:

[0111] The recall result identification submodule is used to identify the first recall results whose corresponding feature similarity is not within a first preset range among multiple first recall results, and obtain one or more second recall results;

[0112] The first negative sample acquisition submodule is used to acquire first selection information for each first recall result within a first preset historical time range, and select one or more from one or more second recall results as first negative samples based on the first selection information.

[0113] In one possible implementation, the sample selection module includes:

[0114] The selection information acquisition submodule is used to acquire selection information for each first recall result within a first preset historical time range, wherein the selection information for each first recall result represents the information of the recall result selected by the user in each first recall result;

[0115] The first negative sample selection submodule is used to select one or more as the first negative sample from multiple first recall results according to the first selection information and the first preset probability.

[0116] In one possible implementation, the first negative sample selection submodule includes:

[0117] The selection probability acquisition unit is used to acquire the selection probability corresponding to each of multiple preset time ranges. Among the selection probabilities corresponding to each of the multiple preset time ranges, the selection probability corresponding to the preset time range closer to the current time is higher.

[0118] The probability selection unit is used to select one or more as first negative samples from multiple first recall results based on the second selection information corresponding to multiple preset time ranges and the selection probability corresponding to multiple preset time ranges.

[0119] In one possible implementation, the first negative sample selection submodule includes:

[0120] The third recall result identification unit is used to identify the recall results other than the first negative sample among multiple first recall results, and obtain the third recall result;

[0121] The probability identification unit is used to identify the recall results of the preset target type in the third recall results, and select one or more of the identified recall results as the first negative samples according to the second preset probability, wherein the second preset probability is greater than the first preset probability.

[0122] As can be seen, the apparatus of this application embodiment can obtain the feature similarity between at least one search term sample and multiple first recall results corresponding to each search term sample, and obtain and select one or more first negative samples from the multiple first recall results according to the first selection information, thereby inputting the at least one search term sample and the one or more first negative samples into the recall model to be trained for training, thereby obtaining a trained recall model. Thus, the selection of training samples can be carried out without manual intervention, improving the efficiency of training sample selection and thus improving the training efficiency of the model.

[0123] In a fourth aspect of this invention, a recommended device is also provided, see [link to relevant documentation]. Figure 8 The aforementioned device includes:

[0124] The search term acquisition module 801 is used to acquire the search terms to be searched;

[0125] The matching degree recognition module 802 is used to input the search term into the pre-trained recall model so that the pre-trained recall model can identify one or more recall results that match the search term and the matching degree corresponding to each recall result. The pre-trained recall model is a network model trained by any of the recall model training methods mentioned above.

[0126] The result feedback module 803 is used to sort one or more recall results in descending order based on the matching degree corresponding to each recall result, and select the top N results for feedback.

[0127] As can be seen, the apparatus of this application embodiment can input search terms into a pre-trained recall model, so that the pre-trained recall model can identify one or more recall results that match the search terms and the matching degree corresponding to each recall result. Based on the matching degree corresponding to each recall result, one or more recall results are sorted in descending order, and the top N are selected for feedback, thereby realizing the calculation and feedback of recall results based on search terms by the trained recall model.

[0128] This invention also provides an electronic device, such as... Figure 9 As shown, it includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0129] Memory 903 is used to store computer programs;

[0130] When processor 901 executes a program stored in memory 903, it performs the following steps:

[0131] Obtain the feature similarity between at least one search term sample and multiple first recall results corresponding to each search term sample, wherein each first recall result corresponds to a feature similarity, and the feature similarity represents the similarity between the first recall result and at least one search term sample;

[0132] Obtain first selection information for each first recall result within a first preset historical time range, and select one or more as first negative samples from multiple first recall results based on the first selection information;

[0133] Input at least one search term sample and one or more first negative samples into the recall model to be trained, and calculate the corresponding calculation result based on the recall model to be trained. Calculate the current loss of the recall model to be trained based on the calculation result, adjust the parameters of the recall model to be trained based on the current loss, and return to the step of inputting at least one search term sample and one or more first negative samples into the recall model to be trained to continue training until the preset iteration stopping condition is reached or the current loss is less than the preset loss threshold, and the trained recall model is obtained.

[0134] This invention also provides an electronic device, such as... Figure 10 As shown, it includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004.

[0135] Memory 1003 is used to store computer programs;

[0136] When processor 1001 executes a program stored in memory 1003, it performs the following steps:

[0137] Get the search terms to be searched;

[0138] The search term is input into the pre-trained recall model so that the pre-trained recall model can identify one or more recall results that match the search term and the matching degree corresponding to each recall result. The pre-trained recall model is a network model trained by any of the recall model training methods mentioned above.

[0139] Based on the matching degree corresponding to each recall result, one or more recall results are sorted in descending order, and the top N results are selected for feedback.

[0140] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0141] The communication interface is used for communication between the aforementioned terminal and other devices.

[0142] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0143] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0144] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the recall model training method described in any of the above embodiments.

[0145] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the recommended methods described in the above embodiments.

[0146] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the recall model training methods described in the above embodiments.

[0147] In yet another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the recommended methods described in the above embodiments.

[0148] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0149] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0150] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, storage media, and computer program products are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0151] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A recall model training method, characterized in that, The method includes: Obtain the feature similarity between at least one search term sample and multiple first recall results corresponding to each search term sample, wherein each first recall result corresponds to a feature similarity, and the feature similarity represents the similarity between the first recall result and the at least one search term sample; Obtain first selection information for each of the first recall results within a first preset historical time range, and select one or more as first negative samples from the plurality of first recall results based on the first selection information; The at least one search term sample and the one or more first negative samples are input into the recall model to be trained, and the corresponding calculation result is calculated according to the recall model to be trained. The current loss of the recall model to be trained is calculated according to the calculation result. The parameters of the recall model to be trained are adjusted according to the current loss. The process is repeated until the preset iteration stop condition is reached or the current loss is less than the preset loss threshold, and the trained recall model is obtained. The step of obtaining selection information for each of the first recall results within a first preset historical time range, and selecting one or more as first negative samples from the plurality of first recall results based on the first selection information, includes: Obtain first selection information for each of the first recall results within a first preset historical time range, wherein the first selection information for each of the first recall results represents information about the recall results selected by the user in each of the first recall results; Based on the first selection information, one or more of the multiple first recall results are selected as first negative samples according to a first preset probability. Different preset time ranges are set with different first preset probabilities, and the first preset probability is higher for preset time ranges closer to the current time.

2. The method according to claim 1, characterized in that, The step of obtaining first selection information for each of the first recall results within a first preset historical time range, and selecting one or more as first negative samples from the plurality of first recall results based on the first selection information, includes: Identify the first recall results whose feature similarity is not within a first preset range among the plurality of first recall results, and obtain one or more second recall results; Obtain first selection information for each of the first recall results within a first preset historical time range, and select one or more as first negative samples from the one or more second recall results based on the first selection information.

3. The method according to claim 1, characterized in that, The first selection information includes second selection information corresponding to multiple preset time ranges. The step of selecting one or more samples as first negative samples from the multiple first recall results according to the first selection information and a first preset probability includes: Obtain the first preset probability corresponding to each of the plurality of preset time ranges; Based on the second selection information corresponding to each of the multiple preset time ranges and the first preset probability corresponding to each of the multiple preset time ranges, one or more are selected as the first negative samples from the multiple first recall results.

4. The method according to claim 1, characterized in that, The step of selecting one or more as first negative samples from the plurality of first recall results according to the first selection information and a first preset probability includes: A third recall result is obtained by identifying the recall results other than the first negative sample among the multiple first recall results; The recall results of the third recall results are identified as having a preset target type, and one or more of the identified recall results are selected as the first negative samples according to a second preset probability, wherein the second preset probability is greater than the first preset probability.

5. A recommended method, characterized in that, The method includes: Get the search terms to be searched; The search term is input into a pre-trained recall model so that the pre-trained recall model can identify one or more recall results that match the search term and the matching degree corresponding to each recall result, wherein the pre-trained recall model is a network model trained by any of the recall model training methods described in claims 1-4. Based on the matching degree corresponding to each recall result, the one or more recall results are sorted in descending order, and the top N results are selected for feedback.

6. A recall model training device, characterized in that, The device includes: The similarity acquisition module is used to acquire the feature similarity between at least one search term sample and multiple first recall results corresponding to each search term sample, wherein each first recall result corresponds to a feature similarity, and the feature similarity represents the similarity between the first recall result and the at least one search term sample; The sample selection module is used to obtain first selection information for each of the first recall results within a first preset historical time range, and select one or more as first negative samples from the plurality of first recall results according to the first selection information. The model training module is used to input the at least one search term sample and the one or more first negative samples into the recall model to be trained, calculate the corresponding calculation result according to the recall model to be trained, calculate the current loss of the recall model to be trained according to the calculation result, adjust the parameters of the recall model to be trained according to the current loss, return to the step of inputting the at least one search term sample and the one or more first negative samples into the recall model to be trained and continue training until a preset iteration stop condition is reached or the current loss is less than a preset loss threshold, and a trained recall model is obtained. The sample selection module includes: The selection information acquisition submodule is used to acquire first selection information for each of the first recall results within a first preset historical time range, wherein the first selection information for each of the first recall results represents the information of the recall results selected by the user in each of the first recall results; The first negative sample selection submodule is used to select one or more as first negative samples from the plurality of first recall results according to the first selection information and a first preset probability; wherein, different preset time ranges are set with different first preset probabilities, and the first preset probability is higher for preset time ranges that are closer to the current time.

7. A recommended device, characterized in that, include: The search term acquisition module is used to acquire the search terms to be searched. The search term input module is used to input the search term into a pre-trained recall model so that the pre-trained recall model can identify one or more recall results that match the search term and the matching degree corresponding to each recall result. The pre-trained recall model is a network model trained by the recall model training method described in any one of claims 1-4. The result feedback module is used to sort the one or more recall results in descending order based on the matching degree corresponding to each recall result, and select the top N results for feedback.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-4 or 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-4 or 5.

Images