Negative sample determination method, device, electronic device and readable storage medium

Abstract

The application provides a negative sample determination method and device, electronic equipment and readable storage medium, and belongs to the technical field of big data. The application obtains a query list corresponding to a search word, the query list including a plurality of query words; determines a positive sample and a target ranking position of the positive sample in the query list among the plurality of query words; for each query word after the target ranking position in the query list, selects a corresponding target word in a preset set according to a selection strategy corresponding to the query word, the preset set including the query word and a recall word in a recall pool; and determines each query word before the target ranking position in the query list and each target word as a negative sample of the search word. In this way, the diversity of the negative sample of the optimization model is improved, thereby improving the optimization effect on the recommendation model.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a negative sample determination method, device, electronic device, and readable storage medium. Background Art

[0002] In a search scenario, the search process typically involves receiving a user's search term through a search bar. The recommendation model generates a query list based on the search term, which contains multiple possible queries. This list is then displayed below the search bar, allowing the user to select the query they are interested in. Currently, the sampling process for this search scenario typically involves using the user's selected query term as a positive sample and other unselected query terms as negative samples. These samples can then be used to optimize the recommendation model.

[0003] For example, a user enters the search term "magic" in the search bar, and "Fantastic Four", "Pokémon", "Pokémon the Movie" and "The Fantastic Tuxedo" appear in the query list. If the user selects "Pokémon" as the last query term, "Pokémon" is sampled as a positive sample, and "Fantastic Four", "Pokémon the Movie" and "The Fantastic Tuxedo" are sampled as negative samples.

[0004] However, in actual applications, it is found that when users select the query term they really want to search, they rarely check other query terms ranked after the query term in the query list. Therefore, for query terms ranked after the positive samples in the query list, directly using them as negative samples for optimizing the recommendation model may not be very useful for model learning, and the optimization effect is poor. Summary of the Invention

[0005] In order to solve the technical problem that directly using query words ranked after positive samples in the query list as negative samples for optimizing the recommendation model results in poor optimization effect, the present application provides a negative sample determination method, device, electronic device and readable storage medium.

[0006] In a first aspect, a negative sample determination method is provided, the method comprising:

[0007] Obtaining a query list corresponding to a search term, wherein the query list includes multiple query terms;

[0008] Determining a positive sample among the plurality of query words and a target ranking position of the positive sample in the query list;

[0009] For each query word in the query list after the target ranking position, a target word corresponding to the query word is selected from a preset set according to a selection strategy corresponding to the query word, the preset set containing the query word and a recall word in a recall pool;

[0010] The query word in the query list before the target ranking position and each target word are determined as negative samples of the search word.

[0011] In one possible implementation, the selecting a target word from a preset set according to a selection strategy corresponding to the query word includes:

[0012] A first selection probability and a second selection probability are determined based on the ranking position of the query word in the query list, wherein the first selection probability is a probability of selecting the query word as the target word, and the second selection probability is a probability of selecting the recall word as the target word;

[0013] The target word is selected from the preset set according to the first selection probability and the second selection probability.

[0014] In one possible implementation, the method further includes:

[0015] Determining whether a query based on the query word has been performed within a preset time period;

[0016] In a case where the query word has not been selected as a negative sample of the search word within a preset time period, the steps of determining a first selection probability and a second selection probability based on the ranking position of the query word in the query list to selecting a target word from a preset set according to the first selection probability and the second selection probability are performed;

[0017] In a case where the query word has been selected as a negative sample of the search word within a preset time period, the query word is determined as the target word.

[0018] In one possible implementation, the determining a positive sample from a plurality of query words includes:

[0019] A query instruction corresponding to the search word is obtained, and the query instruction is parsed to obtain a query word identifier corresponding to the query instruction;

[0020] From a plurality of query words, a query word corresponding to the query word identifier is determined as the positive sample.

[0021] In one possible implementation, the obtaining a query list corresponding to the search word includes:

[0022] The search term is input into a recommendation model so that the recommendation model outputs the query list.

[0023] In one possible implementation, the method further includes:

[0024] Training the recommendation model using the positive samples and the negative samples to obtain an updated model;

[0025] The recommendation model is updated using the updated model.

[0026] In a second aspect, a negative sample determination device is provided, the device comprising:

[0027] An acquisition module, configured to acquire a query list corresponding to a search term, wherein the query list includes a plurality of query terms;

[0028] A first determining module is configured to determine a positive sample among the plurality of query words and a target ranking position of the positive sample in the query list;

[0029] a selection module configured to select, for each query term in the query list whose ranking position is after the target ranking position, a corresponding target term from a preset set according to a selection strategy corresponding to the query term, the preset set including the query term and recall terms in the recall pool;

[0030] The second determining module is configured to determine the query words whose ranking positions are before the target ranking position in the query list, and each of the target words, as negative samples of the search word.

[0031] In one possible implementation, the selection module is specifically configured to:

[0032] Determining a first selection probability and a second selection probability based on the ranking position of the query term in the query list, wherein the first selection probability is a probability of selecting the query term as the target term, and the second selection probability is a probability of selecting the recall term as the target term;

[0033] The target word is selected from the preset set according to the first selection probability and the second selection probability.

[0034] In one possible embodiment, the device further includes:

[0035] A third determining module is used to determine whether a query based on the query term has been performed within a preset time period;

[0036] an execution module, configured to, if the query term has not been selected as a negative sample of the search term within a preset time period, execute the steps of determining a first selection probability and a second selection probability based on the ranking position of the query term in the query list, and selecting the target term from the preset set according to the first selection probability and the second selection probability;

[0037] The fourth determining module is configured to determine the query term as the target term if the query term has been selected as a negative sample of the search term within a preset time period.

[0038] In a possible implementation, the first determining module is specifically configured to:

[0039] Obtaining a query instruction corresponding to the search term, and parsing the query instruction to obtain a corresponding query term identifier;

[0040] Among the multiple query words, the query word corresponding to the query word identifier is determined as the positive sample.

[0041] In one possible implementation, the acquisition module is specifically configured to:

[0042] The search term is input into a recommendation model so that the recommendation model outputs the query list.

[0043] In one possible embodiment, the device further includes:

[0044] A training module, configured to train the recommendation model using the positive samples and the negative samples to obtain an updated model;

[0045] An updating module is configured to update the recommendation model using the updated model.

[0046] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0047] Memory for storing computer programs;

[0048] The processor is configured to implement any of the method steps described in the first aspect when executing a program stored in the memory.

[0049] In a fourth aspect, a computer-readable storage medium is provided, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0050] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned negative sample determination methods.

[0051] Beneficial effects of the embodiments of the present application:

[0052] The embodiments of the present application provide a method, device, electronic device, and readable storage medium for determining negative samples. The present application first obtains a query list corresponding to a search term, the query list including multiple query terms; and determines positive samples and the target ranking position of the positive samples in the query list from the multiple query terms; then, for each query term in the query list whose ranking position is after the target ranking position, selects a corresponding target term from a preset set according to a selection strategy corresponding to the query term, the preset set including the query term and the recall terms in the recall pool; finally, determines the query term in the query list whose ranking position is before the target ranking position, and each target term, as a negative sample of the search term. That is, in the present application, for the query term in the query list whose ranking position is after the positive sample, it is not directly used as a negative sample for optimizing the recommendation model. Instead, the corresponding target term is selected as a negative sample from the preset set constructed by the query term and the recall pool according to a preset strategy. In this way, the diversity of negative samples can be improved, thereby improving the optimization effect of the recommendation model.

[0053] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0056] Figure 1 A flowchart of a negative sample determination method provided in an embodiment of the present application;

[0057] Figure 2 A flowchart of another negative sample determination method provided in an embodiment of the present application;

[0058] Figure 3 A schematic diagram of the structure of a negative sample determination device provided in an embodiment of the present application;

[0059] Figure 4A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0061] The following will describe in detail a negative sample determination method provided by an embodiment of the present application in conjunction with specific implementation methods. Figure 1 The specific steps are as follows:

[0062] S101: Obtain a query list corresponding to a search term, where the query list includes multiple query terms.

[0063] A negative sample determination method provided in an embodiment of the present application is applied to a search scenario. In this search scenario, when a user enters a search term in a search bar, a query list corresponding to the search term is generated, wherein the query list includes multiple query terms.

[0064] For example, when a user enters the search term "天" in the search bar, the generated query list includes five query terms, which are "Demi-Gods and Semi-Devils", "Day Day Up", "Watching Movies Every Day", "Sky" and "Weather Forecast" from top to bottom.

[0065] In one embodiment, a specific implementation of obtaining a query list corresponding to a search term may include: inputting the search term into a recommendation model, so that the recommendation model outputs the query list.

[0066] S102: Determine positive samples from the plurality of query words and target ranking positions of the positive samples in the query list.

[0067] Positive samples refer to the query terms that the user finally selects for query in the query list.

[0068] In one embodiment, determining a positive sample from the plurality of query terms may include: obtaining a query instruction corresponding to the search term, parsing the query instruction to obtain a corresponding query term identifier; and determining, from the plurality of query terms, the query term corresponding to the query term identifier as the positive sample. The query instruction refers to an instruction generated when the user selects a query term for the final search, and the query instruction carries the query term identifier.

[0069] For example, the user clicks on "every day watch movie" in the 5 query words in the above example query list, and a corresponding query instruction is generated according to the user's click behavior and the identification of the specific clicked query word. The identification of the query word is obtained by analyzing the query instruction, and then the query word "every day watch movie" corresponding to the identification is determined as a positive sample.

[0070] S103, for each query word in the query list after the target ranking position, selecting a corresponding target word in a preset set according to the selection strategy corresponding to the query word, the preset set containing the query word and the recall words in the recall pool.

[0071] The recall pool refers to a set containing all recall words based on the search word, excluding the query words on the query list (i.e. recall words).

[0072] For each query word in the query list after the target ranking position (i.e. the position of the positive sample in the query list), it is generally a query word that has not been viewed by the user when selecting, and its reference to model learning is low. Therefore, in the embodiments of the present application, it is not directly used as a negative sample, but a corresponding target word is selected in a preset set according to the selection strategy corresponding to the query word. In this way, for query words with low reference, recall words in part of the preset set can be used to replace them, thereby ensuring the diversity of negative samples.

[0073] As for how to select the corresponding target word in the preset set, it will be described in detail in the following embodiments, which will not be described here.

[0074] S104, determining the query words in the query list before the target ranking position, and each of the target words, as negative samples of the search word.

[0075] In the embodiments of the present application, the query words in the query list before the target ranking position are generally query words that have been viewed by the user when selecting, and their reference to model learning is high. They are determined as negative samples of the search word together with the target words.

[0076] In an embodiment of the present application, first, a query list corresponding to a search term is obtained, the query list including multiple query terms; and a positive sample and a target ranking position of the positive sample in the query list are determined among the multiple query terms; then, for each query term in the query list whose ranking position is after the target ranking position, a corresponding target term is selected from a preset set according to a selection strategy corresponding to the query term, the preset set including the query term and the recall term in the recall pool; finally, the query term in the query list whose ranking position is before the target ranking position, and each target term, are determined as negative samples of the search term. That is, in the present application, for the query term in the query list whose ranking position is after the positive sample, it is not directly used as a negative sample for optimizing the recommendation model, but the corresponding target term is selected as a negative sample in the preset set constructed by the query term and the recall pool according to a preset strategy. In this way, the diversity of negative samples can be improved, thereby improving the optimization effect of the recommendation model.

[0077] In another embodiment of the present application, the method may further include the following steps: training the recommendation model using the positive samples and the negative samples to obtain an updated model; and updating the recommendation model using the updated model. This enables automatic optimization and updating of the recommendation model based on the selected positive and negative samples, improving the efficiency of the optimization and updating.

[0078] See also Figure 2 , is a flow chart of another embodiment of a negative sample determination method provided by an embodiment of the present invention. Figure 2 The process shown above Figure 1 Based on the process shown in FIG, it is described how to select the corresponding target word from the preset set according to the selection strategy corresponding to the query word. Figure 2 As shown, the process may include the following steps:

[0079] S201, determining a first selection probability and a second selection probability based on the ranking position of the query term in the query list, wherein the first selection probability is the probability of selecting the query term as the target term, and the second selection probability is the probability of selecting the recall term as the target term.

[0080] S202: Select the target word from the preset set according to the first selection probability and the second selection probability.

[0081] The following is a unified description of S201 and S202:

[0082] The first selection probability refers to the probability of selecting the query word at this position as the target word, that is, the probability of selecting the query word at this position as a negative sample.

[0083] The second selection probability refers to the probability of selecting the recall word as the target word.

[0084] In actual applications, for each query term in the query list that is ranked after the positive sample, the later the query term is ranked, the greater the probability that the user has not viewed the query term when selecting it, and the lower its reference value for model learning. Therefore, in this embodiment of the application, for each query term in the query list that is ranked after the positive sample, the first selection probability of selecting the query term as a negative sample can be determined based on the query term's ranking position, and the probability of selecting the recall term as a negative sample can be further determined based on the first selection probability.

[0085] The sum of the first and second selection probabilities is 1. Furthermore, the later the query term is positioned, the smaller the first selection probability is, while the larger the second selection probability is. This allows for a lower probability of selection as a negative sample for a less relevant query term. Furthermore, the target term is selected from the preset set based on the first and second selection probabilities.

[0086] As an optional implementation method, the first selection probability corresponding to query word j is , the second selection probability P2 = 1-P1. Where m is the number of sorting positions in the query list, and j is the sorting position of query term j in the sorting sequence.

[0087] For example, there are 10 ranking positions in the query list, that is, 10 query terms are recommended, and the positive sample is at the 6th position, then the first selection probability corresponding to the query term at the 7th position is , then the first selection probability corresponding to the query word at the 8th position is , then the first selection probability corresponding to the query word at the 9th position is , then the first selection probability corresponding to the query word at the 10th position is .

[0088] In this embodiment, the probability of selecting a query term as a negative sample (i.e., the first selection probability) is determined based on its position in the query list, as well as the probability of replacing the query term with a recall term as a negative sample (i.e., the second selection probability). Target terms are then selected from a preset set based on the first and second selection probabilities. This allows for a lower probability of selecting a query term as a negative sample for less relevant query terms in the query list, thereby ensuring optimal performance when subsequently optimizing the recommendation model.

[0089] In another embodiment of the present application, the method may further include the following steps: determining whether a query has been performed based on the query term within a preset time period; in a case where the query term has not been selected as a negative sample of the search term within the preset time period, executing the step of determining the first selection probability and the second selection probability based on the sorting position of the query term in the query list to select the target term in the preset set according to the first selection probability and the second selection probability; in a case where the query term has been selected as a negative sample of the search term within the preset time period, determining the query term as the target term.

[0090] In an embodiment of the present application, for a query term ranked after a positive sample in a query list, if it was selected by a user as the last query term used for a query within a preset time period (e.g., within 90 days), it is considered to have a high reference value as a negative sample. Therefore, the query term is directly selected as the target term, that is, the query term is directly selected as a negative sample. If it was not selected by the user as the last query term used for a query within the preset time period, the steps of determining a first selection probability and a second selection probability based on the query term's position in the query list and selecting a target term from a preset set according to the first and second selection probabilities are performed. In this way, negative samples with higher reference value can be further selected.

[0091] Based on the same technical concept, the embodiment of the present application also provides a negative sample determination device, such as Figure 3 As shown, the device includes:

[0092] An acquisition module 301 is configured to acquire a query list corresponding to a search term, wherein the query list includes multiple query terms;

[0093] A first determining module 302 is configured to determine a positive sample among the plurality of query words and a target ranking position of the positive sample in the query list;

[0094] A selection module 303 is configured to select, for each query term in the query list whose ranking position is after the target ranking position, a corresponding target term from a preset set according to a selection strategy corresponding to the query term, the preset set including the query term and recall terms in the recall pool;

[0095] The second determining module 304 is configured to determine the query words ranked before the target ranking position in the query list, and each target word, as negative samples of the search word.

[0096] In one possible implementation, the selection module is specifically configured to:

[0097] Determining a first selection probability and a second selection probability based on the ranking position of the query term in the query list, wherein the first selection probability is a probability of selecting the query term as the target term, and the second selection probability is a probability of selecting the recall term as the target term;

[0098] The target word is selected from the preset set according to the first selection probability and the second selection probability.

[0099] In one possible embodiment, the device further includes:

[0100] A third determining module is used to determine whether a query based on the query term has been performed within a preset time period;

[0101] an execution module, configured to, if the query term has not been selected as a negative sample of the search term within a preset time period, execute the steps of determining a first selection probability and a second selection probability based on the ranking position of the query term in the query list, and selecting the target term from the preset set according to the first selection probability and the second selection probability;

[0102] The fourth determining module is configured to determine the query term as the target term if the query term has been selected as a negative sample of the search term within a preset time period.

[0103] In a possible implementation, the first determining module is specifically configured to:

[0104] Obtaining a query instruction corresponding to the search term, and parsing the query instruction to obtain a corresponding query term identifier;

[0105] Among the multiple query words, the query word corresponding to the query word identifier is determined as the positive sample.

[0106] In one possible implementation, the acquisition module is specifically configured to:

[0107] The search term is input into a recommendation model so that the recommendation model outputs the query list.

[0108] In one possible embodiment, the device further includes:

[0109] A training module, configured to train the recommendation model using the positive samples and the negative samples to obtain an updated model;

[0110] An updating module is configured to update the recommendation model using the updated model.

[0111] In an embodiment of the present application, first, a query list corresponding to a search term is obtained, the query list including multiple query terms; and a positive sample and a target ranking position of the positive sample in the query list are determined among the multiple query terms; then, for each query term in the query list whose ranking position is after the target ranking position, a corresponding target term is selected from a preset set according to a selection strategy corresponding to the query term, the preset set including the query term and the recall term in the recall pool; finally, the query term in the query list whose ranking position is before the target ranking position, and each target term, are determined as negative samples of the search term. That is, in the present application, for the query term in the query list whose ranking position is after the positive sample, it is not directly used as a negative sample for optimizing the recommendation model, but the corresponding target term is selected as a negative sample in the preset set constructed by the query term and the recall pool according to a preset strategy. In this way, the diversity of negative samples can be improved, thereby improving the optimization effect of the recommendation model.

[0112] Based on the same technical concept, the embodiment of the present application also provides an electronic device, such as Figure 4 As shown, it includes a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0113] Memory 113, for storing computer programs;

[0114] The processor 111 is configured to execute the program stored in the memory 113 by performing the following steps:

[0115] Obtaining a query list corresponding to a search term, wherein the query list includes multiple query terms;

[0116] Determining a positive sample among the plurality of query words and a target ranking position of the positive sample in the query list;

[0117] For each query word in the query list whose ranking position is after the target ranking position, selecting a corresponding target word from a preset set according to a selection strategy corresponding to the query word, the preset set including the query word and recall words in the recall pool;

[0118] The query words whose ranking positions in the query list are before the target ranking position, and each of the target words, are determined as negative samples of the search word.

[0119] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0120] The communication interface is used for communication between the above electronic device and other devices.

[0121] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0122] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0123] In another embodiment provided in the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of any of the above negative sample determination methods are implemented.

[0124] In another embodiment provided in the present application, a computer program product containing instructions is also provided, and when the computer program product is run on a computer, the computer is caused to execute any of the negative sample determination methods in the above embodiments.

[0125] In the embodiments described above, all or some of the steps can be implemented by software, hardware or firmware, or any combination thereof. When implemented in software, one or more computer programs can be used to perform the steps. The computer programs can be stored in one or more computer-readable storage media, which can be any available media that can be read by a computer. The computer-readable storage media can be a computer program product. By way of example, the computer-readable storage media can include a RAM, a ROM, an EEPROM, a CD-ROM, a floppy disk, a DVD, a Blu-ray disk, a magnetic tape, an optical data storage device, or the like, or any suitable combination thereof. The computer programs can be loaded into a computer, which can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus, to cause the computer to perform the steps.

[0126] It should be noted that the terms "first" and "second" and the like in this text are used only to distinguish one entity or action from another, and do not necessarily require or imply any actual such relationship or order between the entities or actions. Also, the terms "comprises", "comprising", or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0127] The above description is merely one implementation of the application, and those skilled in the art will be able to make modifications and variations to the embodiments without departing from the spirit or scope of the application. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A negative sample determination method, characterized in that: The method comprises: Obtaining a query list corresponding to a search term, wherein the query list includes multiple query terms; Determining a positive sample among the plurality of query words and a target ranking position of the positive sample in the query list; For each query word in the query list whose ranking position is after the target ranking position, selecting a corresponding target word from a preset set according to a selection strategy corresponding to the query word, the preset set including the query word and recall words in the recall pool; Determine the query words in the query list that are ranked before the target ranked position, and each of the target words, as negative samples of the search word; The step of selecting a corresponding target word from a preset set according to a selection strategy corresponding to the query word includes: Determining a first selection probability and a second selection probability based on the ranking position of the query term in the query list, wherein the first selection probability is a probability of selecting the query term as the target term, and the second selection probability is a probability of selecting the recall term as the target term; The target word is selected from the preset set according to the first selection probability and the second selection probability.

2. The method according to claim 1, characterized in that The method further comprises: Determining whether a query based on the query term has been performed within a preset time period; If the query term has not been selected as a negative sample of the search term within a preset time period, performing the steps of determining a first selection probability and a second selection probability based on the ranking position of the query term in the query list and selecting the target term from the preset set according to the first selection probability and the second selection probability; In a case where the query word has been selected as a negative sample of the search word within a preset time period, the query word is determined as the target word.

3. The method according to claim 1, characterized in that Determining positive samples from the plurality of query words includes: Obtaining a query instruction corresponding to the search term, and parsing the query instruction to obtain a corresponding query term identifier; Among the multiple query words, the query word corresponding to the query word identifier is determined as the positive sample.

4. The method according to claim 1, wherein The step of obtaining a query list corresponding to a search term includes: The search term is input into a recommendation model so that the recommendation model outputs the query list.

5. The method according to claim 4, characterized in that The method further comprises: Training the recommendation model using the positive samples and the negative samples to obtain an updated model; The recommendation model is updated using the updated model.

6. A negative sample determination device, characterized in that: The device comprises: An acquisition module, configured to acquire a query list corresponding to a search term, wherein the query list includes a plurality of query terms; A first determining module is configured to determine a positive sample among the plurality of query words and a target ranking position of the positive sample in the query list; a selection module configured to select, for each query term in the query list whose ranking position is after the target ranking position, a corresponding target term from a preset set according to a selection strategy corresponding to the query term, the preset set including the query term and recall terms in the recall pool; A second determination module is configured to determine the query words in the query list that are ranked before the target ranked position, and each of the target words, as negative samples of the search word; The selection module is specifically used to: Determining a first selection probability and a second selection probability based on the ranking position of the query term in the query list, wherein the first selection probability is a probability of selecting the query term as the target term, and the second selection probability is a probability of selecting the recall term as the target term; The target word is selected from the preset set according to the first selection probability and the second selection probability.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1 to 5 when executing a program stored in a memory.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.

Images