Model training method, searching method and related products

Abstract

The invention discloses a model training method, a search method and a related product, the model training method is used for training a target model, the target model comprises a search representation model and a content representation model, and the model training method comprises the following steps: obtaining a training search word and training search content related to the training search word; obtaining training search representation of the training search term by utilizing the search representation model; a training prompt corresponding to the training search content is generated, and the training prompt is used for indicating the content representation model to compress the training search content and comprises at least part of the training search content; obtaining training content representation of the training search content based on the training prompt by utilizing the content representation model; and training the target model based on the training search representation and the training content representation.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a model training method, a search method and related products. Background Art

[0002] Search scenarios are widely present in daily life and work. Specifically, content matching the search terms is obtained by searching based on the search terms. Thanks to its powerful performance, deep learning models are increasingly being used in search scenarios. Therefore, how to train deep learning models for search is of great significance. Summary of the invention

[0003] The present application provides a model training method, a search method and related products, wherein the related products include a model training device, a search device, an electronic device, a computer-readable storage medium and a computer program product.

[0004] In a first aspect, a model training method is provided, the method being used to train a target model, the target model comprising a search representation model and a content representation model, the method comprising:

[0005] Acquire a training search term and training search content related to the training search term;

[0006] Obtaining a training search representation of the training search term using the search representation model;

[0007] generating a training prompt corresponding to the training search content, wherein the training prompt is used to instruct the content representation model to compress the training search content and includes at least part of the training search content;

[0008] Using the content representation model, obtaining a training content representation of the training search content based on the training prompt;

[0009] The target model is trained based on the training search representation and the training content representation.

[0010] In combination with any embodiment of the present application, the first position in the training prompt is used to indicate the compression result of the training search content, and the using the content representation model to obtain the training content representation of the training search content based on the training prompt includes:

[0011] Inputting the training prompt into the content representation model so that the content representation model encodes the training prompt to obtain a training hidden state sequence of the training prompt;

[0012] The training content representation of the training search content is obtained based on the hidden state located at the first position in the training hidden state sequence.

[0013] In combination with any implementation manner of the present application, the training search content includes a first search content and a second search content, the training content representation includes a first content representation of the first search content and a second content representation of the second search content, the first search content is content that matches the training search term, and the second search content is content that does not match the training search term;

[0014] The training of the target model based on the training search representation and the training content representation includes:

[0015] Determining a first similarity between the training search representation and the first content representation;

[0016] determining a second similarity between the training search representation and the second content representation;

[0017] Determining a content loss of the target model based on the first similarity and the second similarity;

[0018] Parameters of the target model are updated based on the content loss.

[0019] In combination with any embodiment of the present application, the second search content includes third search content, the third search content is content that does not match the reference search term, the reference search term is a search term different from the training search term and matches the first search content, and the training content representation includes a third content representation of the third search content;

[0020] Before determining the content loss of the target model based on the first similarity and the second similarity, the method further includes:

[0021] determining a third similarity between the training search representation and the third content representation;

[0022] The determining the content loss of the target model based on the first similarity and the second similarity includes:

[0023] Determining a content contrast loss of the target model based on the first similarity and the second similarity;

[0024] Determining a content boundary loss of the target model based on the first similarity and the third similarity;

[0025] The content loss is determined based on the content contrast loss and the content boundary loss.

[0026] In combination with any embodiment of the present application, the training prompt includes target content in the search content, the training prompt is further used to instruct the content representation model to compress the target content, and the second position in the training prompt is used to indicate the compression result of the target content;

[0027] After obtaining the training hidden state sequence, the method further includes:

[0028] Obtaining a target content representation of the target content based on the hidden state at the second position in the training hidden state sequence;

[0029] The training of the target model based on the training search representation and the training content representation includes:

[0030] Obtaining a content loss based on the training search representation and the training content representation;

[0031] Obtaining a target loss based on the target content representation and the training search representation;

[0032] Based on the content loss and the target loss, obtaining a total loss of the target model;

[0033] Parameters of the target model are updated based on the total loss.

[0034] In combination with any embodiment of the present application, the training prompt further instructs the content representation model to obtain a predicted search term based on the search content, the predicted search term being a search term that matches the training search content, and the third position in the training prompt is used to indicate the predicted search term;

[0035] After obtaining the training hidden state sequence, the method further includes:

[0036] Obtaining a predicted search term representation of the predicted search term based on the hidden state at the third position in the training hidden state sequence;

[0037] The training of the target model based on the training search representation and the training content representation includes:

[0038] Obtaining a content loss based on the training search representation and the training content representation;

[0039] Obtaining a search loss based on the predicted search term representation and the training search term representation;

[0040] Based on the content loss and the search loss, obtaining a total loss of the target model;

[0041] Parameters of the target model are updated based on the total loss.

[0042] In combination with any embodiment of the present application, the training prompt indicates to compress the search content based on the predicted search term;

[0043] The obtaining the training content representation of the training search content based on the hidden state located at the first position in the training hidden state sequence includes:

[0044] The training content representation of the training search content is obtained based on the hidden state located at the first position and the hidden state located at the third position in the training hidden state sequence.

[0045] In combination with any implementation manner of the present application, the step of obtaining the training search representation of the training search term by using the search representation model includes:

[0046] The training search term and the training category are concatenated to obtain a target search text, wherein the training category is the category to which the training search term belongs;

[0047] The target search text is input into the search representation model, and the representation output by the search representation model is obtained as the training search representation.

[0048] In a second aspect, a model training method is provided for training a target model, wherein the target model includes a search representation model and a content representation model, and the method includes:

[0049] Acquire a training search term and training search content related to the training search term;

[0050] Obtaining a training search representation of the training search term using the search representation model;

[0051] generating a training prompt corresponding to the training search content, wherein the training prompt is used to instruct the content representation model to compress the training search content and includes at least part of the training search content;

[0052] Using the content representation model, obtaining a training content representation of the training search content based on the training prompt;

[0053] Based on the training search representation and the training content representation, performing a first training on the target model;

[0054] The search representation model after the first training is used as the first search representation model;

[0055] Determining a second search representation model, wherein the structure of the second search representation model is different from the structure of the first search representation model, or the number of parameters of the second search representation model is less than the number of parameters of the first search representation model;

[0056] performing a second training on the second search representation model based on the training search terms and the search representations obtained by inputting the training search terms into the first search representation model;

[0057] The content representation model after the first training and the second search representation model after the second training are used as target models.

[0058] In a third aspect, a search method is provided, the method being used to determine matching content that matches a target search term from a database to be searched using a target model, the target model being trained according to the first aspect and any implementation manner thereof, the target model being trained according to the second aspect, the database to be searched comprising content to be searched and content representations to be searched of the content to be searched, the content representations to be searched being obtained by processing the content to be searched using a content representation model of the target model, the method comprising:

[0059] Obtaining the target search term;

[0060] Inputting the target search term into the search representation model of the target model to obtain the target search representation of the target search term;

[0061] The target search representation is used to search a database to be searched, and the content to be searched having the content representation to be searched that matches the target search representation is obtained as the matching content.

[0062] In a fourth aspect, a model training device is provided for training a target model, wherein the target model includes a search representation model and a content representation model, and the model training device includes:

[0063] An acquisition unit, used for acquiring a training search word and training search content related to the training search word;

[0064] A processing unit, configured to obtain a training search representation of the training search term using the search representation model;

[0065] A generating unit, configured to generate a training prompt corresponding to the training search content, wherein the training prompt is used to instruct the content representation model to compress the training search content and includes at least part of the training search content;

[0066] The processing unit is configured to obtain a training content representation of the training search content based on the training prompt by using the content representation model;

[0067] A training unit is used to train the target model based on the training search representation and the training content representation.

[0068] In combination with any embodiment of the present application, the first position in the training prompt is used to indicate the compression result of the training search content, and the processing unit is specifically used to:

[0069] Inputting the training prompt into the content representation model so that the content representation model encodes the training prompt to obtain a training hidden state sequence of the training prompt;

[0070] The training content representation of the training search content is obtained based on the hidden state located at the first position in the training hidden state sequence.

[0071] In combination with any implementation manner of the present application, the training search content includes a first search content and a second search content, the training content representation includes a first content representation of the first search content and a second content representation of the second search content, the first search content is content that matches the training search term, and the second search content is content that does not match the training search term;

[0072] The training unit is specifically used for:

[0073] Determining a first similarity between the training search representation and the first content representation;

[0074] determining a second similarity between the training search representation and the second content representation;

[0075] Determining a content loss of the target model based on the first similarity and the second similarity;

[0076] Parameters of the target model are updated based on the content loss.

[0077] In combination with any embodiment of the present application, the second search content includes third search content, the third search content is content that does not match the reference search term, the reference search term is a search term different from the training search term and matches the first search content, and the training content representation includes a third content representation of the third search content;

[0078] The processing unit is further used for:

[0079] determining a third similarity between the training search representation and the third content representation;

[0080] The training unit is specifically used for:

[0081] Determining a content contrast loss of the target model based on the first similarity and the second similarity;

[0082] Determining a content boundary loss of the target model based on the first similarity and the third similarity;

[0083] The content loss is determined based on the content contrast loss and the content boundary loss.

[0084] In combination with any embodiment of the present application, the training prompt includes target content in the search content, the training prompt is further used to instruct the content representation model to compress the target content, and the second position in the training prompt is used to indicate the compression result of the target content;

[0085] The processing unit is further used for:

[0086] Obtaining a target content representation of the target content based on the hidden state at the second position in the training hidden state sequence;

[0087] The training unit is specifically used for:

[0088] Obtaining a content loss based on the training search representation and the training content representation;

[0089] Obtaining a target loss based on the target content representation and the training search representation;

[0090] Based on the content loss and the target loss, obtaining a total loss of the target model;

[0091] Parameters of the target model are updated based on the total loss.

[0092] In combination with any embodiment of the present application, the training prompt further instructs the content representation model to obtain a predicted search term based on the search content, the predicted search term being a search term that matches the training search content, and the third position in the training prompt is used to indicate the predicted search term;

[0093] The processing unit is further configured to obtain a predicted search term representation of the predicted search term based on the hidden state located at the third position in the training hidden state sequence;

[0094] The training unit is specifically used for:

[0095] Obtaining a content loss based on the training search representation and the training content representation;

[0096] Obtaining a search loss based on the predicted search term representation and the training search term representation;

[0097] Based on the content loss and the search loss, obtaining a total loss of the target model;

[0098] Parameters of the target model are updated based on the total loss.

[0099] In combination with any embodiment of the present application, the training prompt indicates to compress the search content based on the predicted search term;

[0100] The processing unit is specifically used for:

[0101] The training content representation of the training search content is obtained based on the hidden state located at the first position and the hidden state located at the third position in the training hidden state sequence.

[0102] In conjunction with any implementation manner of the present application, the processing unit is specifically used to:

[0103] The training search term and the training category are concatenated to obtain a target search text, wherein the training category is the category to which the training search term belongs;

[0104] The target search text is input into the search representation model, and the representation output by the search representation model is obtained as the training search representation.

[0105] In a fifth aspect, a model training device is provided for training a target model, wherein the target model includes a search representation model and a content representation model, and the model training device includes:

[0106] An acquisition unit, used for acquiring a training search word and training search content related to the training search word;

[0107] A processing unit, configured to obtain a training search representation of the training search term using the search representation model;

[0108] A generating unit, configured to generate a training prompt corresponding to the training search content, wherein the training prompt is used to instruct the content representation model to compress the training search content and includes at least part of the training search content;

[0109] The processing unit is configured to obtain a training content representation of the training search content based on the training prompt by using the content representation model;

[0110] A training unit, configured to perform a first training on the target model based on the training search representation and the training content representation;

[0111] The processing unit is used to use the search representation model after the first training as the first search representation model;

[0112] a determining unit, configured to determine a second search representation model, wherein the structure of the second search representation model is different from that of the first search representation model, or the number of parameters of the second search representation model is less than that of the first search representation model;

[0113] The training unit is configured to perform a second training on the second search representation model based on a training search term and a search representation obtained by inputting the training search term into the first search representation model;

[0114] The processing unit is used to use the content representation model after the first training and the second search representation model after the second training as the target model.

[0115] In a sixth aspect, a search device is provided, characterized in that the search device is used to determine matching content matching a target search term from a database to be searched using a target model, the target model is trained according to the first aspect and any implementation manner thereof, the target model is trained according to the second aspect, the database to be searched includes content to be searched and content representation to be searched of the content to be searched, the content representation to be searched is obtained by processing the content to be searched using a content representation model of the target model, and the search device includes:

[0116] An acquisition unit, used for acquiring the target search term;

[0117] A processing unit, configured to input the target search term into a search representation model of the target model to obtain a target search representation of the target search term;

[0118] The retrieval unit is used to use the target search representation to retrieve the to-be-searched database, and obtain the to-be-searched content having the to-be-searched content representation matching the target search representation as the matching content.

[0119] In the seventh aspect, an electronic device is provided, comprising: a processor and a memory, the memory being used to store computer program code, the computer program code comprising computer instructions, and when the processor executes the computer instructions, the electronic device executes the first aspect and any one of its embodiments described above, and when the processor executes the computer instructions, the electronic device either executes the second aspect described above, and when the processor executes the computer instructions, the electronic device either executes the third aspect described above.

[0120] In an eighth aspect, another electronic device is provided, comprising: a processor, a sending device, an input device, an output device and a memory, wherein the memory is used to store computer program code, wherein the computer program code includes computer instructions, and when the processor executes the computer instructions, the electronic device executes the first aspect and any one of its embodiments described above, and the computer program code includes computer instructions, and when the processor executes the computer instructions, the electronic device either executes the second aspect described above, and when the processor executes the computer instructions, the electronic device either executes the third aspect described above.

[0121] In the ninth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the first aspect and any one of its embodiments described above, or when the program instructions are executed by the processor, the processor is caused to execute the second aspect described above, or when the program instructions are executed by the processor, the processor is caused to execute the third aspect described above.

[0122] In the tenth aspect, a computer program product is provided, which includes a computer program or instructions, which, when the computer program or instructions are run on a computer, enable the computer to execute the above-mentioned first aspect and any one of its embodiments; when the computer program or instructions are run on a computer, enable the computer to execute the above-mentioned second aspect; and when the computer program or instructions are run on a computer, enable the computer to execute the above-mentioned third aspect.

[0123] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

[0124] In the present application, after obtaining the training search terms and the training search content related to the training search terms, the model training device uses the search representation model in the target model to obtain the training search representation of the training search terms. Then, a training prompt corresponding to the training search content is generated, wherein the training prompt is used to instruct the content representation model to compress the training search content and include at least part of the content of the training search content. Then, the content representation model is used to obtain the training content representation of the training search content based on the training prompt. Finally, the target model can be trained based on the training search representation and the training content representation. In some embodiments, the content representation model is a language model with a relatively large scale, such as a large language model (LLM), wherein the relatively large scale includes: a large number of parameters, a large amount of data required for training, and a large amount of calculation required for training, that is, the content representation model in the target model has a relatively strong performance, so the representation of the content generated based on the content representation model in the target model can improve the accuracy of the information carried by the representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0125] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0126] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0127] Figure 1 A flowchart of a model training method provided in an embodiment of the present application;

[0128] Figure 2 A schematic diagram of the relationship between a reference search term, a training search term, a bridge search content, and a third search content provided in an embodiment of the present application;

[0129] Figure 3 A schematic diagram of a training framework provided in an embodiment of the present application;

[0130] Figure 4 A flowchart of another model training method provided in an embodiment of the present application;

[0131] Figure 5 A schematic diagram of another training framework provided in an embodiment of the present application;

[0132] Figure 6 A schematic diagram of another training framework provided in an embodiment of the present application;

[0133] Figure 7 A flowchart of a search method provided in an embodiment of the present application;

[0134] Figure 8 A schematic diagram of the structure of a model training device provided in an embodiment of the present application;

[0135] Fig. 9 A schematic diagram of the structure of another model training device provided in an embodiment of the present application;

[0136] Fig.10 A schematic diagram of the structure of a search device provided in an embodiment of the present application;

[0137] Fig.11 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0138] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0139] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0140] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0141] The embodiment of the present application provides two different model training methods and a search method, wherein the executor of the model training method is a model training device, wherein the model training device can be any electronic device that can execute the technical solution disclosed in the embodiment of the method of the present application. Optionally, the model training device can be one of the following: a computer, a server. It should be understood that the embodiment of the method of the present application can also be implemented by a processor executing a computer program code. The two model training methods provided in the embodiment of the present application are both used to train a target model, wherein the target model includes a search representation model and a content representation model, the search representation model is used to generate a representation of a search term, and the content representation model is used to generate a representation of content. Optionally, the target model is a dual-tower model including a search representation model and a content representation model. In the implementation of the present application, the content includes: text, video, image, audio. Optionally, the model training devices that execute the two different model training methods are different devices.

[0142] The following describes the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Figure 1 , Figure 1 A flow chart of a model training method provided in an embodiment of the present application is provided, and the flow chart is a flow chart of the first model training method.

[0143] 101. Obtain training search terms and training search content related to the training search terms.

[0144] In an embodiment of the present application, the training search term can be any search term, and the search term is the basis for the search. For example, if the training search term is food, then the content to be searched is related to food. In one implementation method of obtaining the training search term, the model training device receives the training search term input by the user through the input component to obtain the training search term. The training search content includes content that matches the training search term and content that does not match the training search term. For example, when the content is text, the training search content includes text that matches the training search term and text that does not match the training search term. In one possible implementation method, the training search content matches the training search term, which means that the semantics of the training search content matches the phonetic of the training search term. In another possible implementation method, the training search content matches the training search term, which means that the training search content can be searched by searching based on the training search term. For example, if the training search content is a note, the training search term matches the note, which means that the note can be searched by searching based on the training search term. In another possible implementation, the training search content matches the training search term, which means that the training search content can be found by searching based on the training search term, and when the training search content is displayed as a search result of the training search term, the training search content is viewed. For example, the training search content is a note, and the training search term matches the note, which means that the note can be found by searching based on the training search term, and when the note is displayed as a search result of the training search term, the user views the note.

[0145] 102. Obtain a training search representation of the training search term using the search representation model.

[0146] In an embodiment of the present application, the search representation model is used to generate a representation (embedding) of the search term. Since the search representation model is used to generate a representation of the search term, the model training device inputs the training search term into the search representation model to obtain the representation of the training search term, which is the training search representation. The search representation model can be any language model such as BERT and LLM. Optionally, the search representation model includes an r-layer pre-trained language representation model (bidirectional encoder representation from transformers, BERT), wherein the r-layer BERT is connected in series, and specifically, the output of the previous layer of BERT is the input of the next layer of BERT. After the training search term is input into the first layer of BERT, after being processed by the r-layer BERT, the pooled representation (pooled embedding) output by the pooling layers ((pooling layers)) in the last layer of BERT is used as the training search representation. Optionally, the larger r is, the more layers of BERT, and the stronger the performance of the search representation model. The specific value of r can be determined according to actual needs. For example, r is 4, or r is 5.

[0147] In one possible implementation, the training search terms and training categories are concatenated to obtain a target search text, wherein the training category is the category to which the training search terms belong. The target search text is input into a search representation model to obtain a representation output by the search representation model as a training search representation.

[0148] In one possible implementation, the model training device determines a category matching the training search term from the preset categories as the training category. For example, the preset categories include entertainment, sports, and food, and the training search term is delicious Hunan cuisine, then the category matching the training search term in the preset categories is food, and accordingly, the training category is food.

[0149] In another possible implementation, the model training device determines the secondary category matching the training search term and the primary category to which the secondary category belongs from the preset category system. The secondary category and the primary category are used as training categories, and the training categories include the secondary category matching the training search term and the primary category to which the secondary category belongs.

[0150] In this implementation, the preset category system includes primary categories and secondary categories, wherein the secondary categories are subcategories of the primary categories, and the secondary categories may also include leaf nodes. For example, the preset category system includes the following primary categories: entertainment, sports, and food, wherein entertainment includes the following secondary categories: music, movies, and TV series, sports includes the following secondary categories: basketball, football, and table tennis, and food includes the following secondary categories: Hunan cuisine, Cantonese cuisine, and Sichuan cuisine. If the training search term is delicious Hunan cuisine, then the secondary category that matches the training search term is Hunan cuisine, and the primary category to which Hunan cuisine belongs is food. Therefore, the training categories include Hunan cuisine and food.

[0151] Since the training category is a category that matches the training search term, the training category carries information for determining the content that matches the training search term. For example, in the case where the training category is food, it can be determined that the text that matches the training search term is about food, which can reduce the probability of using content that is not related to food as content that matches the training search term. Therefore, after the model training device splices the training search term and the training category to obtain the target search text, the target search text is input into the search representation model, which can enable the search representation model to generate a training search representation based on the training search term and the training category, thereby improving the accuracy of the information carried by the training search representation.

[0152] In a possible implementation, the model training device performs word segmentation on the target search text to obtain a word segmentation result. Generate word embeddings in the word segmentation result to obtain word embeddings. Optionally, the word segmentation result is converted to the semantic space of the search representation model using a word embedding matrix to obtain word embeddings. The word embeddings are input into the search representation model to obtain a representation output by the search representation model as a training search representation. Optionally, the search representation model is BERT.

[0153] In this implementation, the model training device determines the training category to which the training search term belongs, and concatenates the training search term with the training category to obtain the target search text. The target search text is input into the search representation model, so that the search representation model can generate a training search representation based on the training search term and the training category, thereby improving the accuracy of the information carried by the training search representation.

[0154] 103. Generate a training prompt corresponding to the training search content.

[0155] In an embodiment of the present application, a training prompt (promt) is used to instruct a content representation model to compress training search content, and includes at least part of the training search content. The content representation model is used to generate a representation of the content. Optionally, the content representation model is an arbitrary language model such as an LLM. Optionally, the content representation model is a 7B (billions) LLM, that is, the parameter amount of the LLM of the content representation model is 7 billion. The content representation model can compress the information of the training search content by compressing the training search content to obtain a representation of the training search content.

[0156] Optionally, the training prompt instructs the content representation model to compress the training search content, which may be to instruct the content representation model to represent the training search content by compressing the content, wherein the amount of information in the compressed content is less than the amount of information in the training search content. For example, the training search content includes a paragraph or a sentence, and the compressed content is a word or a short sentence used to represent the paragraph or sentence.

[0157] 104. Using the content representation model, obtain a training content representation of the training search content based on the training prompt.

[0158] After the training prompt is input into the content representation model, the training prompt can be used to guide the content representation model to compress the training search content. The content representation model can obtain the training content representation of the training search content by compressing the training search content.

[0159] Optionally, after the training prompt is input into the content representation model, the content representation model may obtain a training hidden state sequence of the training prompt by encoding the training prompt, wherein the processing result may be obtained by encoding the training hidden state sequence, and the processing result is the result after the result obtained by compressing the training search content is written into the training prompt. Because the training hidden state sequence is used to determine the processing result, the training hidden state sequence includes the hidden state of the processing result, that is, the training hidden state sequence includes the hidden state of the result obtained by compressing the training search content. The training hidden state sequence corresponds to the training prompt, and specifically, if the word at the reference position in the training prompt is called the reference word, then the hidden state at the reference position in the training hidden state sequence is the representation of the reference word encoded by the content representation model. Therefore, the content representation model may obtain the training content representation based on the hidden state in the training hidden state sequence corresponding to the compression result of the training search content.

[0160] 105. Based on the training search representation and the training content representation, train the target model.

[0161] Since the training search representation is obtained through the search representation model in the target model, and the training content representation is obtained through the content representation model in the target model, the target model can be trained based on the training search representation and the training content representation. It should be understood that training the target model refers to updating the parameters of the search representation model in the target model and the parameters of the content representation model in the target model.

[0162] In one possible implementation, based on the similarity between the training search representation and the training content representation, the total loss of the target model can be determined, and then the parameters of the target model can be updated based on the total loss to complete the training of the target model. Optionally, when the training search term does not match the training search content, the similarity is negatively correlated with the total loss; when the training search term matches the training search content, the similarity is negatively correlated with the total loss.

[0163] In an embodiment of the present application, after obtaining the training search terms and the training search content related to the training search terms, the model training device uses the search representation model in the target model to obtain the training search representation of the training search terms. Then, a training prompt corresponding to the training search content is generated, wherein the training prompt is used to instruct the content representation model to compress the training search content and include at least part of the content of the training search content. Then, the content representation model is used to obtain the training content representation of the training search content based on the training prompt. Finally, the target model can be trained based on the training search representation and the training content representation. In some embodiments, the content representation model is a language model with a relatively large scale, such as LLM, wherein the relatively large scale includes: a large number of parameters, a large amount of data required for training, and a large amount of calculation required for training, that is, the content representation model in the target model has a relatively strong performance, so generating a representation of the content based on the content representation model in the target model can improve the accuracy of the information carried by the representation.

[0164] As an optional implementation, the first position in the training prompt is used to indicate the compression result of the training search content. Optionally, in the training prompt, the first position is a preset character (null character), for example, the first position is a space symbol or a null character. Optionally, the first position is the last position in the training prompt, for example, the training search content is a note. When s is 1, the training prompt is: Please compress the notes into one word: ". At this time, the position of " is the first position, that is, the first position is the last position in the training prompt. Because the content representation model uses the information of the word in the front order to generate the hidden state of the word in the back order when encoding the training prompt, when the first position is the last position in the training prompt, the content representation model can use the information of all the words in the training prompt to generate the hidden state corresponding to the training search content, thereby improving the accuracy of the information carried by the hidden state, so that when the training content representation is obtained based on the hidden state, the accuracy of the information carried by the training content representation can be improved. Optionally, the first position is used to write the word obtained by compressing the training search content, that is, when the compressed word should be written into the training prompt, the position of the compressed word in the training prompt is the first position.

[0165] In this embodiment, the model training device performs the following steps during the execution of step 104:

[0166] 2001. Input the training prompts into the content representation model so that the content representation model encodes the training prompts to obtain a training hidden state sequence of the training prompts.

[0167] After the training prompt is input into the content representation model, the content representation model can obtain the training hidden state sequence of the training prompt by encoding the training prompt, wherein the processing result can be obtained by encoding the training hidden state sequence, and the processing result is the result after the result obtained by compressing the training search content is written into the training prompt. Because the training hidden state sequence is used to determine the processing result, the training hidden state sequence includes the hidden state of the processing result, that is, the training hidden state sequence includes the hidden state of the result obtained by compressing the training search content. Among them, the training hidden state sequence corresponds to the training prompt. Specifically, if the word located at the reference position in the training prompt is called the reference word, then the hidden state located at the reference position in the training hidden state sequence is the representation of the reference word encoded by the content representation model. Therefore, the content representation model can obtain the training content representation based on the hidden state in the training hidden state sequence corresponding to the compression result of the training search content.

[0168] 2002. Based on the hidden state located at the first position in the training hidden state sequence, a training content representation of the training search content is obtained.

[0169] In an embodiment of the present application, the training content representation is a representation of the training content, and the training content representation includes information about the training search content. Because the training hidden state sequence corresponds to the training prompt, and the training hidden state sequence includes the hidden state of the result obtained by compressing the training content, the hidden state of the result obtained by compressing the training content is at the first position in the training hidden state sequence. Therefore, based on the hidden state at the first position in the training hidden state sequence, a training content representation of the training content can be obtained. In one possible implementation, the hidden state at the first position in the training hidden state sequence is used as the training content representation of the training content. In another possible implementation, the hidden state at the first position in the training hidden state sequence is input into a linear layer to reduce the dimension of the hidden state corresponding to the first position to obtain a training content representation with a preset dimension. Optionally, the preset value is 128.

[0170] In this embodiment, after the training prompt is input into the content representation model of the target model, the content representation model can obtain the training hidden state sequence of the training prompt by encoding the training prompt, wherein the processing result can be obtained by encoding the training hidden state sequence, and the processing result is the result after the result obtained by compressing the training search content is written into the first position in the training prompt. Because the training hidden state sequence is used to determine the processing result, the training hidden state sequence includes the hidden state of the processing result, that is, the training hidden state sequence includes the hidden state of the result obtained by compressing the training search content. Because the training hidden state sequence corresponds to the training prompt, and the training hidden state sequence includes the hidden state of the result obtained by compressing the training search content, the position of the hidden state of the result obtained by compressing the training search content in the training hidden state sequence is the first position. Therefore, based on the hidden state at the first position in the training hidden state sequence, the training content representation of the training search content can be obtained.

[0171] As an optional implementation, the training search content includes a first search content and a second search content, wherein the training content representation includes a first content representation of the first search content and a second content representation of the second search content, the first search content is content that matches the training search term, that is, the first search content is a positive sample, and the second search content is content that does not match the training search term, that is, the second search content is a negative sample. In this implementation, the model training device performs the following steps during the execution of step 105:

[0172] 3001. Determine a first similarity between a training search representation and a first content representation.

[0173] Optionally, the model training device determines the cosine similarity between the training search representation and the first content representation to obtain a first similarity.

[0174] 3002. Determine a second similarity between the training search representation and the second content representation.

[0175] Optionally, the model training device determines the cosine similarity between the training search representation and the second content representation to obtain a second similarity.

[0176] 3003. Determine the content loss of the target model based on the first similarity and the second similarity.

[0177] In the embodiment of the present application, the content loss represents the accuracy of the information carried by the training search content representation generated by the content representation model, and the accuracy of the information carried by the training search representation generated by the search representation model. Since the training search term matches the first search content, the first similarity is the similarity between the training search representation of the training search term and the first content representation of the first search content. Therefore, the larger the first similarity, the higher the accuracy of the information carried by the first search content representation and the accuracy of the information carried by the training search representation. Correspondingly, the first similarity is negatively correlated with the content loss. Since the training search term does not match the second search content, the second similarity is the similarity between the training search representation of the training search term and the second training search content representation of the second search content. Therefore, the smaller the second similarity, the higher the accuracy of the information carried by the second training search content representation and the accuracy of the information carried by the training search representation. Correspondingly, the second similarity is positively correlated with the content loss. Therefore, in the case where the first similarity is negatively correlated with the content loss and the second similarity is positively correlated with the content loss, the content loss of the target model can be determined based on the first similarity and the second similarity.

[0178] In one possible implementation, the model training device determines the content contrast loss of the target model based on the first similarity and the second similarity. Specifically, the training search term matches the first search content, the first similarity is negatively correlated with the content contrast loss, and the training search term matches the second search content, the second similarity is positively correlated with the content contrast loss. Based on the content difference between the first similarity and the second similarity, the content difference loss is determined, wherein the content difference loss is negatively correlated with the content difference. The content contrast loss and the content difference loss are weightedly summed to obtain the content loss.

[0179] Optionally, the content contrast loss is determined by:

[0180]

[0181] Among them, loss d1 is the content contrast loss, B is the number of training search contents, q i To train the search representation, d i is the first training search content representation, d jis the representation of the jth search content in the set of search contents. ∑ represents summation. log(·) represents a logarithmic function with base 10. exp(·) represents an exponential function with a natural number e as the base. cos(·) represents a cosine function. s represents a temperature coefficient.

[0182] In this implementation, the first similarity and the second similarity are both the similarities between the training search representation and the training search content representation. If the matching relationship between the training search term and the search content is called a content matching relationship, wherein the content matching relationship includes the training search term matching the first search content and the training search term not matching the second search content. Then the content comparison loss can represent the matching degree between the first similarity, the second similarity and the content matching relationship. Specifically, the smaller the content comparison loss, the higher the matching degree between the first similarity, the second similarity and the content matching relationship, which means that the information carried by the training search representation, the information carried by the first training search content representation, and the information carried by the second training search content representation are all high. The content difference loss can represent the content difference between the first similarity and the second similarity. In this way, by weighted summing the content comparison loss and the content difference loss, the content loss is obtained, so that the content loss can represent the matching degree between the first similarity and the content matching relationship, the matching degree between the second similarity and the content matching relationship, and the content difference between the first similarity and the second similarity, and then by reducing the content loss, the matching degree between the first similarity, the second similarity and the content matching relationship can be improved, and the content difference can be reduced, wherein improving the matching degree can improve the accuracy of the information carried by the training search representation and the accuracy of the information carried by the training search content representation, and reducing the content difference can shorten the distance between the training search representation and the first training search content representation, and can lengthen the distance between the training search representation and the second training search content representation. Therefore, when distinguishing positive samples and negative samples based on the representation of the search term output by the search representation model and the representation of the search content output by the content representation model, the accuracy of the distinction can be improved.

[0183] As an optional implementation, the second search content includes a third search content, and the third search content is content that does not match the reference search term, and the reference search term is a search term different from the training search term, and matches the first search content, that is, the third search content is a difficult negative sample in the negative sample. The training content representation includes a third content representation of the third search content. Since the first search content matches both the training search term and the reference search term, it means that there is at least one search content that matches both the reference search term and the training search term. For the convenience of expression, the search content that matches both the reference search term and the training search term is referred to as a bridge search content below. In this implementation, the first search content is a bridge search content.

[0184] See also Figure 2 , Figure 2A schematic diagram of the relationship between a reference search term, a training search term, a bridge search content, and a third search content provided in an embodiment of the present application. Figure 2 In , q1 is the training search term, q2 is the reference search term, d1 and d3 are the search contents that match the training search term, d2 is the search contents that do not match the training search term, and d3 and d4 are the search contents that match the reference search term. Figure 2 As shown, d3 matches both the reference search term and the training search term, that is, d2 is a bridge search content. d5 is a search content that does not match the reference search term, that is, d5 is the third search content.

[0185] In one possible implementation, Figure 2 The mismatch between the search term and the search content means that when the search term is used as the search basis to search for the search content, and the search content is displayed as the search result, the search content has not been viewed. In this case, the third search content is the search result obtained by searching based on the reference search term, but after the third search content is displayed as the search result of the reference search term, the third search content has not been viewed. For example, Figure 2 In , after the user inputs the training search term q1, the search results obtained include search content d1, search content d2, and search content d3. Search content d1, search content d2, and search content d3 are displayed to the user as the search results of the training search term q1. The user clicks to view search content d1 and search content d3, but does not click to view search content d2. At this time, search content d1 and search content d3 are both search contents that match the training search term q1, that is, search content d1 and search content d3 are both first search contents, and search content d2 is a search content that does not match the training search term q1, that is, search content d2 is the second search content. After the user inputs the reference search term q2, the search results obtained include search content d3, search content d4, and search content d5. Search content d3, search content d4, and search content d5 are displayed to the user as search results of reference search term q2. The user clicks to view search content d3 and search content d4, but does not click to view search content d5. At this time, search content d3 and search content d4 are both search content that matches the training search term q1, and search content d5 is a search content that does not match the training search term q1. Accordingly, search content d5 is the third search content of the training search term q1.

[0186] In another possible implementation, Figure 2 The mismatch between the search term and the search content means that the correlation between the search term and the search content is low. Optionally, low correlation means that the correlation is less than or equal to the correlation threshold, and high correlation means that the correlation is greater than the correlation threshold. At this time, the correlation between the third search content and the reference search term is low. For example, Figure 2In the figure, the correlation between search content d1 and training search term q1 and the correlation between search content d3 and training search term q1 are both high, and the correlation between search content d2 and training search term q1 is low, that is, search content d1 and search content d3 are both the first search content, and search content d2 is the second search content. The correlation between search content d3 and training search term q1 and the correlation between search content d4 and reference search term q2 are both high, and the correlation between search content d5 and reference search term q2 is low. Search content d5 is a search content that does not match training search term q1. Accordingly, search content d5 is the third search content of training search term q1.

[0187] In this embodiment, before determining the content loss of the target model based on the first similarity and the second similarity, the model training device further performs the following steps: determining a third similarity between the training search representation and the third content representation. After obtaining the third similarity, the "determining the content loss of the target model based on the first similarity and the second similarity" is implemented by performing the following steps: determining the content contrast loss of the target model based on the first similarity and the second similarity. Determining the content margin loss of the target model based on the first similarity and the third similarity. Determining the content loss based on the content contrast loss and the content margin loss.

[0188] The above-mentioned content boundary loss represents the content boundary difference between the first similarity and the third similarity. Specifically, when the content boundary difference is less than or equal to the content threshold, the content boundary loss is determined based on the content boundary difference, wherein the content boundary loss is negatively correlated with the content boundary difference. When the content boundary difference is greater than the content threshold, the content boundary loss is determined to be the content value, wherein when the content boundary difference is the content threshold, the content boundary loss is the content value.

[0189] In this implementation, when the content boundary loss is the content value, the content boundary loss converges. That is, when the content boundary difference is less than or equal to the content threshold, the content boundary loss does not converge, and then the content boundary loss is determined based on the content boundary difference by using the negative correlation between the content boundary loss and the content boundary difference, so as to reduce the content boundary loss to the content value by increasing the content boundary difference, thereby converging the content boundary loss. When the content boundary difference is greater than the content threshold, the content boundary loss is determined to be the content value, and the content boundary loss converges at this time. In other words, based on this implementation, the expected value of the content boundary difference can be made the content threshold. Specifically, the expected value of the difference between the first similarity and the third similarity is the content threshold. It should be understood that the greater the difference between the first similarity and the third similarity, the higher the accuracy of distinguishing whether the search content is the first search content or the third search content based on the similarity between the training search representation and the representation of the training search content, so the specific value of the content threshold can be set according to actual needs.

[0190] Optionally, the content boundary loss is determined by the following formula:

[0191] loss d2 =max(0,-[cos(q i ,d p )-cos(q i ,d n )]+margin1)...Formula (2)

[0192] Among them, loss d2 is the content boundary loss, q i To train the search representation, d p is the first search content representation, d n is the third search content representation, margin1 is the above content threshold, max(a, b) represents the maximum value of a and b, and cos(·) represents the cosine function.

[0193] The content boundary loss can be determined by the above formula, which can be obtained by cos(q i ,d p )-cos(q i ,d n ) is less than margin1, the content boundary loss is greater than 0, and in cos(q i ,d p )-cos(q i ,d n ) is greater than or equal to margin1, the content boundary loss is set to 0. In this way, when the content boundary loss converges, cos(q i ,d p )-cos(q i ,d n ) is greater than or equal to margin1. Since cos(q i ,d p ) is the cosine similarity between the training search representation and the first search content representation, cos(q i ,d n ) is the cosine similarity between the training search representation and the third search content representation, so that cos(q i ,d p )-cos(q i ,d n ) is greater than or equal to margin1, which can widen the gap between the two cosine similarities, thereby improving the accuracy of distinguishing whether the search content is the first search content or the third search content based on the similarity between the training search representation and the representation of the training search content. Therefore, when distinguishing positive samples and difficult negative samples based on the training search representation and the representation of the training search content, the distinction accuracy can be improved.

[0194] 3004. Update parameters of the target model based on the content loss.

[0195] Based on the content loss, the back-propagation gradient of the target model is determined, and the search content representation model is back-propagated based on the gradient. The parameters of the search content representation model are updated through back-propagation until the content loss converges and the parameter updating stops.

[0196] As an optional implementation, the training prompt includes the target content in the search content, the training prompt is also used to instruct the content representation model to compress the target content, and the second position in the training prompt is used to indicate the compression result of the target content. Optionally, the target content is one of the following: title, subject, tag.

[0197] Optionally, the second position is before the first position. The training prompt includes a word sequence, and the content representation model processes the training prompt by processing the word sequence. When the content representation model processes the word sequence, it processes the words in the word sequence according to the word order of the words in the word sequence, wherein the word order can represent the order in which the words in the word sequence are processed by the content representation model. When the content representation model processes words with a later word order, it will use the information of the words with a earlier word order. For example, the training prompt includes word A, word B, and word C, wherein the word order of word A is before the word order of word B, and the word order of word B is before the word order of word C. The hidden state sequence of the training prompt in the semantic space of the content representation model is the training hidden state sequence, wherein the training hidden state sequence includes hidden state D, hidden state E, and hidden state F, hidden state D is the embedding of word A, hidden state E is the embedding of word B, and hidden state F is the embedding of word C. Then, when the content representation model generates the training hidden state sequence, it first generates hidden state D, then uses the information of hidden state D and word B to generate hidden state E, and finally uses the information of hidden state D, hidden state E, and word C to generate hidden state F. The second position is before the first position, that is, the word sequence corresponding to the second position is before the word sequence corresponding to the first position, that is, when the content representation model generates the training hidden state sequence, it will use the hidden state corresponding to the second position to generate the hidden state corresponding to the first position. And because the hidden state corresponding to the second position is the hidden state of the target content, and the hidden state corresponding to the first position is the hidden state of the training search content, when the content representation model generates the training hidden state sequence, it will use the hidden state of the target content to generate the hidden state of the training search content.

[0198] After obtaining the training hidden state sequence, the model training device further performs the following steps: obtaining a target content representation of the target content based on the hidden state located at the second position in the training hidden state sequence.

[0199] In an embodiment of the present application, the target content representation is a representation of the target content, and the target content representation includes information about the target content. Because the training hidden state sequence corresponds to the training prompt, and the training hidden state sequence includes a hidden state of the result obtained by compressing the target content, the position of the hidden state of the result obtained by compressing the target content in the training hidden state sequence is the second position. Therefore, based on the hidden state at the second position in the training hidden state sequence, a target content representation of the target content can be obtained. In one possible implementation method, the hidden state at the second position in the training hidden state sequence is used as the target content representation of the target content.

[0200] After obtaining the target content representation, the model training device performs the following steps in the process of "training the target model based on the training search representation and the training content representation": obtaining content loss based on the training search representation and the training content representation. Obtaining target loss based on the target content representation and the training search representation. Obtaining the total loss of the target model based on the content loss and the target loss. Updating the parameters of the target model based on the total loss.

[0201] In the embodiment of the present application, the target loss represents the accuracy of the information carried by the target content representation generated by the content representation model, and the accuracy of the information carried by the training search representation generated by the search representation model. When the training search terms match the training search content, the greater the similarity between the target content representation and the training search representation, the smaller the target loss; when the training search terms do not match the training search content, the greater the similarity between the target content representation and the training search representation, the greater the target loss.

[0202] In one possible implementation, the target content representation includes a first target content representation corresponding to the first search content and a second target content representation corresponding to the second search content. The model training device determines a fourth similarity between the first target content representation and the training search representation. Determine a fifth similarity between the second target content representation and the training search representation. Based on the fourth similarity and the fifth similarity, determine the target contrast loss of the target model. Specifically, if the training search term matches the first search content, the fourth similarity is negatively correlated with the target contrast loss, and if the training search term does not match the second search content, the fifth similarity is positively correlated with the target contrast loss. Based on the target difference between the fourth similarity and the fifth similarity, determine the target difference loss, wherein the target difference loss is negatively correlated with the target difference. Perform a weighted sum of the target contrast loss and the target difference loss to obtain the target loss.

[0203] Optionally, the target contrast loss is determined by:

[0204]

[0205] Among them, losst1 is the target contrast loss, B is the number of training search contents, q i To train the search representation, t i is the first target content representation, t j is the representation of the target content of the j-th search content in the set of training search content. ∑ represents summation. log(·) represents a logarithmic function with base 10. exp(·) represents an exponential function with a natural number e as the base. cos(·) represents a cosine function. s is a temperature coefficient.

[0206] In this implementation, the fourth similarity and the fifth similarity are both the similarities between the training search representation and the target content representation. If the matching relationship between the training search term and the target content is called a target matching relationship, the target matching relationship includes the training search term matching the training search content corresponding to the target content, and the training search term not matching the training search content corresponding to the target content. Then the target contrast loss can represent the matching degree between the fourth similarity, the fifth similarity and the target matching relationship. Specifically, the smaller the target contrast loss, the higher the matching degree between the fourth similarity, the fifth similarity and the target matching relationship, which means that the information carried by the training search representation, the information carried by the first target content representation, and the information carried by the second target content representation are all high. The target difference loss can represent the target difference between the fourth similarity and the fifth similarity. In this way, by performing weighted summation on the target contrast loss and the target difference loss, the target loss is obtained, so that the target loss can represent both the matching degree of the fourth similarity and the target matching relationship, and the matching degree of the fifth similarity and the target matching relationship, and the target difference between the fourth similarity and the fifth similarity. Then, by reducing the target loss, the matching degree of the fourth similarity, the fifth similarity and the target matching relationship can be improved, and the target difference can be reduced. Among them, improving the matching degree can improve the accuracy of the information carried by the training search representation and the accuracy of the information carried by the target content representation. Reducing the target difference can shorten the distance between the training search representation and the first target content representation, and can increase the distance between the training search representation and the second target content representation.

[0207] As an optional implementation, the second search content includes the third search content, the target content in the third search content is the third target content, and the target content representation includes the third target content representation of the third target content. Before obtaining the target loss based on the target content representation and the training search representation, the model training device also performs the following steps: determining the sixth similarity between the training search representation and the third target content representation. Determine the difference between the fourth similarity and the sixth similarity to obtain the target boundary difference. When the target boundary difference is less than or equal to the target threshold, the target boundary loss is determined based on the target boundary difference, and the target boundary loss is negatively correlated with the target boundary difference. When the target boundary difference is greater than the target threshold, the target boundary loss is determined to be the target value, and when the target boundary difference is the target threshold, the target boundary loss is the target value.

[0208] In this embodiment, when the target boundary loss is the target value, the target boundary loss converges. That is, when the target difference is less than or equal to the target threshold, the target boundary loss does not converge, and then based on the negative correlation between the target boundary loss and the target boundary difference, the target boundary loss is determined based on the target boundary difference, so as to reduce the target boundary loss to the target value by increasing the target boundary difference, and then the target boundary loss converges. In the case where the target boundary difference is greater than the target threshold, the target boundary loss is determined to be the target value, and the target boundary loss converges. In other words, based on this embodiment, the expected value of the target boundary difference can be made the target threshold. Specifically, the expected value of the difference between the fourth similarity and the sixth similarity is the target threshold. It should be understood that the greater the difference between the fourth similarity and the sixth similarity, the higher the accuracy of distinguishing whether the search content corresponding to the target content is the first search content or the third search content based on the similarity between the training search representation and the representation of the target content, so the specific value of the target threshold can be set according to actual needs.

[0209] Optionally, the target boundary loss is determined by:

[0210] loss t2 =max(0,-[cos(q i t p )-cos(q i ,t n )]+margin2)...Formula (4)

[0211] Among them, loss t2 is the target boundary loss, q i To train the search representation, t p is the first target content representation, t n is the third target content representation, margin2 is the target threshold, max(a, b) represents the maximum value of a and b, and cos(·) represents the cosine function.

[0212] The target boundary loss is determined by the above formula, which can be obtained by cos(q i ,t p )-cos(q i ,t n ) is less than margin2, making the target boundary loss greater than 0. i ,t p )-cos(q i ,t n ) is greater than or equal to margin2, the target boundary loss is set to 0. In this way, when the target boundary loss converges, cos(p i ,t p )-cos(q i ,t n ) is greater than or equal to margin2. Since cos(q i ,t p ) is the cosine similarity between the training search representation and the first target content representation, cos(q i ,d n ) is the cosine similarity between the training search representation and the third target content representation, so that cos(q i ,t p )-cos(q i ,t n ) is greater than or equal to margin2, which can widen the gap between the two cosine similarities, thereby improving the accuracy of distinguishing whether the search content corresponding to the third target content is the first search content or the third search content based on the similarity between the training search representation and the third target content representation.

[0213] Optionally, after obtaining the target loss, the total loss is obtained by performing a weighted summation of the target loss and the content loss.

[0214] Since in the training prompt, the second position is before the first position, that is, the word order of the second position is before the word order of the first position, and the content representation model pays more attention to the content with a later word order than to the content with a forward word order, the content representation model pays less attention to the content corresponding to the second position when processing the training prompt, which may lead to low accuracy of the content corresponding to the second position, that is, low accuracy of the target content representation of the target content. Therefore, in the case where the target model includes a content representation model, the content loss and the target loss are weighted and summed to obtain the total loss, and then the parameters of the content representation model in the target model are updated based on the total loss, which can improve the accuracy of the representation of the target content generated by the content representation model.

[0215] As an optional implementation, the training prompt further instructs the content representation model to obtain a predicted search term based on the search content, where the predicted search term is a search term that matches the training search content, and the third position in the training prompt is used to indicate the predicted search term.

[0216] Optionally, the training prompt is used to guide the content representation model to compress the training search content based on the training search content and the predicted search term. The third position in the training prompt is used to write the predicted search term, and the third position is located before the first position. That is, the word sequence corresponding to the third position is before the word sequence corresponding to the first position, that is, when the content representation model generates a training hidden state sequence, it will use the hidden state corresponding to the third position to generate the hidden state corresponding to the first position. And because the hidden state corresponding to the third position is the hidden state of the predicted search term, and the hidden state corresponding to the first position is the hidden state of the training search content, the content representation model will use the hidden state of the predicted search term to generate the hidden state of the training search content when generating a training hidden state sequence.

[0217] After obtaining the training hidden state sequence, the model training device further performs the following steps: based on the hidden state located at the third position in the training hidden state sequence, obtain a predicted search term representation of the predicted search term.

[0218] In an embodiment of the present application, the predicted search term representation is a representation of the predicted search term, and the predicted search term representation includes information about the predicted search term. Because the training hidden state sequence corresponds to the training prompt, and the training hidden state sequence includes the hidden state of the predicted search term, the position of the hidden state of the predicted search term in the training hidden state sequence is the third position. Therefore, based on the hidden state at the third position in the training hidden state sequence, the predicted search term representation of the predicted search term can be obtained. In one possible implementation method, the hidden state at the third position in the training hidden state sequence is used as the predicted search term representation of the predicted search term.

[0219] After obtaining the predicted search term representation, the model training device performs the following steps in the process of "training the target model based on the training search representation and the training content representation": based on the training search representation and the training content representation, obtain the content loss. Based on the predicted search term representation and the training search representation, obtain the search loss. Based on the content loss and the search loss, obtain the total loss of the target model. Based on the total loss, update the parameters of the target model.

[0220] In the embodiment of the present application, the search loss represents the accuracy of the information carried by the predicted search term representation generated by the content representation model, and the accuracy of the information carried by the training search representation generated by the search representation model. When the training search term matches the training search content, the greater the similarity between the predicted search term representation and the training search representation, the smaller the search loss; when the training search term does not match the training search content, the greater the similarity between the predicted search term representation and the training search representation, the greater the search loss.

[0221] In one possible implementation, the predicted search term representation includes a first predicted search term representation corresponding to the first search content and a second predicted search term representation corresponding to the second search content. The model training device determines the seventh similarity between the first predicted search term representation and the training search representation. Determine the eighth similarity between the second predicted search term representation and the training search representation. Based on the seventh similarity and the eighth similarity, determine the search contrast loss of the target model. Specifically, the training search term matches the first search content, the seventh similarity is negatively correlated with the search contrast loss, and the training search term matches the second search content, and the eighth similarity is positively correlated with the search contrast loss. Based on the search difference between the seventh similarity and the eighth similarity, determine the search difference loss, wherein the search difference loss is negatively correlated with the search difference. Perform a weighted sum of the search contrast loss and the search difference loss to obtain the search loss.

[0222] Optionally, the search contrast loss is determined by:

[0223]

[0224] Among them, loss p1 is the search contrast loss, B is the number of training search contents, q i To train the search representation, p i is the first predicted search term representation, p j is the representation of the predicted search term of the j-th search content in the set of training search content. ∑ represents summation. log(·) represents a logarithmic function with base 10. exp(·) represents an exponential function with a natural number e as the base. cos(·) represents a cosine function. s is a temperature coefficient.

[0225] In this implementation, the seventh similarity and the eighth similarity are both the similarities between the training search representation and the predicted search term representation. If the matching relationship between the training search term and the training search is called a search matching relationship, the search matching relationship includes the matching of the training search term and the training search content corresponding to the training search, and the mismatch between the training search term and the training search content corresponding to the training search. Then the search comparison loss can represent the matching degree of the seventh similarity, the eighth similarity and the search matching relationship. Specifically, the smaller the search comparison loss, the higher the matching degree of the seventh similarity, the eighth similarity and the search matching relationship, which means that the information carried by the training search representation, the information carried by the first predicted search term representation, and the information carried by the second predicted search term representation are all high. The search difference loss can represent the search difference between the seventh similarity and the eighth similarity. In this way, by performing weighted summation of the search contrast loss and the search difference loss, the search loss is obtained, so that the search loss can represent both the matching degree between the seventh similarity and the search matching relationship and the matching degree between the eighth similarity and the search matching relationship, and the search difference between the seventh similarity and the eighth similarity. Further, by reducing the search loss, the matching degree between the seventh similarity, the eighth similarity and the search matching relationship can be improved, and the search difference can be reduced. Among them, improving the matching degree can improve the accuracy of the information carried by the training search representation and the accuracy of the information carried by the predicted search word representation. Reducing the search difference can shorten the distance between the training search representation and the first predicted search word representation, and can increase the distance between the training search representation and the second predicted search word representation.

[0226] As an optional implementation, the second search content includes the third search content, the predicted search term corresponding to the third search content is the third predicted search term, and the predicted search term representation includes the third predicted search term representation of the third predicted search term. Before obtaining the search loss based on the predicted search term representation and the training search representation, the model training device also performs the following steps: determining the ninth similarity between the training search representation and the third predicted search term representation. Determine the difference between the seventh similarity and the ninth similarity to obtain the search boundary difference. When the search boundary difference is less than or equal to the search threshold, the search boundary loss is determined based on the search boundary difference, and the search boundary loss is negatively correlated with the search boundary difference. When the search boundary difference is greater than the search threshold, the search boundary loss is determined as the search value, and when the search boundary difference is the search threshold, the search boundary loss is the search value.

[0227] In this embodiment, when the search boundary loss is the search value, the search boundary loss converges. That is, when the search boundary difference is less than or equal to the search threshold, the search boundary loss does not converge, and then based on the negative correlation between the search boundary loss and the search boundary difference, the search boundary loss is determined based on the search boundary difference, so as to reduce the search boundary loss to the search value by increasing the search boundary difference, and then the search boundary loss converges. In the case where the search boundary difference is greater than the search threshold, the search boundary loss is determined to be the search value, and the search boundary loss converges. That is to say, based on this embodiment, the expected value of the search boundary difference can be made the search threshold. Specifically, the expected value of the difference between the seventh similarity and the ninth similarity is the search threshold. It should be understood that the greater the difference between the seventh similarity and the ninth similarity, the higher the accuracy of distinguishing whether the search content corresponding to the predicted search term is the first search content or the third search content based on the similarity between the training search representation and the representation of the predicted search term, so the specific value of the search threshold can be set according to actual needs.

[0228] Optionally, the search boundary loss is determined by:

[0229] loss p2 =max(0, -[cos(q i , p p )-cos(q i , p n )]+margin3)..Formula (6)

[0230] Among them, loss p2 is the search boundary loss, q i To train the search representation, p p is the first predicted search term representation, p n is the third predicted search term representation, margin3 is the above search threshold, max(a, b) represents the maximum value of a and b, and cos(·) represents the cosine function.

[0231] The search boundary loss is determined by the above formula, which can be obtained by cos(q i ,p p )-cos(q i ,p n ) is less than margin3, making the search boundary loss greater than 0, in cos(q i ,p p )-cos(q i ,p n ) is greater than or equal to margin3, the search boundary loss is set to 0. In this way, when the search boundary loss converges, cos(q i ,p p )-cos(pi ,p n ) is greater than or equal to margin3. Since cos(q i ,p p ) is the cosine similarity between the training search representation and the first predicted search term representation, cos(q i ,d n ) is the cosine similarity between the training search representation and the third predicted search term representation, so that cos(q i ,p p )-cos(q i ,p n ) is greater than or equal to margin3, which can widen the gap between the two cosine similarities, thereby improving the accuracy of distinguishing whether the search content corresponding to the third predicted search term is the first search content or the third search content based on the similarity between the training search representation and the third predicted search term representation.

[0232] Optionally, after obtaining the search loss, the total loss is obtained by weighted summing the search loss and the content loss. Optionally, after obtaining the content loss, the target loss, and the search loss, the model training device obtains the total loss by weighted summing the content loss, the target loss, and the search loss.

[0233] Since the third position is before the first position in the training prompt, the content representation model will use the hidden state of the predicted search term to generate the hidden state of the training search content when generating the training hidden state sequence, that is, the training prompt guides the content representation model to generate the hidden state of the training search content through the chain of thought (CoT). Therefore, the accuracy of the hidden state of the predicted search term will affect the accuracy of the hidden state of the training search content, and then affect the accuracy of the representation of the training search content. Therefore, when the target model includes the content representation model, the search loss and the content loss are weighted and summed to obtain the total loss. In this way, the parameters of the content representation model are updated based on the total loss, which can improve the accuracy of the hidden state of the predicted search term generated by the content representation model, and then improve the accuracy of the representation of the training search content.

[0234] As an optional implementation, the training prompt indicates to compress the search content based on the predicted search words. The model training device implements "obtaining the training content representation of the training search content based on the hidden state at the first position in the training hidden state sequence" by executing the following steps: obtaining the training content representation of the training search content based on the hidden state at the first position and the hidden state at the third position in the training hidden state sequence. In this way, the training search content representation can be generated by using the information of the target content, the information of the predicted search words, and the information of the training search content, thereby improving the accuracy of the training search content representation.

[0235] As an optional implementation, the training search content is a note, and the training prompt is: "Note: {{"title":"{}", compressed into one word: "<TITLE_EMB> "."topic":"{}","content":"{}"}}, predict a predicted search term:

[0236] “<QUERY_EMB> ", combined with the predicted search terms, compress the above content into one word: "". For example, the title of the note is: I would like to ask if everyone is like this, the topic related to the note is: Taking you out of the siege, and the content of the note is: In an ordinary second-tier university, there are really not many design positions in the civil service exam, and it is difficult to learn. Then the training prompt is: "Note: {{"Title":"{I would like to ask if everyone is like this}", compressed into one word: "<TITLE_EMB> "."Topic":"{Take you out of the siege}","Content":"{There are not many design positions in ordinary second-tier universities preparing for civil service exams, and it is difficult to learn}"}}, predict a predicted search term: "<QUERY_EMB> ", combined with the predicted search terms, compress the above content into one word: "".

[0237] See also Figure 3 , Figure 3 A schematic diagram of a training framework provided in an embodiment of the present application. The training framework is used to implement the training method described above. Figure 3 As shown, the search content representation model of the target model includes LLM, and the search representation tower of the target model includes r layers of BERT, wherein the r layers of BERT are connected in series in sequence. Optionally, the larger r is, the more layers of BERT there are, and the stronger the performance of the search representation model is. The specific value of r can be determined according to actual needs. For example, r is 4, or r is 5. The input of the search content representation model includes training prompts, which include: Note content: {'title'..., 'content', ...}, compression..., wherein the title is the title of the note, the content is the content of the note, and compression refers to compressing the note into one word. It should be understood that in Figure 3 In the example, the note is the training search content and the title is the target content. The input of the search representation tower includes the training search term, the first-level category, and the second-level category. The second-level category is the second-level category to which the training search term belongs, and the first-level category is the second-level category to which the second-level category belongs.

[0238] After the training search terms, primary categories, and secondary categories are input into the search representation tower, the training prompts are first segmented using a tokenizer, and then the word embedding of the segmentation result is determined, and then the word embedding is input into the r-layer BERT, so that the r-layer BERT outputs the training search representation of the training search terms. After the training prompts are input into the search content representation model, the training prompts are first segmented using a tokenizer, and then the word embedding of the segmentation result of the training prompt is determined, and then the word embedding is input into the LLM, so that the LLM outputs the representation of the notes in the training prompts (i.e., the training search content representation). Figure 3 As shown, the training search content representation output by LLM includes positive text representation and negative text representation, wherein the positive text representation is the first training search content representation mentioned above, that is, the representation of the positive sample, and the negative text representation is the second training search content representation mentioned above, that is, the representation of the negative sample. The total loss of the target model can be determined by contrastive learning. Specifically, the total loss can be obtained based on the training search representation, the positive text representation, and the negative text representation. Finally, the parameters of the target model are updated based on the total loss until the total loss converges and the parameter update stops.

[0239] See also Figure 4 , Figure 4 A flow chart of another model training method provided in an embodiment of the present application is provided, and the flow chart is a flow chart of the second model training method.

[0240] 401. Obtain training search terms and training search content related to the training search terms.

[0241] The implementation of this step can refer to step 101 and will not be repeated here.

[0242] 402. Obtain a training search representation of the training search term using the search representation model.

[0243] The implementation of this step can refer to step 102, which will not be described in detail here.

[0244] 403. Generate a training prompt corresponding to the training search content.

[0245] The implementation of this step can refer to step 103, which will not be described in detail here.

[0246] 404. Using the content representation model, obtain a training content representation of the training search content based on the training prompt.

[0247] The implementation of this step can refer to step 104, which will not be described in detail here.

[0248] 405. Based on the training search representation and the training content representation, perform a first training on the target model.

[0249] In the embodiment of the present application, the implementation process of the first training can refer to the process of training the target model in step 105, which will not be repeated here.

[0250] 406. Use the search representation model after the first training as the first search representation model.

[0251] Specifically, based on the total loss, the back-propagation gradient of the target model is determined, and the search representation model is back-propagated based on the gradient, and the parameters of the search representation model are updated by back-propagation until the total loss converges to obtain a first search representation model. Since the first search representation model is obtained by updating the parameters of the search representation model, the semantic space of the first search representation model is the same as the semantic space of the search representation model.

[0252] See also Figure 5 , Figure 5 Schematic diagram of another training framework provided in an embodiment of the present application. The training framework is used to perform a first training on a target model. Figure 5 As shown, the search content model of the target model includes LLM, and the search representation model of the target model includes LLM. The input of the search content model includes training prompts, which include: note content: {'title'..., 'content', ...}, compressed..., where title is the title of the note, content is the content of the note, and compression refers to compressing the note into one word. It should be understood that in Figure 5 In the example, the note is the training search content and the target content is the title. The input of the search representation model includes the training search term, the first-level category, and the second-level category. The second-level category is the second-level category to which the training search term belongs, and the first-level category is the second-level category to which the second-level category belongs.

[0253] After the training search terms, primary categories, and secondary categories are input into the search representation model, the training prompts are first segmented using a word segmenter, and then the word vector of the segmentation result is determined, and then the word vector is input into the LLM, so that the LLM outputs the training search representation of the training search terms. After the training prompts are input into the search content representation model, the training prompts are first segmented using a word segmenter, and then the word vector of the segmentation result of the training prompt is determined, and then the word vector is input into the LLM, so that the LLM outputs the representation of the notes in the training prompts (i.e., the training search content representation). Figure 5As shown, the training search content representation output by LLM includes positive text representation and negative text representation, wherein the positive text representation is the first training search content representation described above, and the negative text representation is the second training search content representation described above. The total loss of the target model can be determined by contrastive learning, and specifically the total loss can be obtained based on the training search representation, the positive text representation, and the negative text representation. Finally, the parameters of the target model are updated based on the total loss until the total loss converges, and the parameter update is stopped. The search representation model after the first training is used as the first search representation model.

[0254] 407. Determine a second search representation model.

[0255] In an embodiment of the present application, the structure of the second search representation model is different from that of the first search representation model, or the parameter amount of the second search representation model is less than that of the first search representation model. Because the structure of the second search representation model is different from that of the first search representation model, the semantic space of the second search representation model is different from that of the first search representation model. And because the semantic space of the first search representation model is the same as the semantic space of the search representation model, and the semantic space of the search representation model is the same as the semantic space of the search content representation model, the semantic space of the search content representation model is different from the semantic space of the second search representation model. At this time, if it is necessary to calculate the representation of the text output by the search content representation model and the representation of the search term output by the second search representation model, it is necessary to first align the representation of the text output by the search content representation model with the representation of the search term output by the second search representation model. Therefore, if the second search representation model is directly used as the search representation model of the target model and the search content representation model is used as the search content representation model of the target model, it is not possible to directly determine whether the search term and the text match by calculating the similarity between the representation of the search term output by the second search representation model and the representation of the text output by the search content representation model. Instead, it is necessary to first align the semantic space of the search content representation model with the semantic space of the second search representation model so that the representation of the text output by the search content representation model is aligned with the representation of the search term output by the second search representation model.

[0256] Since the semantic space of the search content representation model is aligned with the semantic space of the second search representation model, the model with a large number of parameters in the search content representation model and the second search representation model will be aligned to the model with a small number of parameters, which will cause the performance of the model with a large number of parameters to decline. For example, the search content representation model is LLM, and the second search representation model includes 4 layers of BERT. Then the search content representation model has more parameters than the second search representation model, that is, the search content representation model is a model with a large number of parameters, and the second search representation model is a model with a small number of parameters. At this time, if the semantic space of the search content representation model is aligned with the semantic space of the second search representation model, the performance of the search content representation model will decline.

[0257] Optionally, the structure of the second search representation model is different from that of the first search representation model, or the structure of the second search representation model is different from that of the first search representation model and the number of parameters of the second search representation model is less than that of the first search representation model.

[0258] 408. Perform a second training on the second search representation model based on the training search terms and the search representations obtained by inputting the training search terms into the first search representation model.

[0259] In order to avoid the performance degradation of the model with a large number of parameters due to the alignment of the semantic space, in step 408, based on the training search words and the search representation obtained by inputting the training search words into the first search representation model, the second search representation model is trained for the second time to obtain the second search representation model after the second training, so that the semantic space of the first search representation model can be aligned with the semantic space of the second search representation model, and finally the semantic space of the first search representation model is the same as the semantic space of the second search representation model after the second training. Because the semantic space of the first search representation model is the same as the semantic space of the search representation model, and the semantic space of the search representation model is the same as the semantic space of the search content representation model, the semantic space of the second search representation model after the second training is the same as the semantic space of the search content representation model. At this time, if the target model is obtained based on the second search representation model after the second training and the search content representation model, then it can be directly determined whether the search word matches the content by calculating the similarity between the representation of the search word output by the second search representation model after the second training and the representation of the content output by the search content representation model. This can avoid aligning the semantic space of the search content representation model with the semantic space of the second search representation model, and further avoid a performance degradation of the model with a large number of parameters in the search content representation model and the second search representation model.

[0260] Optionally, the search representation model and the search content representation model are both LLM, and the second search representation model is BERT. In this case, if the semantic space of the search content representation model is aligned with the semantic space of the second search representation model, the performance of the search content representation model will decline. In step 407, the parameters of the search representation model are updated based on the total loss to obtain the first search representation model, which can avoid aligning the semantic space of the search content representation model with the semantic space of the search representation model, thereby avoiding the performance of the search content representation model from declining. In step 408, based on the training search terms and the search representation obtained by inputting the training search terms into the first search representation model, the second search representation model is subjected to a second training to obtain a second search representation model after the second training, so that the semantic space of the second search representation model after the second training can be made the same as the semantic space of the search content representation model. In this way, the target model is obtained based on the search content representation model and the second search representation model after the second training. When the structure of the search representation model is different from that of the search content representation model and the parameter quantity of the search representation model is greatly different from that of the search content representation model, it is possible to avoid aligning the semantic space of the search representation model with the semantic space of the search content representation model, thereby avoiding a decrease in the performance of the search content representation model.

[0261] In one possible implementation, the training search terms are input into the first search representation model to obtain the teacher search representation. The training search terms are input into the second search representation model to obtain the student search representation. Based on the distillation difference between the teacher search representation and the student search representation, the knowledge distillation loss is obtained, wherein the knowledge distillation loss is positively correlated with the distillation difference. Based on the knowledge distillation loss, the parameters of the second search representation model are updated to obtain the second search representation model after the second training.

[0262] In this implementation, the knowledge distillation loss can characterize the difference between the teacher's search representation and the student's search representation, and then based on the knowledge distillation loss, the parameters of the second search representation model are updated to obtain the second search representation model after the second training, which can narrow the difference between the representation of the search term output by the first search representation model and the representation of the search term output by the second search representation model, thereby reducing the difference between the representation of the search term output by the first search representation model and the representation of the search term output by the second search representation model after the second training.

[0263] As an optional implementation, the model training device implements "obtaining knowledge distillation loss based on the distillation difference between the teacher search representation and the student search representation" by performing the following steps: obtaining mean square error loss based on the mean square error between the teacher search representation and the student search representation, wherein the mean square error loss is positively correlated with the mean square error. Obtaining cosine loss based on the cosine similarity between the teacher search representation and the student search representation, wherein the cosine loss is negatively correlated with the cosine similarity. Performing weighted summation of the mean square error loss and the cosine loss to obtain the knowledge distillation loss.

[0264] Optionally, the mean square error loss is determined by:

[0265]

[0266] Among them, MSE is the mean square error loss, n is the number of training search terms, and y t is the teacher search representation output by the first search representation model, y s is the student search representation output by the second search representation model, ∑ n · represents the sum of the differences between n teacher search representations and the corresponding n student search representations.

[0267] Optionally, the cosine loss is determined by:

[0268]

[0269] Among them, Cosine Similarity Loss is the cosine loss, y is the teacher search representation, represents the student search representation, |y| represents the modulus of the teacher search representation, represents the model of the student's search representation, Represents the product of the teacher's search representation and the student's search representation.

[0270] 409. Use the content representation model after the first training and the second search representation model after the second training as target models.

[0271] In an embodiment of the present application, a model training device obtains training search terms and training search content related to the training search terms. The training search representation of the training search terms is obtained using the search representation model. A training prompt corresponding to the training search content is generated. The training content representation of the training search content is obtained based on the training prompt using the content representation model. The target model is first trained based on the training search representation and the training content representation. The search representation model after the first training is used as the first search representation model. A second search representation model is determined. The second search representation model is second trained based on the training search terms and the search representation obtained by inputting the training search terms into the first search representation model. The content representation model after the first training and the second search representation model after the second training are used as the target model. The performance of the search content representation model can be avoided from being reduced due to the alignment of the semantic space of the search content representation model with the semantic space of the second search representation model, thereby improving the performance of the search content representation model in the target model, thereby improving the performance of the target model.

[0272] See also Figure 6 , Figure 6 A schematic diagram of another training framework provided in an embodiment of the present application. The training framework is used to implement "performing a second training on a second search representation model based on a training search term and a search representation obtained by inputting the training search term into the first search representation model". Figure 6 As shown, the first search representation model includes LLM, and the second search representation model includes r layers of BERT, wherein the r layers of BERT are connected in series in sequence. Optionally, the larger r is, the more layers of BERT there are, and the stronger the performance of the search representation model is. The specific value of r can be determined according to actual needs. For example, r is 4, or r is 5. The input of the first search representation model and the input of the second search representation model both include training search terms, primary categories, and secondary categories, wherein the secondary categories are the secondary categories to which the training search terms belong, and the primary categories are the secondary categories to which the secondary categories belong.

[0273] After the training search terms, first-level categories, and second-level categories are input into the first search representation model, the training prompts are first segmented using a word segmenter, and then the word vector of the segmentation result is determined, and then the word vector is input into the LLM, so that the LLM outputs the representation of the training search terms (i.e., the teacher search representation). After the training search terms, first-level categories, and second-level categories are input into the second search representation model, the training prompts are first segmented using a word segmenter, and then the word vector of the segmentation result is determined, and then the word vector is input into the r-layer BERT, so that the r-layer BERT outputs the representation of the training search terms (i.e., the student search representation). Then, the parameters of the second search representation model are updated through knowledge distillation to obtain the second search representation model after the second training. Specifically, based on the teacher search representation and the student search representation, the mean square error loss and the cosine loss are determined. Then, the mean square error loss and the cosine loss are weighted summed to obtain the knowledge distillation loss. Based on the knowledge distillation loss, the parameters of the second search representation model are updated to obtain the second search representation model after the second training.

[0274] The embodiment of the present application also provides a search method, the execution subject of the search method is a search device, wherein the search device can be any electronic device that can execute the technical solution disclosed in the embodiment of the method of the present application. Optionally, the search device can be one of the following: a computer, a server. The search method is used to determine the matching content that matches the target search term from the database to be searched using a target model, wherein the target model is trained according to the first model training method described above, or the target model is trained according to the second model training method described above. The database to be searched includes the content to be searched and the content representation to be searched of the content to be searched, wherein the content representation to be searched is obtained by processing the content to be searched using the content representation model of the target model. Please refer to Figure 7 , Figure 7 A flowchart of a search method provided in an embodiment of the present application.

[0275] 701. Obtain a target search term.

[0276] In the embodiment of the present application, the target search term is the basis for the search. For example, if the target search term is food, then the information to be searched is related to food. In an implementation method of obtaining the target search term, the search device receives the target search term input by the user through the input component to obtain the target search term.

[0277] In another implementation of obtaining the target search word, the search device receives the target search word sent by the terminal to obtain the target search word.

[0278] In another implementation of obtaining the target search term, the search device obtains the target search term by performing optical character recognition (OCR) on the text and / or image. For example, the search device obtains the word in the text by performing OCR on the text as the target search term. For another example, the search device obtains the word in the image by performing OCR on the image as the target search term. For another example, the search device obtains the word in the document by performing OCR on the text and image in the document as the target search term.

[0279] 702. Input the target search term into the search representation model of the target model to obtain the target search representation of the target search term.

[0280] In this step, when the target model is trained by the second model training method, the search representation model of the target model is the second search representation model after the second training described above.

[0281] Since the search representation model of the target model is used to generate the representation of the search term, the search device inputs the target search term into the search representation model of the target model to obtain the representation of the target search term, that is, the target search representation.

[0282] 703 : Use the target search representation to search the to-be-searched database, and obtain the to-be-searched content having the to-be-searched content representation matching the target search representation as the matching content.

[0283] In an embodiment of the present application, the database to be searched includes content to be searched, wherein the content to be searched includes: text, image, video, audio. Based on the target search term, the content to be searched that matches the target search term can be determined from the database to be searched. The database to be searched also includes a representation of the content to be searched. Specifically, in the data to be searched, each content to be searched has a representation, which is the content representation to be searched. In the database to be searched, the content representation to be searched of the content to be searched is obtained using the content representation model in the target model. Optionally, by inputting the content to be searched in the database to be searched into the content representation model, the content representation to be searched of the content to be searched in the database to be searched can be obtained.

[0284] Since the target search representation is obtained through the search representation model in the target model, and the representation of the content to be searched in the database to be searched is obtained through the search content representation model in the target model, the search device can determine whether the target search representation matches the representation of the content to be searched in the database to be searched. Specifically, when the target search representation is obtained through the search representation model in the target model, and the representation of the content to be searched in the database to be searched is obtained through the search content representation model in the target model, the dimension of the target search representation is the same as the dimension of the representation of the content to be searched in the database to be searched, and the search device can further calculate the similarity between the target search representation and the representation of the content to be searched in the database to be searched, and can determine whether the target search representation matches the representation of the content to be searched in the database to be searched based on the similarity.

[0285] The target search representation matches the representation of the content to be searched in the database to be searched, indicating that the target search term corresponding to the target search representation matches the content to be searched corresponding to the representation of the content to be searched. Therefore, the search device uses the target search representation to retrieve the database to be searched, and can obtain the content to be searched having the representation of the content to be searched that matches the target search representation as the matching content. At this time, the matching content is the search result obtained by searching based on the target search term, wherein the matching content is the content that matches the target search term. Optionally, the semantics of the matching content matches the semantics of the target search term.

[0286] In the embodiment of the present application, after the search device obtains the target search word, it inputs the target search word into the search representation model of the target model, and then obtains the target search representation of the target search word output by the search representation model. Since the target search representation is obtained through the search representation model of the target model, and the representation of the content to be searched in the database to be searched is obtained through the search content representation model in the target model, the search device can determine whether the target search representation matches the representation of the content to be searched in the database to be searched. Then, the search device uses the target search representation to retrieve the database to be searched, and can obtain matching content with the representation of the content to be searched that matches the target search representation, wherein the matching content is the content that matches the target search word, that is, the matching content is the search result obtained by searching based on the target search word. Because the content representation model in the target model has strong performance, and the representation of the content to be searched in the database to be searched is obtained based on the content representation model, the accuracy of the information carried by the representation of the content to be searched in the database to be searched is high, so by using the target search representation to retrieve the database to be searched and obtaining the search result, the accuracy of the search result can be improved.

[0287] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0288] The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below.

[0289] See also Figure 8 , Figure 8 A structural diagram of a model training device provided in an embodiment of the present application. The model training device 1 is used to train a target model, the target model includes a search representation model and a content representation model, and the model training device 1 includes: an acquisition unit 11, a processing unit 12, a generation unit 13, and a training unit 14, specifically:

[0290] An acquisition unit 11 is used to acquire a training search term and training search content related to the training search term;

[0291] A processing unit 12, configured to obtain a training search representation of the training search term using the search representation model;

[0292] A generating unit 13, configured to generate a training prompt corresponding to the training search content, wherein the training prompt is used to instruct the content representation model to compress the training search content and includes at least part of the training search content;

[0293] The processing unit 12 is used to obtain the training content representation of the training search content based on the training prompt by using the content representation model;

[0294] The training unit 14 is used to train the target model based on the training search representation and the training content representation.

[0295] In combination with any embodiment of the present application, the first position in the training prompt is used to indicate the compression result of the training search content, and the processing unit 12 is specifically used to:

[0296] Inputting the training prompt into the content representation model so that the content representation model encodes the training prompt to obtain a training hidden state sequence of the training prompt;

[0297] The training content representation of the training search content is obtained based on the hidden state located at the first position in the training hidden state sequence.

[0298] In combination with any implementation manner of the present application, the training search content includes a first search content and a second search content, the training content representation includes a first content representation of the first search content and a second content representation of the second search content, the first search content is content that matches the training search term, and the second search content is content that does not match the training search term;

[0299] The training unit 14 is specifically used for:

[0300] Determining a first similarity between the training search representation and the first content representation;

[0301] determining a second similarity between the training search representation and the second content representation;

[0302] Determining a content loss of the target model based on the first similarity and the second similarity;

[0303] Parameters of the target model are updated based on the content loss.

[0304] In combination with any embodiment of the present application, the second search content includes third search content, the third search content is content that does not match the reference search term, the reference search term is a search term different from the training search term and matches the first search content, and the training content representation includes a third content representation of the third search content;

[0305] The processing unit 12 is further configured to:

[0306] determining a third similarity between the training search representation and the third content representation;

[0307] The training unit 14 is specifically used for:

[0308] Determining a content contrast loss of the target model based on the first similarity and the second similarity;

[0309] Determining a content boundary loss of the target model based on the first similarity and the third similarity;

[0310] The content loss is determined based on the content contrast loss and the content boundary loss.

[0311] In combination with any embodiment of the present application, the training prompt includes target content in the search content, the training prompt is further used to instruct the content representation model to compress the target content, and the second position in the training prompt is used to indicate the compression result of the target content;

[0312] The processing unit 12 is further used for:

[0313] Obtaining a target content representation of the target content based on the hidden state at the second position in the training hidden state sequence;

[0314] The training unit 14 is specifically used for:

[0315] Obtaining a content loss based on the training search representation and the training content representation;

[0316] Obtaining a target loss based on the target content representation and the training search representation;

[0317] Based on the content loss and the target loss, obtaining a total loss of the target model;

[0318] Parameters of the target model are updated based on the total loss.

[0319] In combination with any embodiment of the present application, the training prompt further instructs the content representation model to obtain a predicted search term based on the search content, the predicted search term being a search term that matches the training search content, and the third position in the training prompt is used to indicate the predicted search term;

[0320] The processing unit 12 is further configured to obtain a predicted search term representation of the predicted search term based on the hidden state located at the third position in the training hidden state sequence;

[0321] The training unit 14 is specifically used for:

[0322] Obtaining a content loss based on the training search representation and the training content representation;

[0323] Obtaining a search loss based on the predicted search term representation and the training search term representation;

[0324] Based on the content loss and the search loss, obtaining a total loss of the target model;

[0325] Parameters of the target model are updated based on the total loss.

[0326] In combination with any embodiment of the present application, the training prompt indicates to compress the search content based on the predicted search term;

[0327] The processing unit 12 is specifically used for:

[0328] The training content representation of the training search content is obtained based on the hidden state located at the first position and the hidden state located at the third position in the training hidden state sequence.

[0329] In combination with any embodiment of the present application, the processing unit 12 is specifically used for:

[0330] The training search term and the training category are concatenated to obtain a target search text, wherein the training category is the category to which the training search term belongs;

[0331] The target search text is input into the search representation model, and the representation output by the search representation model is obtained as the training search representation.

[0332] In an embodiment of the present application, after obtaining the training search terms and the training search content related to the training search terms, the model training device uses the search representation model in the target model to obtain the training search representation of the training search terms. Then, a training prompt corresponding to the training search content is generated, wherein the training prompt is used to instruct the content representation model to compress the training search content and include at least part of the content of the training search content. Then, the content representation model is used to obtain the training content representation of the training search content based on the training prompt. Finally, the target model can be trained based on the training search representation and the training content representation. In some embodiments, the content representation model is a language model with a relatively large scale, such as LLM, wherein the relatively large scale includes: a large number of parameters, a large amount of data required for training, and a large amount of calculation required for training, that is, the content representation model in the target model has a relatively strong performance, so generating a representation of the content based on the content representation model in the target model can improve the accuracy of the information carried by the representation.

[0333] See also Fig. 9 , Fig. 9 A schematic diagram of the structure of another model training device provided in an embodiment of the present application. The model training device 2 is used to train a target model, the target model includes a search representation model and a content representation model, and the model training device 2 includes: an acquisition unit 21, a processing unit 22, a generation unit 23, a training unit 24, and a determination unit 25. Specifically:

[0334] An acquisition unit 21, configured to acquire a training search term and training search content related to the training search term;

[0335] A processing unit 22, configured to obtain a training search representation of the training search term using the search representation model;

[0336] A generating unit 23, configured to generate a training prompt corresponding to the training search content, wherein the training prompt is used to instruct the content representation model to compress the training search content and includes at least part of the training search content;

[0337] The processing unit 22 is configured to obtain a training content representation of the training search content based on the training prompt by using the content representation model;

[0338] A training unit 24, configured to perform a first training on the target model based on the training search representation and the training content representation;

[0339] The processing unit 22 is used to use the search representation model after the first training as the first search representation model;

[0340] A determining unit 25, configured to determine a second search representation model, wherein the structure of the second search representation model is different from that of the first search representation model, or the number of parameters of the second search representation model is less than that of the first search representation model;

[0341] The training unit 24 is configured to perform a second training on the second search representation model based on a training search term and a search representation obtained by inputting the training search term into the first search representation model;

[0342] The processing unit 22 is used to use the content representation model after the first training and the second search representation model after the second training as the target model.

[0343] In an embodiment of the present application, a model training device obtains training search terms and training search content related to the training search terms. The training search representation of the training search terms is obtained using the search representation model. A training prompt corresponding to the training search content is generated. The training content representation of the training search content is obtained based on the training prompt using the content representation model. The target model is first trained based on the training search representation and the training content representation. The search representation model after the first training is used as the first search representation model. A second search representation model is determined. The second search representation model is second trained based on the training search terms and the search representation obtained by inputting the training search terms into the first search representation model. The content representation model after the first training and the second search representation model after the second training are used as the target model. The performance of the search content representation model can be avoided from being reduced due to the alignment of the semantic space of the search content representation model with the semantic space of the second search representation model, thereby improving the performance of the search content representation model in the target model, thereby improving the performance of the target model.

[0344] See also Fig.10 , Fig.10A schematic diagram of the structure of a search device provided in an embodiment of the present application. The search device 3 is used to determine matching content that matches a target search term from a database to be searched using a target model, the target model is trained according to the first aspect and any implementation method thereof, the target model is trained according to the second aspect, the database to be searched includes content to be searched and a content representation to be searched of the content to be searched, the content representation to be searched is obtained by processing the content to be searched using a content representation model of the target model, the search device 3 includes: an acquisition unit 31, a processing unit 32, a retrieval unit 33, specifically:

[0345] An acquisition unit 31, configured to acquire the target search term;

[0346] A processing unit 32 is used to input the target search term into the search representation model of the target model to obtain a target search representation of the target search term;

[0347] The retrieval unit 33 is configured to use the target search representation to retrieve the to-be-searched database, and obtain the to-be-searched content having the to-be-searched content representation matching the target search representation as the matching content.

[0348] In the embodiment of the present application, after the search device obtains the target search word, it inputs the target search word into the search representation model of the target model, and then obtains the target search representation of the target search word output by the search representation model. Since the target search representation is obtained through the search representation model of the target model, and the representation of the content to be searched in the database to be searched is obtained through the search content representation model in the target model, the search device can determine whether the target search representation matches the representation of the content to be searched in the database to be searched. Then, the search device uses the target search representation to retrieve the database to be searched, and can obtain matching content with the representation of the content to be searched that matches the target search representation, wherein the matching content is the content that matches the target search word, that is, the matching content is the search result obtained by searching based on the target search word. Because the content representation model in the target model has strong performance, and the representation of the content to be searched in the database to be searched is obtained based on the content representation model, the accuracy of the information carried by the representation of the content to be searched in the database to be searched is high, so by using the target search representation to retrieve the database to be searched and obtaining the search result, the accuracy of the search result can be improved.

[0349] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0350] Fig.11A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device 4 includes a processor 41 and a memory 42. Optionally, the electronic device 4 also includes an input device 43 and an output device 44. The processor 41, the memory 42, the input device 43 and the output device 44 are coupled via a connector, and the connector includes various interfaces, transmission lines or buses, etc., which are not limited in the embodiments of the present application. It should be understood that in each embodiment of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, it can be connected through various interfaces, transmission lines, buses, etc.

[0351] The processor 41 may include one or more processors, for example, one or more central processing units (CPUs). When the processor is a CPU, the CPU may be a single-core CPU or a multi-core CPU. Optionally, the processor 41 may be a processor group consisting of multiple CPUs, and the multiple processors are coupled to each other through one or more buses. Optionally, the processor may also be other types of processors, etc., which are not limited in the embodiments of the present application.

[0352] The memory 42 can be used to store computer program instructions and various computer program codes including the program code for executing the program code of the present application. Optionally, the memory includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM), or portable read only memory (CD-ROM), which is used for related instructions and data.

[0353] The input device 43 is used to input data and / or signals, and the output device 44 is used to output data and / or signals. The input device 43 and the output device 44 can be independent devices or an integrated device.

[0354] It can be understood that in the embodiment of the present application, the memory 42 can be used not only to store relevant instructions, but also to store relevant data. The embodiment of the present application does not limit the specific data stored in the memory.

[0355] Understandably, Fig.11Only a simplified design of an electronic device is shown. In practical applications, the electronic device may also include other necessary components, including but not limited to any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of the present application are within the protection scope of the present application.

[0356] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0357] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. Those skilled in the art can also clearly understand that the descriptions of the various embodiments of the present application have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, refer to the records of other embodiments.

[0358] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0359] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0360] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0361] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0362] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by a computer program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: a read-only memory (ROM) or a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

Claims

1. A model training method, characterized in that: For training a target model, the target model includes a search representation model and a content representation model, and the method includes: Acquire a training search term and training search content related to the training search term; Obtaining a training search representation of the training search term using the search representation model; generating a training prompt corresponding to the training search content, wherein the training prompt is used to instruct the content representation model to compress the training search content and includes at least part of the training search content; Using the content representation model, obtaining a training content representation of the training search content based on the training prompt; The target model is trained based on the training search representation and the training content representation.

2. The method according to claim 1, characterized in that The first position in the training prompt is used to indicate the compression result of the training search content, and the using the content representation model to obtain the training content representation of the training search content based on the training prompt includes: Inputting the training prompt into the content representation model so that the content representation model encodes the training prompt to obtain a training hidden state sequence of the training prompt; The training content representation of the training search content is obtained based on the hidden state located at the first position in the training hidden state sequence.

3. The method according to claim 1 or 2, characterized in that: The training search content includes a first search content and a second search content, the training content representation includes a first content representation of the first search content and a second content representation of the second search content, the first search content is content that matches the training search term, and the second search content is content that does not match the training search term; The training of the target model based on the training search representation and the training content representation includes: Determining a first similarity between the training search representation and the first content representation; determining a second similarity between the training search representation and the second content representation; Determining a content loss of the target model based on the first similarity and the second similarity; Parameters of the target model are updated based on the content loss.

4. The method according to claim 3, characterized in that The second search content includes third search content, the third search content is content that does not match the reference search term, the reference search term is a search term different from the training search term and matches the first search content, and the training content representation includes a third content representation of the third search content; Before determining the content loss of the target model based on the first similarity and the second similarity, the method further includes: determining a third similarity between the training search representation and the third content representation; The determining the content loss of the target model based on the first similarity and the second similarity includes: Determining a content contrast loss of the target model based on the first similarity and the second similarity; Determining a content boundary loss of the target model based on the first similarity and the third similarity; The content loss is determined based on the content contrast loss and the content boundary loss.

5. The method according to claim 2, characterized in that: The training prompt includes target content in the search content, and the training prompt is further used to instruct the content representation model to compress the target content, and the second position in the training prompt is used to indicate the compression result of the target content; After obtaining the training hidden state sequence, the method further includes: Obtaining a target content representation of the target content based on the hidden state at the second position in the training hidden state sequence; The training of the target model based on the training search representation and the training content representation includes: Obtaining a content loss based on the training search representation and the training content representation; Obtaining a target loss based on the target content representation and the training search representation; Based on the content loss and the target loss, obtaining a total loss of the target model; Parameters of the target model are updated based on the total loss.

6. The method according to claim 2, characterized in that The training prompt further instructs the content representation model to obtain a predicted search term based on the search content, the predicted search term being a search term that matches the training search content, and the third position in the training prompt is used to indicate the predicted search term; After obtaining the training hidden state sequence, the method further includes: Obtaining a predicted search term representation of the predicted search term based on the hidden state at the third position in the training hidden state sequence; The training of the target model based on the training search representation and the training content representation includes: Obtaining a content loss based on the training search representation and the training content representation; Obtaining a search loss based on the predicted search term representation and the training search term representation; Based on the content loss and the search loss, obtaining a total loss of the target model; Parameters of the target model are updated based on the total loss.

7. The method according to claim 6, characterized in that The training prompt indicates to compress the search content based on the predicted search term; The obtaining the training content representation of the training search content based on the hidden state located at the first position in the training hidden state sequence includes: The training content representation of the training search content is obtained based on the hidden state located at the first position and the hidden state located at the third position in the training hidden state sequence.

8. The method according to claim 1 or 2, characterized in that: The step of obtaining the training search representation of the training search term by using the search representation model includes: The training search term and the training category are concatenated to obtain a target search text, wherein the training category is the category to which the training search term belongs; The target search text is input into the search representation model, and the representation output by the search representation model is obtained as the training search representation.

9. A model training method, characterized in that: For training a target model, the target model includes a search representation model and a content representation model, and the method includes: Acquire a training search term and training search content related to the training search term; Obtaining a training search representation of the training search term using the search representation model; generating a training prompt corresponding to the training search content, wherein the training prompt is used to instruct the content representation model to compress the training search content and includes at least part of the training search content; Using the content representation model, obtaining a training content representation of the training search content based on the training prompt; Based on the training search representation and the training content representation, performing a first training on the target model; The search representation model after the first training is used as the first search representation model; Determining a second search representation model, wherein the structure of the second search representation model is different from the structure of the first search representation model, or the number of parameters of the second search representation model is less than the number of parameters of the first search representation model; performing a second training on the second search representation model based on the training search terms and the search representations obtained by inputting the training search terms into the first search representation model; The content representation model after the first training and the second search representation model after the second training are used as target models.

10. A search method, characterized in that: The method is used to determine matching content that matches a target search term from a database to be searched using a target model, wherein the target model is obtained by training according to the method according to any one of claims 1 to 9, the database to be searched includes content to be searched and content representations to be searched of the content to be searched, and the content representations to be searched are obtained by processing the content to be searched using a content representation model of the target model, and the method comprises: Obtaining the target search term; Inputting the target search term into the search representation model of the target model to obtain the target search representation of the target search term; The target search representation is used to search a database to be searched, and the content to be searched having the content representation to be searched that matches the target search representation is obtained as the matching content.

11. A model training device, characterized in that: Used to train a target model, the target model includes a search representation model and a content representation model, and the model training device includes: An acquisition unit, used for acquiring a training search word and training search content related to the training search word; A processing unit, configured to obtain a training search representation of the training search term using the search representation model; A generating unit, configured to generate a training prompt corresponding to the training search content, wherein the training prompt is used to instruct the content representation model to compress the training search content and includes at least part of the training search content; The processing unit is configured to obtain a training content representation of the training search content based on the training prompt by using the content representation model; A training unit is used to train the target model based on the training search representation and the training content representation.

12. A model training device, characterized in that: Used to train a target model, the target model includes a search representation model and a content representation model, and the model training device includes: An acquisition unit, used for acquiring a training search word and training search content related to the training search word; A processing unit, configured to obtain a training search representation of the training search term using the search representation model; A generating unit, configured to generate a training prompt corresponding to the training search content, wherein the training prompt is used to instruct the content representation model to compress the training search content and includes at least part of the training search content; The processing unit is configured to obtain a training content representation of the training search content based on the training prompt by using the content representation model; A training unit, configured to perform a first training on the target model based on the training search representation and the training content representation; The processing unit is used to use the search representation model after the first training as the first search representation model; a determining unit, configured to determine a second search representation model, wherein the structure of the second search representation model is different from that of the first search representation model, or the number of parameters of the second search representation model is less than that of the first search representation model; The training unit is configured to perform a second training on the second search representation model based on a training search term and a search representation obtained by inputting the training search term into the first search representation model; The processing unit is used to use the content representation model after the first training and the second search representation model after the second training as the target model.

13. A search device, characterized in that: The search device is used to determine matching content that matches a target search term from a database to be searched using a target model, wherein the target model is obtained by training according to the method according to any one of claims 1 to 9, the database to be searched includes content to be searched and content representations to be searched of the content to be searched, and the content representations to be searched are obtained by processing the content to be searched using a content representation model of the target model, and the search device includes: An acquisition unit, used for acquiring the target search term; A processing unit, configured to input the target search term into a search representation model of the target model to obtain a target search representation of the target search term; The retrieval unit is used to use the target search representation to retrieve the to-be-searched database, and obtain the to-be-searched content having the to-be-searched content representation matching the target search representation as the matching content.

14. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store computer program code, wherein the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method according to any one of claims 1 to 8; when the processor executes the computer instructions, the electronic device either executes the method according to claim 9; when the processor executes the computer instructions, the electronic device either executes the method according to claim 10.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method of any one of claims 1 to 8; when the program instructions are executed by the processor, the processor is caused to execute the method of claim 9; when the program instructions are executed by the processor, the processor is caused to execute the method of claim 10.

16. A computer program product, characterized in that The computer program product includes a computer program or instructions; when the computer program or instructions are run on a computer, the computer is caused to execute the method described in any one of claims 1 to 8; when the computer program or instructions are run on a computer, the computer is caused to execute the method described in claim 9; when the computer program or instructions are run on a computer, the computer is caused to execute the method described in claim 10.

Images