Search relevance model training method, relevance evaluation method, and related apparatuses

Abstract

The application provides a training method of a search relevance model, a relevance evaluation method and related devices. The method comprises: inputting a sample into a plurality of teacher models to obtain a basic relevance score output by each teacher model; inputting the sample, a search term type and a teacher model scoring confidence into a gating model to dynamically obtain the weight of different teacher models; weighting and fusing the basic relevance scores output by the plurality of teacher models based on the weight output by the gating model to obtain a target relevance score; inputting the sample into a student model to train the student model with the target relevance score as a target; and adding an entity extraction auxiliary task to the teacher model and extracting key attributes by using an external knowledge graph, and improving the attention weight of the key attributes through a post-interaction link when training the student model. The student model serves as an online search relevance model. The application can improve the relevance discrimination ability of search terms and commodities in a search system to a certain extent.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for training a search relevance model, a relevance evaluation method, and related apparatus. Background Technology

[0002] In e-commerce search scenarios, search relevance models are typically used to determine the relevance between products and user needs based on search terms and product information, so as to support the subsequent search for products that can adequately meet user needs.

[0003] However, the relevant technologies still have the problem of low relevance of product information displayed under the corresponding search terms due to inaccurate judgment of the semantic relevance between search terms and products. Summary of the Invention

[0004] In view of this, one or more embodiments of this application provide a method for training a search relevance model, a method for evaluating relevance, and a related apparatus, which can improve the ability to distinguish the relevance between search terms and products to a certain extent.

[0005] In a first aspect, one or more embodiments of this application propose a method for training a search relevance model, comprising: inputting samples into multiple teacher models to obtain a basic relevance score output by each teacher model; at least some of the multiple teacher models are trained based on different model foundations; the samples include search terms and product information; inputting the samples into a gating model to obtain weights corresponding to each teacher model; wherein, the gating model identifies the dominant model among the multiple teacher models and assigns a weight greater than the weight assigned to the non-dominant model; performing a weighted fusion based on the basic relevance scores output by the multiple teacher models and the weights output by the gating model to obtain a target relevance score; inputting the samples into a student model and training the student model with the target relevance as the objective; wherein, the student model serves as the search relevance model.

[0006] Secondly, one or more embodiments of this application propose a relevance evaluation method, including: inputting the acquired search terms and product information into a search relevance model obtained by the aforementioned method, and obtaining a relevance score between the search terms and product information.

[0007] Thirdly, one or more embodiments of this application propose a training apparatus for a search relevance model, comprising: a first input module for inputting samples into multiple teacher models to obtain a basic relevance score output by each teacher model; at least some of the multiple teacher models are trained based on different model foundations; the samples include search terms and product information; a second input module for inputting the samples into a gating model to obtain a weight corresponding to each teacher model; wherein the gating model identifies a dominant model among the multiple teacher models and assigns a weight greater than the weight assigned to a non-dominant model; a fusion module for performing weighted fusion based on the basic relevance scores output by the multiple teacher models and the weights output by the gating model to obtain a target relevance score; and a training module for inputting the samples into a student model and training the student model with the target relevance as the objective; wherein the student model serves as the search relevance model.

[0008] Fourthly, one or more embodiments of this application propose a relevance evaluation device for inputting search terms and product information into a search relevance model obtained by the aforementioned method, and obtaining a relevance score between the search terms and product information.

[0009] Fifthly, one or more embodiments of this application provide a computer device including a memory and a processor, wherein the memory stores at least one computer program, which is loaded and executed by the processor to implement the method as described above.

[0010] In a sixth aspect, one or more embodiments of this application provide a computer program product including computer instructions that, when executed by a processor, implement the method as described above.

[0011] In a seventh aspect, one or more embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the processor to implement the method as described above.

[0012] As can be seen from the above embodiments, in this application, multiple embodiments obtain basic relevance scores by inputting samples including search terms and product information into multiple teacher models trained based on different model foundations, and inputting the samples into a gating model to identify the dominant model among the multiple teacher models and output corresponding weights. Then, the basic relevance scores are weighted and fused based on the weights to obtain a target relevance score, and the student model is trained with the target relevance score as the training target. This achieves the advantage-selective fusion of the relevance discrimination ability of multiple teacher models and distillation into the student model, thereby improving the relevance of search terms and products in different search scenarios. Attached Figure Description

[0013] Figure 1 This is a schematic diagram illustrating the training process of a search relevance model training method provided in one embodiment of this application.

[0014] Figure 2 This is a flowchart of a training method for a search relevance model provided in one embodiment of this application.

[0015] Figure 3 This is a schematic diagram of a training module for a search relevance model provided in one embodiment of this application.

[0016] Figure 4 This is a schematic diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments.

[0018] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0019] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0020] In e-commerce search scenarios, search relevance models typically categorize products based on the relevance between search terms and product information to support subsequent product ranking and display. With the increasing diversity of search scenarios and the continuous growth in the number of products, user search terms exhibit significant differences in expression, semantic focus, and specific needs, placing higher demands on relevance assessment capabilities.

[0021] In practice, search relevance models often rely on a single model or fixed strategy to model the relevance between search terms and product information. While this approach can meet basic needs in some scenarios, due to the limited capabilities of search relevance models or their inability to adapt to the semantic differences in different search scenarios, it can easily lead to incorrect evaluations of product relevance for certain search terms. This results in a low degree of matching between the displayed product information and the user's actual needs, affecting the overall user search experience.

[0022] Therefore, the relevant technologies still suffer from insufficient understanding of the relevance between search terms and products, resulting in a low degree of matching between the product information displayed in the search results and the user's actual needs.

[0023] In several embodiments provided in this application, the training method and relevance evaluation method for the search relevance model can be applied to electronic devices with certain computing power and network access capabilities. This electronic device can be a desktop computer, laptop computer, tablet computer, smartphone, or a server. Specifically, the electronic device includes a processor, memory, and a network access module for network communication. The server can be an electronic device with strong data processing capabilities; of course, a server can also refer to a server cluster formed by multiple electronic devices, or a quantum server built using a quantum computer.

[0024] Please see Figure 1 One embodiment of this application provides an application scenario example of a search relevance model training method and a product relevance judgment method. This application scenario example can be applied to an internet medical e-commerce platform to judge and categorize products based on relevance when a user enters search terms, in order to support subsequent product sorting and display.

[0025] In this scenario example, the e-commerce platform deploys a training device for the search relevance model and a relevance evaluation device. During operation, the platform accumulates a large amount of historical search data as training samples. Each training sample includes the user's search term and corresponding product information. For example, if a user enters the search term "children's fever reducer" on the platform, candidate product information may include product titles, product descriptions, product images, and product attribute information.

[0026] During the training phase, the training device can input the sample into multiple teacher models to obtain the basic relevance scores output by each teacher model. In this example scenario, the multiple teacher models include a teacher model built based on the BERT model, a teacher model built based on the multimodal BERT model, and a teacher model built based on the large language model. For the search term "children's fever reducer" and a specific product information, each teacher model can output the following basic relevance scores: the teacher model built based on the BERT model outputs 0.72, the teacher model built based on the multimodal BERT model outputs 0.81, and the teacher model built based on the large language model outputs 0.76.

[0027] In this example scenario, each teacher model can output the corresponding result confidence score along with the basic relevance score. For instance, the teacher model built on the BERT model outputs a result confidence score of 0.65, the teacher model built on the multimodal BERT model outputs a result confidence score of 0.88, and the teacher model built on the large language model outputs a result confidence score of 0.70. The training device can input search terms, product information, and result confidence scores into the gating model together. In the result confidence-based approach, the gating model can assign weights to the three components separately, such as 0.3, 0.5, and 0.2.

[0028] In another scenario, gating models can also determine weights based on label consistency learned during the pre-training phase. For example, in historical training data, for search samples with the same or similar labels as "children's fever reducer," the teacher model built on the BERT model has consistent judgments with the training labels in 82 out of 100 samples, the teacher model built on the multimodal BERT model has consistent judgments in 75 samples, and the teacher model built on the large language model has consistent judgments in 68 samples. In this case, the gating model can assign weights to the three based on the consistency results mentioned above, for example, 0.4, 0.35, and 0.25 respectively.

[0029] In this example scenario, search terms can also have type labels. For example, "children's fever reducer" can be labeled as a "medication-related search term." Historical data from the platform shows a correlation between different teacher models and different search types. For instance, in the "medication-related search term" scenario, the teacher model built on a large language model performs more stably in understanding semantics such as applicable population and ingredient restrictions. Based on this correlation, the gating model can assign weights to the three types separately, for example, 0.25, 0.25, and 0.5, using type labels.

[0030] In this example scenario, the gating model can combine the weights obtained through the different methods described above to obtain the target weights for each teacher model. For instance, the target weight for the teacher model built based on the BERT model can be 0.3 × 0.4 × 0.25 = 0.03, the target weight for the teacher model built based on the multimodal BERT model can be 0.5 × 0.35 × 0.25 = 0.04375, and the target weight for the teacher model built based on the large language model can be 0.2 × 0.25 × 0.5 = 0.025. The training device can normalize these target weights to obtain the weight ratios used for weighted fusion, for example, 0.304, 0.443, and 0.253, respectively.

[0031] Subsequently, the training device can perform weighted fusion of the basic relevance scores output by each teacher model based on the normalized target weights. For example, in the above example, the target relevance score can be calculated as: 0.72×0.304 +0.81×0.443 + 0.76×0.253 ≈ 0.776. This target relevance score is used as the result of multiple teacher models jointly judging the relevance between the search term and the product information. In some implementations, the training device can also select a set of weights from the weights allocated by the aforementioned multiple advantage model selection methods and perform weighted fusion to obtain the target relevance score.

[0032] In further training, the training device inputs samples into the student model and trains the model using the target relevance score as the training objective, so that the student model's predicted relevance score is close to the target relevance score. In this scenario example, the student model adopts a dual-tower model structure, encoding the search terms and product information separately, and performing interactive modeling through the post-interaction module after encoding is completed.

[0033] In this scenario example, at least one teacher model can also identify entity words in the samples during the sample processing process. For example, it can identify entity words such as "children," "fever reducer," and "acetaminophen" from search terms and product information. The training device further obtains the attribute information corresponding to the above entity words from a specified health knowledge graph, where attributes such as "applicable population" and "drug ingredients" are used as key attributes, and entity words corresponding to these key attributes are identified as key entity words.

[0034] The training device provides key entity words to the post-interaction module of the student model, so that when the post-interaction module generates codeword weights in search terms and product information based on the bidirectional attention mechanism, it assigns higher weights to codewords corresponding to key entity words than to codewords corresponding to non-key entity words. This guides the student model to pay more attention to semantic information that plays a major role in relevance discrimination during the training process.

[0035] After the model training is completed, the e-commerce platform deploys the trained student model as a search relevance model in the relevance evaluation device. When a user enters a search term on the platform, the relevance evaluation device can obtain the search term and candidate product information, and input the search term and product information into the search relevance model, outputting a relevance score between the search term and the product information in the product information set.

[0036] Please see Figure 1 and Figure 2 One embodiment of this application provides a method for training a search relevance model. This method can be applied to a training device, which can be an electronic device with certain computing power and network access capabilities. Of course, in some embodiments, the training device can also be software running on the electronic device. The method for training the search relevance model may include the following steps.

[0037] Step S110: Input the sample into multiple teacher models to obtain the basic relevance score output by each teacher model; at least some of the multiple teacher models are trained based on different model bases; the sample includes search terms and product information.

[0038] Step S120: Input the sample into the gating model to obtain the weight of each teacher model; wherein, the gating model identifies the dominant model among the multiple teacher models and assigns a greater weight to the dominant model than to the non-dominant model.

[0039] Step S130: Based on the basic relevance scores output by the multiple teacher models, the target relevance score is obtained by weighted fusion based on the weights output by the gating model.

[0040] Step S140: Input the sample into the student model and train the student model with the target relevance as the objective; wherein the student model serves as the search relevance model.

[0041] In this embodiment, the training device acquires samples for model training and inputs these samples into multiple teacher models. The samples are data units used for model training, and they include search terms and product information. The search terms represent the user's search needs, and the product information represents the description of the products.

[0042] In this embodiment, the teacher model is used to determine the relevance between search terms and product information. At least some of the multiple teacher models are trained on different model foundations, resulting in differences in semantic modeling methods or applicable search scenarios. By inputting the sample into the multiple teacher models, the training device obtains a basic relevance score output by each teacher model for the sample. The basic relevance score refers to the score calculated by the corresponding teacher model based on the sample, which characterizes the degree of relevance between the search term and the product information.

[0043] After obtaining the basic relevance score, the training device inputs the samples into a gating model to determine the weights for each teacher model. The gating model identifies the teacher models that are superior to the sample among the multiple teacher models, assigning them a higher weight than that assigned to non-superior models. The superior model refers to the teacher model that has a relatively better ability to distinguish the relevance between search terms and product information in the current sample. By assigning different weights to each teacher model through the gating model, the teacher model with better discriminative ability can occupy a higher weight ratio in the subsequent fusion process.

[0044] In this embodiment, the training device performs a weighted fusion of the basic relevance scores output by the multiple teacher models and the weights output by the gating model to obtain a target relevance score. This target relevance score comprehensively reflects the target relevance level between the search term and product information under the joint evaluation of the multiple teacher models, thereby avoiding the problem of insufficient adaptability to different search scenarios when using a fixed fusion strategy.

[0045] After obtaining the target relevance score, the training device inputs the sample into the student model and trains the student model using the target relevance score as the training objective, so that the student model's predicted relevance score is close to the target relevance score. The student model learns the relevance discrimination rules formed under the joint guidance of multiple teacher models, enabling it to inherit the relevance discrimination capabilities of multiple teacher models. In this embodiment, the trained student model serves as a search relevance model, used to evaluate the relevance between search terms and product information.

[0046] In some implementations, the training device can obtain the result confidence scores of the multiple teacher models corresponding to the sample output; wherein, the result confidence scores are used to represent the degree of confidence of the corresponding teacher model in the basic relevance score of the output; the training device can input the result confidence scores along with the sample into the gating model to obtain the weights corresponding to each teacher model; wherein, the gating model uses the teacher model with the specified result confidence scores among the multiple teacher models as the dominant model.

[0047] In this embodiment, the training device can also obtain the confidence scores of the outputs of the multiple teacher models corresponding to the samples. The confidence scores represent the degree of confidence that the corresponding teacher model has in its basic relevance score for its output. When a teacher model outputs a basic relevance score for a sample, it can also output the corresponding confidence scores.

[0048] In this embodiment, when the training device inputs the samples into the gating model to determine the weights of each teacher model, it can input the result confidence level along with the samples into the gating model. The gating model, based on the search terms and product information features represented by the samples and combined with the result confidence level, comprehensively evaluates the discriminative reliability of the multiple teacher models under the current samples, thereby obtaining the weights of each teacher model.

[0049] In this embodiment, a teacher model with a specified result confidence level can refer to a teacher model whose result confidence level is higher than that of other teacher models. Of course, in other embodiments, a teacher model with a specified result confidence level can also refer to a teacher model whose result confidence level is higher than a specified confidence level threshold.

[0050] Specifically, the gating model identifies the teacher model with a specified result confidence level among the multiple teacher models as the dominant model, and assigns a greater weight to the dominant model than to the non-dominant models. By incorporating the result confidence level into the input of the gating model, it can prioritize the output of the teacher model with higher confidence level when multiple teacher models give different baseline relevance scores for the same sample. This achieves accurate selection of dominant teacher models at the sample granularity and provides a reliable basis for subsequent weighted fusion of baseline relevance scores.

[0051] In some implementations, the gating model selects a dominant model from among the plurality of teacher models based on the label consistency learned during pre-training; wherein, the gating model identifies a teacher model that has a specified label consistency for the sample from among the plurality of teacher models, and that teacher model is selected as the dominant model.

[0052] In this embodiment, the gating model can also select the dominant model among the multiple teacher models for the sample based on the label consistency learned during pre-training. The label consistency represents the degree of consistency between the basic relevance score output by the corresponding teacher model for the sample and the label corresponding to the sample, thereby characterizing the discriminative reliability of the corresponding teacher model on the sample. During the pre-training phase, the gating model can learn the label consistency features of the multiple teacher models under different samples based on the label information of the training samples, enabling the gating model to refer to the label consistency in the subsequent weight determination process.

[0053] In this embodiment, when the training device inputs the sample into the gating model to determine the weight of each teacher model, the gating model can evaluate the discrimination reliability of the multiple teacher models under the current sample based on the label consistency learned in the pre-training, thereby obtaining the weight of each teacher model.

[0054] In this embodiment, a teacher model with specified label consistency can refer to a teacher model whose label consistency is higher than that of other teacher models. Of course, in other embodiments, a teacher model with specified label consistency can also refer to a teacher model whose label consistency is higher than a specified label consistency threshold.

[0055] Specifically, the gating model identifies teacher models from among the multiple teacher models that demonstrate consistent labeling for the sample. These teacher models are designated as the dominant models, and their weights are greater than those assigned to non-dominant models. By introducing a dominant model selection method based on label consistency, the gating model can prioritize teacher model outputs that are more consistent with the labels at the sample granularity level, thus providing a reliable basis for subsequent weighted fusion of basic relevance scores.

[0056] In some implementations, the search terms in the samples have type labels; the training device can input the samples and the type labels together into the gating model to obtain the weights corresponding to each teacher model; wherein, the gating model selects the teacher model among the multiple teacher models that has a specified association with the type label as the dominant model.

[0057] In this embodiment, the search terms in the samples may also have type tags. The type tags are used to indicate the search type or semantic category to which the search term belongs, so as to reflect the differences in the expression, semantic focus, or applicable search scenarios of the search terms.

[0058] In this embodiment, when the training device inputs the samples into the gating model to determine the weights of each teacher model, it can input the samples and the type labels together into the gating model. The gating model evaluates the suitability of the multiple teacher models for the corresponding search type based on the search terms and product information features represented by the samples, combined with the type labels, thereby obtaining the weights of each teacher model.

[0059] Specifically, the gating model identifies teacher models among the multiple teacher models that have a specified association with the type label as the dominant models, and assigns a greater weight to the dominant models than to the non-dominant models. The specified association indicates that the corresponding teacher model has a higher relevance discrimination ability under the corresponding search type.

[0060] In this embodiment, at least some of the multiple teacher models can be trained based on different model foundations, thus resulting in differences in semantic modeling methods or feature focuses among the different teacher models. For example, the multiple teacher models may include: a teacher model built based on the BERT model, a teacher model built based on a multimodal BERT model, and a teacher model built based on a large language model. The teacher model built based on the BERT model can be used to determine the semantic relevance between search terms and text content in product information; the teacher model built based on the multimodal BERT model can be used to determine relevance by combining multimodal information such as text and images in product information; and the teacher model built based on the large language model can be used to understand the complex semantics, colloquial expressions, or long-tail expressions of search terms and output corresponding relevance determination results.

[0061] In this implementation, samples corresponding to different search terms may differ in their expression and semantic features. For example, some samples primarily express the user's search intent in text form, while others may contain product information combining text and images, or text content with obvious domain characteristics or professional terminology. Correspondingly, different teacher models differ in model structure, training data sources, or feature modeling methods, resulting in different relevance discrimination advantages for each teacher model when processing samples corresponding to different types of tags.

[0062] For example, for search types primarily based on text semantic understanding, a teacher model built on the BERT model may be more advantageous in determining the relevance between search terms and product information; for search types including image and text information, a teacher model built on a multimodal BERT model may be more accurate in understanding the relationship between search terms and product information; for search types containing colloquial expressions, strong inference components, or long-tail expressions, a teacher model built on a large language model may be more advantageous in understanding the semantics of search terms and determining relevance. By introducing the aforementioned type labels into the gating model, the gating model can differentiate the suitability of the multiple teacher models based on the type to which the search term belongs, thereby dynamically selecting the teacher model that is better at handling that type of sample as the superior model under different type labels.

[0063] In some implementations, the training device may use at least one teacher model from the plurality of teacher models to identify entity words in the sample; determine key entity words among the identified entity words; wherein the key entity words are provided to the post-interaction module of the student model so that when the post-interaction module generates the weights of the search terms and codewords in the product information based on the bidirectional attention mechanism, the weight assigned to the key entity words is greater than the weight of the non-key entity words.

[0064] In this embodiment, the training device can also use at least one of the multiple teacher models to identify entity words in the samples. Entity words represent word units with clear semantic meaning in the search term or product information, such as words used to characterize product attributes, technical terms, or domain concepts. The teacher model can simultaneously perform entity recognition tasks while judging the relevance of the samples, thereby obtaining the corresponding entity words in the samples.

[0065] In this embodiment, the training device can further identify key entity words among the identified entity words. Key entity words refer to entity words that play a major role in determining the relevance between search terms and product information, and are used to represent semantic information that contributes more to the relevance judgment. By distinguishing between key and non-key entity words, entity words that contribute less to the relevance judgment can be avoided as the main reference, thereby reducing the impact of noise information on model training.

[0066] In this embodiment, the student model can adopt a dual-tower model structure. The dual-tower model includes a query encoding tower for encoding search terms and a product encoding tower for encoding product information. The query encoding tower and the product encoding tower independently encode the input search terms and product information respectively to obtain corresponding codeword representations. By adopting a dual-tower model structure, the semantic features of search terms and product information can be modeled separately while ensuring encoding efficiency.

[0067] In this embodiment, the key entity words can be provided to the post-interaction module of the student model. The post-interaction module is set up after the query coding tower and the product coding tower have completed independent encoding, and is used to introduce cross-tower interactive computation between the encoding results of the dual-tower model. Based on a bidirectional attention mechanism, the post-interaction module performs association modeling on the search terms and the corresponding codewords in the product information to generate weights corresponding to each codeword.

[0068] Specifically, in the process of generating codeword weights based on the bidirectional attention mechanism, the post-interaction module assigns greater weight to codewords corresponding to key entity words than to codewords corresponding to non-key entity words. By assigning higher attention weights to key entity words, the student model can focus more on semantic units that play a major role in relevance discrimination during the post-interaction stage, thereby guiding the student model to learn the teacher model's focus on key semantic information discrimination during training.

[0069] In this embodiment, based on the dual-tower model structure, the student model can combine the key entity words identified by the teacher model in the post-interaction module to focus on modeling the key semantic relationship between search terms and product information. This improves the student model's ability to understand the key information required for relevance judgment without increasing the complexity of the dual-tower model itself.

[0070] In some implementations, the training device can derive the attributes corresponding to identified entity words from a specified health knowledge graph; wherein, some entity words correspond to key attributes; and entity words corresponding to the key attributes are designated as key entity words.

[0071] In this embodiment, after identifying entity words in the sample, the training device can further determine key entity words among them. To achieve accurate determination of key entity words, the training device can utilize a specified health knowledge graph to perform attribute parsing on the identified entity words.

[0072] The designated health knowledge graph is used to store structured knowledge information related to the health domain, including entities and the relationships between entities, as well as the attribute information corresponding to the entities. The attributes are used to describe the semantic role or transaction meaning of entity words in the health domain, such as semantic features representing generic drug names, ingredients, disease names, indications, or core attributes of products.

[0073] In this embodiment, the training device can obtain attribute information corresponding to identified entity words from the specified health knowledge graph. Some of the attributes corresponding to entity words are pre-defined or marked as key attributes. These key attributes represent attribute types that have a high semantic contribution to the relevance determination between search terms and product information.

[0074] In this embodiment, the training device identifies the entity words corresponding to the key attributes as the key entity words. By utilizing the specified health knowledge graph to determine the attributes of the entity words, the determination process of key entity words no longer relies solely on statistical features or contextual information, but incorporates prior knowledge in the health domain, thereby improving the accuracy and stability of key entity word recognition.

[0075] This application also provides a relevance evaluation method. The relevance evaluation method can be applied to a relevance evaluation device, which can be applied to the aforementioned electronic device possessing certain computing power and network access capabilities. Of course, in some embodiments, the relevance evaluation device can also be software running on the electronic device. The relevance evaluation method includes: inputting the acquired search terms and product information into a search relevance model obtained by the aforementioned method, and obtaining a relevance score between the search terms and the product information.

[0076] In this embodiment, the relevance evaluation method can be executed by a relevance evaluation device. The relevance evaluation device is used in e-commerce scenarios to output a relevance score for the product information corresponding to the user-input search terms, indicating the relevance between the search terms and the products.

[0077] In this embodiment, the relevance evaluation device can acquire search terms and product information, and input the search terms and product information into a search relevance model trained by the aforementioned training method. The search relevance model is a model obtained through joint guidance from multiple teacher models and learning by student models, and has already learned the relevance relationship between search terms and product information.

[0078] Through the above methods, the relevance evaluation device can efficiently evaluate the relevance between search terms and product information based on the trained search relevance model.

[0079] Please see Figure 3This application also provides a training apparatus for a search relevance model. The training apparatus includes a first input module, a second input module, a fusion module, and a training module.

[0080] The first input module is used to input samples into multiple teacher models and obtain the basic relevance score output by each teacher model; at least some of the multiple teacher models are trained based on different model bases; the samples include search terms and product information.

[0081] The second input module is used to input the sample into the gating model to obtain the weight of each teacher model; wherein, the gating model identifies the dominant model among the multiple teacher models and assigns a greater weight to the dominant model than to the non-dominant model.

[0082] The fusion module is used to perform weighted fusion based on the basic relevance scores output by the multiple teacher models and the weights output by the gating model to obtain the target relevance score.

[0083] The training module is used to input the samples into the student model and train the student model with the target relevance as the objective; wherein the student model serves as the search relevance model.

[0084] In this embodiment, the functions and effects achieved by the training device can be explained in comparison with the aforementioned embodiments, and will not be repeated here.

[0085] One or more embodiments of this application propose a relevance evaluation device for inputting acquired search terms and product information into a search relevance model obtained by the aforementioned method, and obtaining a relevance score between the search terms and product information.

[0086] In this embodiment, the functions and effects of the correlation evaluation device can be explained in comparison with the aforementioned embodiments, and will not be repeated here.

[0087] Please see Figure 4 This application also provides a computer device comprising: a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the method described above.

[0088] The memory, processor, and communication interface in the computer device can communicate with each other via the system bus and network communication.

[0089] In this embodiment, the functions and effects implemented by the computer device can be explained by referring to the foregoing embodiments, and will not be repeated here.

[0090] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the processor to implement the method as described above.

[0091] The functions and effects achieved in this embodiment can be explained by referring to other embodiments, and will not be repeated here.

[0092] This application also provides a computer program product containing instructions, including a computer program / instructions that, when executed by a processor, implement the method as described above.

[0093] The functions and effects achieved in this embodiment can be explained by referring to other embodiments, and will not be repeated here.

[0094] It is understood that the specific examples in this document are only intended to help those skilled in the art better understand the embodiments of this application, and are not intended to limit the scope of the invention.

[0095] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0096] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and the implementation methods in this application are not limited in this respect.

[0097] Unless otherwise stated, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this application includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0098] It is understood that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0099] It is understood that the memory in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Specifically, non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0100] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0103] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0104] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0105] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] The above description is merely a specific embodiment of this application, but the scope of protection of this invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A method for training a search relevance model, characterized in that, include: Input the sample into multiple teacher models to obtain the basic relevance score output by each teacher model; At least some of the teacher models are trained based on different model foundations; the samples include search terms and product information. The samples are input into a gating model to obtain the weights for each teacher model; wherein, the gating model identifies the dominant model among the multiple teacher models and assigns a greater weight to the dominant model than to the non-dominant model. Based on the basic relevance scores output by the multiple teacher models, the target relevance score is obtained by weighted fusion based on the weights output by the gating model. The sample is input into the student model, and the student model is trained with the target relevance as the objective; wherein, the student model serves as the search relevance model.

2. The method according to claim 1, characterized in that, The method further includes: obtaining the result confidence scores of the multiple teacher models corresponding to the sample outputs; wherein, the result confidence scores are used to represent the degree of confidence of the corresponding teacher model in the basic relevance score of the output; The process of inputting the sample into a gating model to obtain the weight of each teacher model includes: inputting the result confidence along with the sample into the gating model to obtain the weight of each teacher model; wherein the gating model uses the teacher model with the specified result confidence among the multiple teacher models as the dominant model.

3. The method according to claim 1, characterized in that, The gating model selects the dominant model from among the multiple teacher models based on the label consistency learned during pre-training; wherein, the gating model identifies the teacher model that has a specified label consistency for the sample from among the multiple teacher models, and this teacher model is selected as the dominant model.

4. The method according to claim 1, characterized in that, The search terms in the sample have type labels; The samples are input into the gating model to obtain the weights for each teacher's model, including: The samples and the type labels are input into the gating model to obtain the weights for each teacher model; wherein, the gating model selects the teacher models among the multiple teacher models that have a specified correlation with the type labels as the dominant models.

5. The method according to claim 1, characterized in that, The method further includes: The entity words in the sample are identified by using at least one teacher model from the plurality of teacher models corresponding to the sample; Key entity words are identified among the identified entity words; wherein, the key entity words are provided to the post-interaction module of the student model so that when the post-interaction module generates the weights of the code words in the search terms and product information based on the bidirectional attention mechanism, the weights assigned to the key entity words are greater than the weights of the non-key entity words.

6. The method according to claim 5, characterized in that, Key entity words are identified from the identified entity words, including: The attributes corresponding to the identified entity words are derived from the specified health knowledge graph; among them, the attributes corresponding to some entity words are key attributes; the entity words corresponding to the key attributes are the key entity words.

7. A correlation evaluation method, characterized in that, The method includes: The obtained search terms and product information are input into the search relevance model obtained by any one of the methods described in claims 1 to 6 to obtain the relevance score between the search terms and product information.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, causes the processor to implement the method as described in any one of claims 1 to 7.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 7.

Images