Semantic relevance prediction models, methods, devices, storage media, and computer equipment

Abstract

This application discloses a semantic relevance prediction model, method, apparatus, storage medium, and computer device, relating to the field of Internet technology. The model includes a student model and an online prompting model. The student model comprises: a parameter-sharing dual-tower network for extracting features from the first and second keywords to be predicted, obtaining first keyword features and second keyword features; a thought chain tower network for extracting features from the thought chain prompts generated by the online prompting model, obtaining thought chain features; the thought chain tower network and the dual-tower network do not share parameters; an expert hybrid network for fusing the first keyword features, second keyword features, and thought chain features, obtaining keyword fusion features; and a deep neural network for performing semantic relevance prediction based on the keyword fusion features, obtaining the semantic relevance prediction result between the first and second keywords. The above model can improve the accuracy of relevance prediction.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a semantic relevance prediction model, method, apparatus, storage medium and computer device. Background Technology

[0002] In information retrieval and natural language processing, semantic relevance models are widely used to measure the degree of semantic association between words, phrases, or text fragments. These models play a crucial role in improving search engine performance, recommendation system accuracy, and the performance of intelligent question-answering systems. With the development of deep learning technology, neural network-based semantic matching methods have gradually become mainstream. This method can automatically learn semantic representations from large-scale data and provide more effective relevance judgment criteria for downstream tasks.

[0003] Currently, common semantic relevance models generally employ a dual-tower or dual-encoder structure. These structures use two encoders with shared or independent parameters to independently encode two input keywords, mapping them to fixed-length vector representations. Finally, the relevance score between the two keywords is calculated based on the similarity between the vectors. However, when applying these structures to complex semantic environments, the models often fail to capture high-order interaction information or deep contextual relationships between keywords during the feature extraction stage. This results in insufficient ability to express complex semantic relationships, severely impacting the accuracy of semantic relevance prediction. Summary of the Invention

[0004] In view of this, embodiments of this application provide a semantic relevance prediction model, method, apparatus, storage medium, and computer device, with the main objective of solving the technical problem of low prediction accuracy of semantic relevance prediction models.

[0005] According to a first aspect of this application, a semantic relevance prediction model is provided, the semantic relevance prediction model comprising a student model obtained by teacher model distillation and a pre-trained online prompting model, the student model comprising:

[0006] A parameter-sharing dual-tower network is used to extract features from the first and second keywords to be predicted, resulting in features of the first and second keywords.

[0007] The thought chain tower network is used to extract features from the thought chain prompts generated by the online prompt model to obtain thought chain features. The parameters of the thought chain tower network and the dual tower network are not shared.

[0008] An expert hybrid network is used to fuse the first keyword features, the second keyword features, and the thought chain features to obtain keyword fusion features;

[0009] A deep neural network is used to predict semantic relevance based on the fusion features of the keywords, and to obtain the semantic relevance prediction results between the first keyword and the second keyword.

[0010] According to a second aspect of this application, a semantic relevance prediction method is provided, the method comprising:

[0011] Obtain the first keyword and the second keyword to be predicted, and input the first keyword and the second keyword into the semantic relevance prediction model described above;

[0012] The online suggestion model in the semantic relevance prediction model is used to predict the relevance of the first keyword and the second keyword, and the reason for the relevance prediction and the relevance prediction result are obtained.

[0013] Based on the reasons for the correlation prediction and the correlation prediction results, a thinking chain prompt is generated, and the thinking chain prompt is input into the student model in the semantic correlation prediction model;

[0014] The student model performs a correlation analysis on the first keyword and the second keyword based on the thought chain prompt information, and obtains the correlation prediction results of the first keyword and the second keyword.

[0015] According to a third aspect of this application, a semantic relevance prediction apparatus is provided, the apparatus comprising:

[0016] The keyword acquisition module is used to acquire the first keyword and the second keyword to be predicted, and input the first keyword and the second keyword into the semantic relevance prediction model mentioned above;

[0017] The relevance prediction module is used to predict the relevance of the first keyword and the second keyword through the online prompting model in the semantic relevance prediction model, and to obtain the reason for the relevance prediction and the relevance prediction result.

[0018] The thinking chain reasoning module is used to generate thinking chain prompts based on the relevance prediction reasons and the relevance prediction results, and input the thinking chain prompts into the student model in the semantic relevance prediction model;

[0019] The relevance prediction module is used by the student model to perform relevance analysis on the first keyword and the second keyword based on the thought chain prompt information, and to obtain the relevance prediction results of the first keyword and the second keyword.

[0020] According to a fourth aspect of this application, a storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the above-described semantic relevance prediction method.

[0021] According to a fifth aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described semantic relevance prediction method.

[0022] By employing the above technical solutions, the semantic relevance prediction model, method, apparatus, storage medium, and computer device provided in this application, through setting a student model obtained by distillation of the teacher model in the semantic relevance prediction model, can reduce the performance overhead of the model when predicting semantic relevance online, thereby improving the prediction efficiency of relevance. Furthermore, by setting an online prompting model in the semantic relevance prediction model and using the online prompting model to generate thought chain prompt information, high-level contextual information between keyword pairs can be extracted, thereby helping to improve the accuracy of relevance prediction. In addition, by setting a parameter-sharing dual-tower structure and a parameter-independent thought chain tower network in the student model, and setting an expert hybrid network for feature fusion, multimodal features in the keyword pairs to be predicted and the thought chain prompt information can be accurately extracted, thereby significantly improving the student model's ability to capture high-level contextual information, and thus enhancing the accuracy of relevance prediction while maintaining low computational overhead.

[0023] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0025] Figure 1 This illustration shows a schematic diagram of the model architecture of a student model in a semantic relevance prediction model provided in an embodiment of this application;

[0026] Figure 2 This illustration shows a schematic diagram of the student model in another semantic relevance prediction model provided in an embodiment of this application;

[0027] Figure 3 This illustration shows a schematic diagram of the model architecture during training of a semantic relevance prediction model provided in an embodiment of this application.

[0028] Figure 4 A flowchart illustrating a semantic relevance prediction method provided in an embodiment of this application is shown.

[0029] Figure 5 A flowchart illustrating another semantic relevance prediction method provided in an embodiment of this application is shown;

[0030] Figure 6 A schematic diagram of the structure of a semantic relevance prediction device provided in an embodiment of this application is shown. Detailed Implementation

[0031] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0032] Currently, common semantic relevance models generally employ a dual-tower or dual-encoder structure. These structures use two encoders with shared or independent parameters to independently encode two input keywords, mapping them to fixed-length vector representations. Finally, the relevance score between the two keywords is calculated based on the similarity between the vectors. However, when applying these structures to complex semantic environments, the models often fail to capture high-order interaction information or deep contextual relationships between keywords during the feature extraction stage. This results in insufficient ability to express complex semantic relationships, severely impacting the accuracy of semantic relevance prediction.

[0033] To address the above problems, in one embodiment, such as Figure 1 As shown, a semantic relevance prediction model is provided. This model includes a student model obtained by distilling a teacher model and a pre-trained online prompting model 20. The student model includes a parameter-sharing dual-tower network 11, a thought chain tower network 12 (which does not share parameters with the dual-tower network 11), an expert hybrid network 13, and a deep neural network 14. The parameter-sharing dual-tower network 11 can be used to extract features from the first and second keywords to be predicted, obtaining first keyword features and second keyword features. The thought chain tower network 12 can be used to extract features from the thought chain prompts generated by the online prompting model, obtaining thought chain features. The expert hybrid network 13 can be used to fuse the first keyword features, the second keyword features, and the thought chain features, obtaining keyword fusion features. The deep neural network 14 can be used to predict semantic relevance based on the keyword fusion features, obtaining the semantic relevance prediction results between the first and second keywords.

[0034] The semantic relevance prediction model refers to a model used to predict the relevance between two keywords; the online suggestion model refers to a pre-trained language model that runs online in real time and can automatically generate structured reasoning suggestions based on input keywords, guiding the student model to perform deeper semantic analysis; the student model refers to a lightweight model learned from a more powerful but computationally complex teacher model (such as a large language model) through knowledge distillation techniques. It maintains high prediction accuracy while significantly reducing inference latency and resource consumption, making it suitable for online real-time search scenarios. In this embodiment, the student model can learn the output of the teacher model by minimizing its output distribution, thereby obtaining reasoning capabilities close to those of the teacher model. This method preserves the student model's ability to understand complex semantics while meeting the requirements of low latency and high concurrency data processing.

[0035] Specifically, refer to Figure 1 The student model can extract features from the first and second keywords to be predicted using a parameter-shared dual-tower network 11. Simultaneously, it can extract features from the thought chain prompts generated by the online prompt model 20 using a parameter-independent thought chain tower network 12, thus generating thought chain features reflecting the logic of part-of-speech differences. Subsequently, the student model can dynamically fuse the above three types of feature vectors using an expert hybrid network 13 (MoE). The expert hybrid network can select different expert models (such as specification matching experts, semantic similarity experts, and category matching experts) based on the differences in the input features, and perform weighted calculations to obtain keyword fusion features. Finally, the student model can use a deep neural network 14 to predict semantic relevance based on the fused features and output the semantic relevance prediction results between the first and second keywords.

[0036] In this embodiment, the thought chain prompts generated by the online prompting model provide higher-level contextual information (such as specification differences, semantic relationships, etc.) compared to simply inputting the first and second keywords to be predicted. Therefore, the student model needs to introduce additional parameters to effectively extract and learn this type of knowledge. Based on this, referring to... Figure 1The network structure of the post-interactive ReprBERT for the student model can be improved by adding an independent COT tower (i.e., the thought chain tower network 12) to the original dual-tower network 11 (Query tower and Item tower, used to process the original first and second keywords) to model thought chain prompts, forming a three-tower structure. The output feature vectors of the three towers can be fused through an expert hybrid network 13, which contains multiple expert models with independent parameters. A suitable expert combination can be dynamically selected to weight and fuse the three types of features, thereby obtaining a more accurate representation. In this embodiment, the newly added COT tower does not share parameters with the dual towers, ensuring that the network can focus on extracting high-order logical features. The expert hybrid network can enhance the collaborative expression of multimodal features through expert-level dynamic interaction, ultimately improving the student model's ability to handle complex semantic relationships.

[0037] The semantic relevance prediction model provided in the above embodiments reduces the performance overhead of online semantic relevance prediction by incorporating a student model derived from the teacher model, thereby improving the prediction efficiency. Furthermore, by including an online prompting model and utilizing it to generate thought chain prompts, higher-order contextual information between keyword pairs can be extracted, thus enhancing the accuracy of relevance prediction. In addition, the model employs a parameter-sharing dual-tower structure and a parameter-independent thought chain tower network within the student model, along with an expert hybrid network for feature fusion. This allows for accurate extraction of multimodal features from the keyword pairs to be predicted and the thought chain prompts generated by the online prompting model, significantly improving the student model's ability to capture higher-order contextual information. Consequently, it enhances the accuracy of relevance prediction while maintaining low computational overhead.

[0038] In one embodiment, such as Figure 2 As shown, in the student model, its deep neural network 14 can perform the following operations: obtain explicit cross features and keyword fusion features output by expert hybrid network 13, wherein the explicit cross features can be constructed based on the cross-construction of multiple features in the text features, category features and entity features of the first keyword and the second keyword, and then perform semantic relevance prediction based on the explicit cross features and keyword fusion features to obtain the semantic relevance prediction results of the first keyword and the second keyword.

[0039] Specifically, refer to Figure 2The deep neural network 14 can predict semantic relevance through a combination of explicit cross features and keyword fusion features. Explicit cross features are generated by combining various features from text features, category features, and entity features in the first and second keywords, and are used to capture explicit associations between keywords. Subsequently, the explicit cross features, along with the keyword fusion features (containing implicit higher-order contextual information) output by the expert hybrid network 13, are input into the deep neural network 14. Cross-modal feature interactions are extracted through multi-layer nonlinear transformations in the deep neural network 14, ultimately outputting the semantic relevance prediction result between the first and second keywords. In this embodiment, explicit cross features can compensate for the insufficient modeling of local associations in the dual-tower structure. Simultaneously, by leveraging the fusion capabilities of the expert hybrid network, the modeling depth of global semantic associations can be enhanced.

[0040] The above embodiments, through collaborative modeling of explicit cross features and keyword fusion features, can significantly improve the accuracy of semantic relevance prediction, especially in handling complex contexts and long-tail association scenarios. Compared with the traditional dual-tower structure model, it can effectively improve the accuracy and robustness of semantic relevance prediction. Simultaneously, by introducing explicit cross features, the model's dependence on the distribution of training data can be effectively reduced.

[0041] In one embodiment, such as Figure 1 and Figure 2 As shown, the online prompting model 20 in the semantic relevance prediction model can perform the following operations: when the student model performs semantic relevance prediction, it obtains the first keyword and the second keyword, and performs relevance prediction on the first keyword and the second keyword to obtain the reason for relevance prediction and the result of relevance prediction. Then, based on the reason for relevance prediction and the result of relevance prediction, it generates mind chain prompt information and inputs the mind chain prompt information into the student model.

[0042] Specifically, the online prompting model dynamically generates thought chain prompts to assist the student model in predicting semantic relevance. In this embodiment, when the student model receives the first and second keywords to be predicted, the online prompting model can simultaneously acquire these two keywords and estimate their relevance based on preset rules set in the model. It then outputs a text description containing the basis for the association (reason for relevance estimation) and a preliminary prediction result (relevance estimation result). Subsequently, the online prompting model can construct structured thought chain prompts based on the reason for estimation and the prediction result, such as outputting "The first and second keywords have high relevance because both entity words and modifiers match." This thought chain prompt is then injected into the student model as additional input, enabling the student model to perform more accurate semantic relevance calculations by combining explicit reasoning logic and original features during decoding.

[0043] The above embodiments utilize an online prompting model to dynamically generate thought chain prompts and inject them into the student model for relevance prediction. This can effectively enhance the student model's understanding of keyword association logic, thereby improving the interpretability and accuracy of relevance prediction results while maintaining the model's lightweight nature.

[0044] In one embodiment, refer to Figure 3 The semantic relevance prediction model also includes a pre-trained teacher model and an offline prompting model. The offline prompting model can generate sample thought chain prompts based on sample keyword pairs and their relevance labels in the sample dataset. The teacher model can receive sample keyword pairs and their thought chain prompts as input to generate semantic relevance prediction results. In the process of knowledge distillation, the student model is trained based on the semantic relevance prediction results output by the teacher model and the relevance labels of the sample keyword pairs to obtain the trained student model.

[0045] Specifically, refer to Figure 3 The semantic relevance prediction model can be comprised of two parts: an offline teacher model (left) and an online student model (right). The teacher model, a large model with numerous parameters, can have four inputs: [CLS], Query (first keyword), Item (second keyword), and COT (thought chain hints). The COT is generated by the offline hint model and processed by the BERT module. Subsequently, its output, Teacher Logits, can be distilled and transmitted to the student model. The model's loss function includes Logits distillation loss (measuring the difference between the student model's output and the teacher model's output) and true loss (measuring the difference between the student model's output and the true label). Furthermore, the online student model can adopt a lightweight design and has three inputs: Query (first keyword), Item (second keyword), and COT (thought chain hints). The COT is generated by the online hint model and processed by the ReprBERT module parameters of the student model, supporting online inference.

[0046] In this embodiment, the lack of COT input (i.e., thought chain hints generated by the online hint model) during online inference by the student model leads to a decline in its relevance prediction performance. Therefore, this embodiment dynamically supplements the COT information (thought chain hints) using an online hint model, thereby improving prediction performance. During knowledge distillation, the student model can continuously optimize its output Student Logits using its hybrid loss function, making it increasingly closer to Teacher Logits. In this embodiment, the soft objective of the teacher model can be used to train the student model, enabling the student model to mimic the output distribution of the teacher model, not just the classification labels. Through this method, the student model can learn not only the correct categories but also the confidence level of the teacher model for each category. Furthermore, to improve online efficiency, the student model can reduce computational costs through parameter sharing, while achieving fast inference with a smaller model size. When training the student model, a hybrid loss function can be used, which simultaneously considers the soft objective of the teacher model and the actual hard labels. This ensures that the student model can both mimic the decision boundaries of the teacher model and perform correct classification.

[0047] In one embodiment, the student model can be trained as follows: First, a teacher model is pre-trained based on sample keyword pairs in the sample dataset and sample thought chain hints generated by an offline hint model. After training, the parameters of the teacher model are frozen. Then, the teacher model with frozen parameters is used to predict the semantic relevance of the sample keyword pairs, and the prediction results output by the teacher model are temperature-scaled to obtain a soft target. The soft target can be generated by a temperature-scaled softmax function, where temperature T is an adjustable hyperparameter. When T>1, the output distribution is smoother and contains more information about the relative probabilities of each category. Further, the student model can be iteratively trained based on the same sample keyword pairs and sample thought chain hints generated by the pre-trained online hint model. The loss function of the student model can include the distillation loss between the temperature-scaled output of the student model and the soft target generated by the teacher model, and the cross-entropy loss between the output of the student model and the relevance labels. Finally, when the training reaches a preset number of rounds or the performance of the student model on the validation set reaches a preset convergence target, model training can be stopped, resulting in a trained student model.

[0048] Specifically, refer to Figure 3When training the student model, a sample dataset containing a large number of sample keyword pairs can be read first. Then, an offline teacher model can be trained using this sample dataset, and the parameters of the teacher model can be frozen after training. For example, in a search scenario, the input to the teacher model can be search keywords, such as "AA battery", product keywords, such as "AA battery / 4-pack", and thought chain hints generated by the offline hint model, such as "the modifiers of the search keyword [AA] and the modifiers of the product keyword [AA] are different, which is due to the difference in specifications leading to a weak correlation". Subsequently, the teacher model can process the above features through the BERT module and output the relevance prediction result. Then, the output prediction result is temperature-scaled to obtain the soft target. Furthermore, the student model can use the same sample keyword pairs and the sample thought chain hints generated by the pre-trained online hint model as input, and iteratively train under the constraint of a hybrid loss function. The loss function can include the distillation loss between the relevance prediction result output by the teacher model (soft target) and the relevance prediction result output by the student model (after temperature scaling) and the cross-entropy loss between the student model's prediction result and the labeled result. Training can be stopped when the hybrid loss converges to a preset range, forming a student model that can be deployed online. The parameters of the student model are independent of those of the teacher model, enabling real-time inference and reducing computational costs.

[0049] The above embodiments, by setting up an offline teacher model and an online student model in the semantic relevance prediction model and co-training the two models, combined with multimodal feature modeling of thought chain prompts and a hybrid optimization strategy of distillation loss and cross-entropy loss, enable the student model to inherit the teacher model's ability to discriminate complex semantic relationships while maintaining low computational overhead. This improves the accuracy and robustness of search relevance prediction in the online inference stage.

[0050] In one embodiment, the online prompting model can be trained by the following method: loading a pre-trained offline prompting model, mapping the weight parameters of the offline prompting model to an integer range based on preset quantization parameters, rounding or truncating the mapped weight parameters, converting the activation values ​​of neurons from floating-point numbers to integers during each forward propagation of the offline prompting model, and finally performing quantization-aware training on the offline prompting model with adjusted weight parameters and activation values ​​to obtain the trained online prompting model.

[0051] Specifically, refer to Figure 3Online suggestion models can be implemented by quantizing and compressing offline suggestion models. The core objective is to significantly reduce the model's storage requirements and computational complexity without significantly sacrificing prediction performance. The main quantization process includes: first, loading the pre-trained offline suggestion model and mapping the model weights from floating-point numbers to integers based on preset quantization parameters (e.g., the integer range [-128, 127]). Then, rounding or truncation is used to eliminate floating-point precision redundancy. Subsequently, during each forward propagation, the activation values ​​of neurons are dynamically converted from floating-point numbers to integer representations (e.g., to 8-bit integer representations) in real time to simulate the low-precision computing environment in actual inference scenarios. Through this method, floating-point weights and activation values ​​in the model can be converted to integer representations that occupy fewer bits. Furthermore, to mitigate the performance degradation caused by quantization, Quantization Aware Training (QAT) can be used for end-to-end optimization of the model. This involves inserting pseudo-quantization nodes during the training phase, allowing the model to synchronously learn the quantization error compensation mechanism during backpropagation, ultimately outputting an online suggestion model that can adapt to low-precision computation while maintaining semantic integrity. The above methods can simulate the quantization effect during model training, enabling the model to optimize its performance under quantization conditions.

[0052] The above embodiments, by introducing weighted quantization and dynamic activation quantization during the quantization process of the online suggestion model, and combining them with an error compensation mechanism for quantization-aware training, enable the online suggestion model to compress its storage space and improve inference speed while maintaining high-precision semantic representation capabilities. Furthermore, through this method, the online suggestion model can be compatible with the hardware acceleration instruction set of the offline suggestion model without requiring additional calibration data during actual deployment, thereby significantly reducing the resource consumption of online inference.

[0053] In one embodiment, a semantic relevance prediction method is provided. Taking the application of the above method to computer devices such as servers as an example, Figure 4 As shown, the method includes the following steps:

[0054] Step 101: Obtain the first keyword and the second keyword to be predicted, and input the first keyword and the second keyword into the pre-trained semantic relevance prediction model.

[0055] The pre-trained semantic relevance prediction model refers to a joint architecture that includes an online prompting model and a student model. Its model architecture can be found in any of the above embodiments and will not be repeated here.

[0056] Specifically, the server can obtain the first and second keywords to be predicted through API interfaces and / or user input information, and encapsulate the original text of these two keywords into structured input vectors, which are then synchronously transmitted to the input layer of the semantic relevance prediction model to ensure that the model can start the prediction process based on complete contextual features.

[0057] Step 102: Use the online prompting model in the semantic relevance prediction model to predict the relevance of the first keyword and the second keyword, and obtain the reason for the relevance prediction and the relevance prediction result.

[0058] Specifically, after receiving the first and second keywords, the online suggestion model can concatenate the input keyword pairs in a specific format and use the concatenated features as input to the online suggestion model. Then, the online suggestion model can generate a natural language suggestion containing the reasoning process and reasoning result based on the input keyword pairs, that is, output the reasoning and result of the relevance prediction of the above keyword pairs.

[0059] Step 103: Based on the reasoning and results of the relevance prediction, generate the thinking chain prompt information and input the thinking chain prompt information into the student model in the semantic relevance prediction model.

[0060] Among them, the thought chain prompt information refers to the intermediate text containing reasoning logic generated by the online prompting model, which can simulate the human thinking process when judging whether two keywords are related.

[0061] Specifically, the online suggestion model can generate thought chain suggestions based on the relevance prediction reasons and results of the first and second keywords, and then input these suggestions as additional input along with the original word pairs into the student model. This method transforms abstract relevance judgments into a traceable reasoning process, significantly enhancing the transparency and logic of the semantic relevance prediction model's decisions and helping to improve the accuracy of relevance judgments under complex or fuzzy queries.

[0062] Step 104: The student model performs a correlation analysis on the first keyword and the second keyword based on the thought chain prompts to obtain the correlation prediction results of the first keyword and the second keyword.

[0063] The relevance prediction result refers to the quantitative output of the student model on the degree of relevance between the first keyword and the second keyword under the guidance of the thought chain prompts. It can be expressed as a continuous value (such as a relevance score between 0 and 1) or a discrete category (such as "strongly related", "weakly related", "unrelated"). This result can reflect the final judgment of the student model after comprehensively considering semantics, attributes and reasoning logic.

[0064] Specifically, after receiving the first keyword, the second keyword, and the thought chain hints, the student model can extract and fuse features from these three types of text, and then determine relevance based on the fused features. In this embodiment, the student model can employ a dual-tower neural network and an independent neural network, combined with a hybrid expert network. The dual-tower neural network encodes the first and second keywords separately, the independent neural network encodes the thought chain hints, and the hybrid expert network fuses the three features for subsequent relevance prediction. During inference, the student model not only focuses on matching the surface meaning of keywords but also adjusts attention weights based on logical clues in the thought chain hints, thereby suppressing misjudgments caused by partial word matching. Through this approach, high-quality, interpretable relevance prediction can be achieved on a lightweight student model, thus balancing model performance and computational efficiency, and ultimately improving the accuracy of search results.

[0065] The semantic relevance prediction method provided in the above embodiments, by utilizing an online prompting model to generate thought chain prompts for the keyword pairs to be predicted, can extract high-level contextual information between the keyword pairs, thereby helping to improve the accuracy of relevance prediction. Furthermore, by employing a student model obtained through teacher model distillation for relevance prediction, the method can reduce the performance overhead of the model when predicting semantic relevance online, thus improving the efficiency of relevance prediction. In addition, by accurately extracting multimodal features from the keyword pairs to be predicted and the thought chain prompts generated by the online prompting model using the student model, the method can significantly improve the student model's ability to capture high-level contextual information, thereby enhancing the accuracy of relevance prediction while maintaining low computational overhead.

[0066] In one embodiment, a semantic relevance prediction method is provided for a search scenario. In this scenario, the first keyword and the second keyword can be the search keyword and the product keyword, respectively. Taking the above method applied to computer devices such as servers as an example, ... Figure 5 As shown, the method includes the following steps:

[0067] Step 201: Obtain search keywords and, based on the search keywords, match at least one product keyword. Input the search keywords and product keywords into the semantic relevance prediction model.

[0068] Among them, search keywords refer to words or phrases that users enter in the search box of a search engine or e-commerce platform to express their information needs or purchase intentions; product keywords refer to words or phrases that are closely related to products on the platform and are used to describe product attributes, categories, specifications or uses. They can come from product titles, tags, attribute fields or category information and are the basic unit for information matching and retrieval.

[0069] Specifically, computer devices can acquire user-inputted text content in real time through a front-end user interface, i.e., obtain the user's search keywords. These keywords are then standardized, such as by removing spaces, correcting spelling errors, and normalizing synonyms, to improve keyword matching accuracy. Information can then be searched based on the processed search keywords, and corresponding product keywords can be found in a product keyword database based on the searched product and merchant information. After obtaining the search keywords and product keywords, they can be input into a semantic relevance prediction model. Through this method, candidate product keywords potentially relevant to the user's query can be initially filtered from a massive amount of product information.

[0070] Step 202: Use the online suggestion model in the semantic relevance prediction model to estimate the relevance of search keywords and product keywords, and obtain the reason for the relevance prediction and the relevance prediction result.

[0071] Specifically, after inputting the aforementioned keyword pairs into the semantic relevance prediction model, the model can concatenate the input keyword pairs in a specific format and use this as input to the online suggestion model. Furthermore, the online suggestion model can generate a natural language suggestion containing the reasoning process and results based on the input keyword pairs, i.e., outputting the reasoning and results of the relevance prediction for the search keywords and product keywords.

[0072] Step 203: Based on the reasoning and results of the relevance prediction, generate the thinking chain prompt information and input the thinking chain prompt information into the student model in the semantic relevance prediction model.

[0073] Among them, the thought chain prompt information refers to intermediate text containing reasoning logic generated by the online prompting model, which can simulate the human thought process when judging whether two keywords are related. For example, it first identifies the parts of speech of two keywords, then compares the differences between the two keywords from multiple perspectives, and finally draws a conclusion. This structured prompt information can guide the student model to conduct a more in-depth and interpretable relevance analysis.

[0074] Specifically, after receiving search keywords and product keywords, the online suggestion model can generate thought chain suggestions using pre-defined reasoning steps. For example, the online suggestion model can perform relevance judgment based on pre-defined rule templates, the free reasoning logic of generative models, or dynamically generate relevance judgment results by combining external knowledge bases. Then, the online suggestion model can generate thought chain suggestions based on the relevance reasoning basis and results, and input the thought chain suggestions as additional input along with the original word pairs into the student model. Through the above methods, the abstract relevance judgment can be transformed into a traceable reasoning process, thereby significantly enhancing the transparency and logic of the semantic relevance prediction model's decision-making, and helping to improve the accuracy of relevance judgment under complex or fuzzy queries.

[0075] Step 204: The student model performs a relevance analysis on the search keywords and product keywords based on the mind chain prompts, and obtains the relevance prediction results of the search keywords and product keywords.

[0076] The relevance prediction result refers to the quantitative output of the student model on the degree of relevance between search keywords and product keywords under the guidance of the thinking chain prompts. It can be expressed as a continuous value (such as a relevance score between 0 and 1) or a discrete category (such as "strongly relevant", "weakly relevant", "unrelevant"). This result can reflect the final judgment of the student model after comprehensively considering semantics, attributes and reasoning logic.

[0077] Specifically, after receiving search keywords, product keywords, and thought chain hints, the student model can extract and fuse features from these three types of text, and then determine relevance based on the fused features. In this embodiment, the student model can employ a dual-tower neural network and an independent neural network, combined with a hybrid expert network. The dual-tower neural network encodes search keywords and product keywords separately, the independent neural network encodes thought chain hints, and the hybrid expert network fuses the three features for subsequent relevance prediction. During inference, the student model not only focuses on matching the surface meaning of keywords but also adjusts attention weights based on logical clues in the thought chain hints, thereby suppressing misjudgments caused by partial word matching. Through this approach, high-quality, interpretable relevance prediction can be achieved on a lightweight student model, thus balancing model performance and computational efficiency, and ultimately improving the accuracy of search results.

[0078] Step 205: Based on the relevance prediction results, obtain the search results corresponding to the search keywords.

[0079] The search results refer to the list of products or merchants presented to users after the search system sorts them according to the relevance prediction results. The search results can be sorted based on the relevance scores of all candidate product keywords, and then the top N products with the highest scores are selected as the final display results.

[0080] Specifically, after obtaining the relevance prediction results between search keywords and product keywords, these relevance prediction results can be used as one of the ranking features. They are then combined with other ranking features, such as product sales volume, user ratings, price, and user personalized preferences, and input into the ranking model for comprehensive scoring. In this embodiment, if a product's relevance prediction score is lower than a preset threshold, the product is directly filtered out to avoid displaying obviously irrelevant results. For example, for the search keyword "AA battery," the product keyword "AA battery / 4-pack" is initially recalled because it contains "battery." However, its relevance score is judged as weakly relevant due to the reasoning that "AA and AA have different attributes," thus being downgraded or excluded in the final results. Through this method, it can be ensured that the search results presented to the user are both highly relevant to the search keywords and have good ranking quality, thereby improving search efficiency and user satisfaction.

[0081] In a specific application scenario, taking the search keyword "AA battery" and the product keyword "size 7 battery / 4-pack" as examples, the above semantic relevance prediction method is explained. First, the search system obtains the user-input search keyword "AA battery," then matches it with product keywords containing "battery" or "AA" based on the inverted index, and initially recalls the product keyword "size 7 battery / 4-pack." Next, "AA battery" and "size 7 battery / 4-pack" are input into the semantic relevance prediction model. The online prompting model of the semantic relevance prediction model analyzes the relevance between the two keywords and generates a thought chain prompt: "The modifier 'AA' in the search keyword is different from the modifier '7' in the product keyword. They belong to two different battery specifications and do not meet the semantic relevance of compound words; therefore, they are not related." Subsequently, the student model receives the above thought chain prompt and performs relevance analysis in conjunction with the original word pair to identify key attribute conflicts, ultimately outputting a low-relevance prediction result (e.g., a score of 0.2). Finally, based on this low score, the search system can significantly reduce the ranking of products corresponding to "size 7 batteries / 4-pack" during sorting, or even filter them out, to ensure that users mainly see highly relevant products such as "size 5 batteries" or "AA batteries," thereby providing accurate search results. The above method, through efficient prediction using product keyword matching, deep semantic reasoning, and lightweight models, can significantly improve the accuracy of relevance judgment in complex semantic scenarios such as specification differences and synonyms, while also ensuring the real-time performance and stability of online services and optimizing the user's search experience.

[0082] The above embodiments match at least one product keyword based on search keywords, and use an online suggestion model in the semantic relevance prediction model to perform information reasoning on the search keywords and product keywords to obtain thought chain suggestion information. Then, a student model performs relevance analysis on the search keywords and product keywords based on the thought chain suggestion information to obtain the relevance prediction results between the search keywords and product keywords. Finally, the search results are obtained based on the relevance prediction results. This method enhances the interpretability and logic of relevance prediction by using the real-time generated relevance reasoning process and results as suggestion information, thereby significantly improving the accuracy of relevance judgment in complex semantic scenarios. Furthermore, by using a knowledge-distilled student model for relevance prediction, the method achieves a balance between model performance and prediction efficiency, reducing the computational resources required for relevance prediction, ensuring the real-time performance and stability of online services, and thus optimizing the user's search experience.

[0083] In one embodiment, in step 201, search keywords and product keywords can be obtained in the following way: First, the search keywords input by the user are obtained as the first keyword, and the search keywords are searched for information through a pre-trained retrieval enhancement generation model to obtain preliminary search results. Then, based on the product information and / or merchant information in the preliminary search results, at least one product keyword is matched in the product keyword database as the second keyword.

[0084] Specifically, after obtaining the user's input search keywords, the system first uses a pre-trained Retrieval-Augmented Generation (RAG) model to search for information related to the search keywords. By leveraging the semantic understanding and text generation capabilities of the RAG model, multiple product and merchant information related to the search keywords can be retrieved. Then, the retrieved product information (such as name, specifications, and category) and merchant information (such as store tags) can be matched against the product keyword database to find at least one product keyword related to the search keywords.

[0085] For example, taking the search keyword "AA battery" as an example, after obtaining the search keyword, the search system can first use a retrieval enhancement generative model to search for the keyword to find product information containing tags such as "AA" or "battery". Then, based on the search results, it can perform keyword matching in the product keyword database to match multiple search keywords related to the search keyword, such as "AA battery / 4-pack", "AA model battery / 10-pack", "AAA battery / 4-pack".

[0086] The above embodiments utilize the semantic retrieval and text generation capabilities of the retrieval-enhanced generative model to batch recall product keywords related to search keywords, including keywords with specification differences or synonyms, thereby reducing the problem of missed detections due to literal mismatch and expanding the scope of product search.

[0087] In one embodiment, in steps 202 and 203, the thought chain prompt information can be generated as follows: First, the search keywords are analyzed for part-of-speech tags using an online prompt model, and it is determined whether the search keywords are non-content words, brand words, or generic words. Then, the thought chain method is used to sequentially determine whether the search keywords, when they are brand words, match the brand name corresponding to the product keywords, and whether the search keywords, when they are generic words, match the product subject, modifiers, and product category corresponding to the product keywords. Subsequently, based on the matching results of the search keywords and product keywords, a relevance prediction result and a relevance prediction reason are generated. Finally, based on the relevance prediction result and the relevance prediction reason, a thought chain prompt information is generated, and the thought chain prompt information is input into the student model to use the student model for keyword relevance prediction.

[0088] Specifically, after acquiring search keywords and product keywords, the online prompting model first analyzes the part-of-speech (POS) of the search keywords based on a preset prompt template to determine whether they are non-content terms, brand terms, or generic terms. After determining the POS, it matches the search keywords and product keywords based on the corresponding judgment logic for each term. Finally, based on the relevance judgment reasoning and process of the search keywords, it generates thought chain prompts which are then input into the student model.

[0089] For example, taking the search keyword "size 5 battery" as an example, after obtaining the search keyword, the online prompt model first performs part-of-speech analysis on the keyword and identifies "size 5" as a modifier (indicating battery specifications) and "battery" as the main product term, thus determining it to be a generic term rather than a brand term. Subsequently, a thought chain approach can be used to judge sequentially: when the search keyword is a generic term, it analyzes whether it matches the product main product (battery), modifier (size 7), and product category (battery category) corresponding to the product keyword "size 7 battery / 4-pack". It finds that "size 5" and "size 7" have different specifications (such as AA and AAA models), but the product main product and category are the same. Based on this, the relevance prediction result is generated as "weak correlation", and the reason for the prediction is "the inconsistency in specifications leads to the incomplete satisfaction of the semantic relevance of generic terms". Finally, the thought chain prompt information is integrated as "the modifier [size 5] in the search keyword is different from the modifier [size 7] in the product keyword. They belong to two different battery specifications and do not completely satisfy the semantic relevance of generic terms. Therefore, it is a weak correlation". Finally, this prompt information is input into the student model to assist in relevance prediction.

[0090] The above embodiments, by introducing part-of-speech analysis and structured thought chain reasoning into the online prompt model, can perform refined matching analysis of search keywords and product keywords in multiple dimensions such as brand, subject, modifiers and category. Especially for the specification difference problem in the general word scenario, the online prompt model can enhance the semantic relevance prediction model's ability to identify weak correlations by explicitly generating explainable prediction reasons, thereby improving the accuracy of relevance judgment in complex semantic scenarios.

[0091] In one embodiment, in step 203, the online suggestion model can perform thought chain reasoning on search keywords and product keywords in the following way: when the search keyword is a non-content word, the relevance prediction result is determined to be strongly related; when the search keyword is a brand word, if the brand word matches the brand name corresponding to the product keyword, the relevance prediction result is determined to be strongly related; if the brand word does not match the brand name corresponding to the product keyword, the relevance prediction result is determined to be unrelated; when the search keyword is a generic word, if the generic word matches the product subject, modifier, and product category corresponding to the product keyword, the relevance prediction result is determined to be strongly related; if the generic word matches the product subject corresponding to the product keyword but the modifier does not match, or the product subject matches but the product category does not match, the relevance prediction result is determined to be weakly related; if the generic word does not match the product subject corresponding to the product keyword, the relevance prediction result is determined to be unrelated; finally, based on the judgment conditions of the relevance prediction result, a relevance prediction reason is generated.

[0092] Specifically, when the online suggestion model performs thought chain reasoning on search keywords and product keywords, it can first determine the type of search keywords. When the search keyword is a non-content term, the relevance prediction result can be directly determined as strong correlation. For example, when a user searches for "XX Plaza," since "XX Plaza" itself has no specific target, a strong correlation is achieved simply by the location attribute. When the search keyword is a brand term, if it matches the brand name in the product keyword (i.e., the brand name is the same), it can be determined as strong correlation. If the brand term does not match the brand name in the product keyword (i.e., the brand name is different), it is determined as irrelevant. When the search keyword is a generic term, if it matches the product subject, modifiers, and category of the product keyword, it is determined as strong correlation. For example, "AA battery" matches "AA battery / 8-pack," which is a strong correlation. If the product subject matches but the modifiers do not, such as "AA battery" and "AAA battery / 4-pack," or if the product subject matches but the category does not, such as "AA battery" and "battery charger," it is determined as weak correlation. If the product subject does not match, such as "AA battery" and "power bank," it is determined as irrelevant. Finally, based on the above judgment conditions, a reason for the relevance prediction can be generated. For example, "The search keyword '[size 5 battery]' is a generic term. Its product subject 'battery' matches the product keyword 'size 7 / 4-pack of batteries,' but the modifiers 'size 5' and 'size 7' are inconsistent. Therefore, it is judged to be weakly related."

[0093] The above embodiments, by introducing non-content terms, brand terms, and generic terms for multi-dimensional classification and judgment, and combining product subject, modifiers, and categories for matching rules, can perform structured reasoning on the relevance of search keywords and product keywords. Furthermore, by explicitly defining the judgment conditions for weak relevance, this embodiment can enhance the semantic relevance prediction model's ability to identify fuzzy semantic relationships, thereby ensuring the accuracy of strong relevance results while improving the accuracy and interpretability of discrimination in weak and irrelevant scenarios.

[0094] In one embodiment, in step 204, the student model can perform relevance prediction as follows: First, the student model can extract features from search keywords and product keywords using a parameter-sharing dual-tower network to obtain search keyword features and product keyword features. Simultaneously, it can extract features from thought chain prompts using a thought chain tower network to obtain thought chain features. The thought chain tower network and the dual-tower network do not share parameters. Further, an expert hybrid network can be used to fuse the search keyword features, product keyword features, and thought chain features, and relevance prediction can be performed based on the fused features to obtain the relevance prediction results between the search keywords and product keywords.

[0095] Specifically, after receiving search keywords, product keywords, and thought chain hints generated by the online hint model, the student model can extract features from the search keywords and product keywords through a parameter-sharing dual-tower network. Simultaneously, it can independently extract features from the thought chain hints generated by the online hint model through a thought chain tower network, generating feature vectors reflecting part-of-speech differences. Subsequently, the student model can dynamically fuse these three types of feature vectors using a Mixture of Experts (MoE) network. The MoE network can select different expert models (such as specification matching experts, semantic similarity experts, and category matching experts) based on the differences in the input features, and perform weighted calculations to obtain the fused features. Finally, the student model can perform relevance prediction based on the fused features and output the relevance prediction results between the search keywords and product keywords.

[0096] In this embodiment, the thought chain suggestions generated by the online suggestion model provide higher-level contextual information (such as specification differences, semantic relationships, etc.) compared to simply inputting search keywords and product keywords. Based on this, referring to... Figures 1 to 3 The student model's post-interactive ReprBERT network structure has been improved by adding an independent COT (Cognitive Chain Tower) network to the original dual-tower network. This tower models cognitive chain prompts, forming a three-tower structure. The output feature vectors of the three towers can be fused using an expert hybrid network. This network contains multiple expert models with independent parameters, dynamically selecting suitable expert combinations to weight and fuse search keyword features, product keyword features, and cognitive chain features, thereby obtaining a more accurate representation. In this embodiment, the newly added cognitive chain tower network does not share parameters with the dual-tower network, ensuring that it can focus on extracting higher-order logical features. The expert hybrid network, through expert-level dynamic interaction, strengthens the collaborative expression of multimodal features, ultimately improving the student model's ability to handle complex semantic relationships.

[0097] The above embodiments, by setting a dual-tower structure and an independent thought chain tower network in the student model, and setting an expert hybrid network, can accurately extract multimodal features of search keywords, product keywords and thought chain prompts, thereby significantly improving the student model's ability to capture high-order contextual information, and thus enhancing the accuracy of relevance prediction while maintaining low computational overhead.

[0098] The technical solutions of the above embodiments can all be applied to the transaction and delivery services of instant e-commerce platforms, such as Taobao Flash Sale, Taoxianda, Ele.me takeaway and retail.

[0099] Furthermore, as Figure 4 and Figure 5 The specific implementation of the method shown in this application provides a semantic relevance prediction device, such as... Figure 6 As shown, the device includes:

[0100] The keyword acquisition module 31 is used to acquire the first keyword and the second keyword to be predicted, and input the first keyword and the second keyword into the pre-trained semantic relevance prediction model;

[0101] The relevance prediction module 32 is used to predict the relevance of the first keyword and the second keyword through the online prompting model in the semantic relevance prediction model, and to obtain the relevance prediction reason and relevance prediction result.

[0102] The thinking chain reasoning module 33 is used to generate thinking chain prompt information based on the relevance prediction reason and the relevance prediction result, and input the thinking chain prompt information into the student model in the semantic relevance prediction model;

[0103] The relevance prediction module 34 is used by the student model to perform relevance analysis on the first keyword and the second keyword based on the thought chain prompt information, and to obtain the relevance prediction results of the first keyword and the second keyword.

[0104] In a specific application scenario, the first keyword is a search keyword, and the second keyword is a product keyword. The relevance prediction module 32 is specifically used to: perform part-of-speech analysis on the search keyword using the online suggestion model, and determine whether the search keyword is a non-content word, a brand word, or a generic word; sequentially determine whether the search keyword matches the brand name corresponding to the product keyword when it is a brand word, and whether the search keyword matches the product subject, modifiers, and product category corresponding to the product keyword when it is a generic word; based on the matching results of the search keyword and the product keyword, obtain the reason for the relevance prediction and the relevance prediction result between the search keyword and the product keyword.

[0105] In specific application scenarios, the relevance prediction module 32 is further configured to: determine a strong relevance prediction result when the search keyword is a non-content term; determine a strong relevance prediction result when the search keyword is a brand term if the brand term matches the brand name corresponding to the product keyword; determine an unrelevance prediction result if the brand term does not match the brand name corresponding to the product keyword; determine a strong relevance prediction result when the search keyword is a generic term if the generic term matches the product subject, modifier, and product category corresponding to the product keyword; determine a weak relevance prediction result if the generic term matches the product subject but not the modifier, or matches the product subject but not the product category; and generate a relevance prediction reason based on the judgment conditions of the relevance prediction result.

[0106] In specific application scenarios, the relevance prediction module 34 can be used for: the student model extracting features from the search keywords and product keywords through a parameter-sharing dual-tower network to obtain search keyword features and product keyword features; the student model extracting features from the thought chain prompt information through a thought chain tower network to obtain thought chain features, wherein the parameters of the thought chain tower network and the dual-tower network are not shared; the student model fusing the search keyword features, product keyword features, and thought chain features through an expert hybrid network, and performing relevance prediction based on the fused features to obtain the relevance prediction result between the search keywords and the product keywords.

[0107] In a specific application scenario, the keyword acquisition module 31 is further configured to: acquire the search keyword input by the user as the first keyword, and perform information search on the search keyword through a pre-trained retrieval enhancement generation model to obtain preliminary search results; based on the product information and / or merchant information in the preliminary search results, match at least one product keyword in the product keyword database as the second keyword; then the device further includes a search result determination module, wherein the search result determination module can be used to: obtain the search result corresponding to the search keyword based on the relevance prediction result between the search keyword and the product keyword, and send the search result to the client.

[0108] It should be noted that other corresponding descriptions of the functional units involved in the semantic relevance prediction device provided in this application embodiment can be found in the following references. Figures 4 to 5 The corresponding descriptions in the method will not be repeated here.

[0109] This application also provides a computer device, specifically a personal computer, server, network device, etc. The computer device includes a bus, processor, memory, and communication interface, and may also include input / output interfaces and a display device. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores location information. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0110] Those skilled in the art will understand that the structure of the computer device described above is only a partial structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components, or combine certain components, or have different component arrangements.

[0111] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, having stored thereon a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0112] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0113] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0114] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0115] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0116] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A semantic relevance prediction model, characterized in that, The semantic relevance prediction model includes a student model obtained by distillation of the teacher model and a pre-trained online prompting model, wherein the student model includes: A parameter-sharing dual-tower network is used to extract features from the first and second keywords to be predicted, resulting in features of the first and second keywords. The thought chain tower network is used to extract features from the thought chain prompts generated by the online prompt model to obtain thought chain features. The parameters of the thought chain tower network and the dual tower network are not shared. An expert hybrid network is used to fuse the first keyword features, the second keyword features, and the thought chain features to obtain keyword fusion features; A deep neural network is used to obtain the keyword fusion features and explicit cross features. The explicit cross features are constructed based on the text features, category features and entity features in the first keyword and the second keyword. Semantic relevance prediction is performed based on the explicit cross features and the keyword fusion features to obtain the semantic relevance prediction results between the first keyword and the second keyword. The online prompting model is used to obtain the first keyword and the second keyword when the student model performs semantic relevance prediction, and to perform relevance prediction on the first keyword and the second keyword to obtain the relevance prediction reason and the relevance prediction result; based on the relevance prediction reason and the relevance prediction result, to generate mind chain prompt information, and to input the mind chain prompt information into the student model.

2. The semantic relevance prediction model according to claim 1, characterized in that, The semantic relevance prediction model also includes a pre-trained teacher model and an offline prompting model, wherein, The offline prompting model is used to generate sample thought chain prompts based on sample keyword pairs in the sample dataset and the relevance labels of the sample keyword pairs; The teacher model is used to receive the sample keyword pairs and the sample thought chain prompts as input, and generate semantic relevance prediction results. In the process of knowledge distillation, the student model is trained based on the semantic relevance prediction results output by the teacher model and the relevance labels of the sample keyword pairs to obtain a trained student model.

3. The semantic relevance prediction model according to claim 2, characterized in that, The training method for the student model includes: Based on the sample keyword pairs in the sample dataset and the sample thought chain prompts generated by the offline prompting model, the teacher model is pre-trained and the parameters of the teacher model are frozen. The teacher model with frozen parameters is used to predict the semantic relevance of the sample keyword pairs, and the prediction results output by the teacher model are obtained by temperature scaling to obtain the soft target. Based on the sample keyword pairs and the sample thought chain prompts generated by the pre-trained online prompting model, the student model is iteratively trained. The loss function of the student model includes the distillation loss between the temperature-scaled output of the student model and the soft target generated by the teacher model, and the cross-entropy loss between the output of the student model and the relevant label. When the training reaches a preset number of rounds or the student model's performance on the validation set reaches a preset convergence target, the model training stops, and the trained student model is obtained.

4. The semantic relevance prediction model according to claim 2, characterized in that, The training method for the online prompting model includes: Load the pre-trained offline prompting model, and map the weight parameters of the offline prompting model to an integer range based on preset quantization parameters, and round or truncate the mapped weight parameters. During each forward propagation of the offline prompting model, the activation values ​​of neurons are converted from floating-point numbers to integers; The offline prompting model with adjusted weight parameters and activation values ​​is subjected to quantization perception training to obtain the trained online prompting model.

5. A semantic relevance prediction method, characterized in that, The method includes: Obtain the first keyword and the second keyword to be predicted, and input the first keyword and the second keyword into the semantic relevance prediction model as described in any one of claims 1 to 4; The online suggestion model in the semantic relevance prediction model is used to predict the relevance of the first keyword and the second keyword, and the reason for the relevance prediction and the relevance prediction result are obtained. Based on the reasons for the correlation prediction and the correlation prediction results, a thinking chain prompt is generated, and the thinking chain prompt is input into the student model in the semantic correlation prediction model; The student model performs a correlation analysis on the first keyword and the second keyword based on the thought chain prompt information, and obtains the correlation prediction results of the first keyword and the second keyword.

6. The semantic relevance prediction method according to claim 5, characterized in that, The first keyword is a search keyword, and the second keyword is a product keyword; then, the online suggestion model in the semantic relevance prediction model is used to predict the relevance of the first keyword and the second keyword, obtaining the reason for the relevance prediction and the relevance prediction result, including: The online suggestion model is used to perform part-of-speech analysis on the search keywords and determine whether the search keywords are non-content words, brand words, or generic words. The system uses a chain of thought approach to sequentially determine whether the search keyword matches the brand name corresponding to the product keyword when it is a brand term, and whether the search keyword matches the product subject, modifiers, and product category corresponding to the product keyword when it is a generic term. Based on the matching results of the search keywords and the product keywords, the reasons for the correlation estimation and the correlation estimation results between the search keywords and the product keywords are obtained.

7. The semantic relevance prediction method according to claim 6, characterized in that, The process of obtaining the relevance prediction rationale and relevance prediction result between the search keywords and product keywords based on the matching results of the search keywords and product keywords includes: When the search keyword is not a content term, the relevance prediction result is determined to be strongly relevant; When the search keyword is a brand term, if the brand term matches the brand name corresponding to the product keyword, the relevance prediction result is determined to be strongly related; if the brand term does not match the brand name corresponding to the product keyword, the relevance prediction result is determined to be unrelated. When the search keyword is a generic term, if the generic term matches the product subject, modifiers, and product category corresponding to the product keyword, the relevance prediction result is determined to be strongly related; if the generic term matches the product subject corresponding to the product keyword but the modifiers do not match, or the product subject matches but the product category does not match, the relevance prediction result is determined to be weakly related; if the generic term does not match the product subject corresponding to the product keyword, the relevance prediction result is determined to be unrelated. Based on the judgment conditions of the correlation prediction results, a reason for the correlation prediction is generated.

8. The semantic relevance prediction method according to claim 6 or 7, characterized in that, The student model performs a correlation analysis on the first keyword and the second keyword based on the thought chain prompt information, and obtains the correlation prediction results of the first keyword and the second keyword, including: The student model extracts features from the search keywords and product keywords through a parameter-sharing dual-tower network, thereby obtaining search keyword features and product keyword features; The student model extracts features from the thought chain prompts through the thought chain tower network to obtain thought chain features, wherein the parameters of the thought chain tower network and the dual tower network are not shared. The student model uses an expert hybrid network to fuse the features of the search keywords, the product keywords, and the thought chain, and then performs relevance prediction based on the fused features to obtain the relevance prediction results between the search keywords and the product keywords.

9. The semantic relevance prediction method according to claim 6 or 7, characterized in that, The process of obtaining the first and second keywords to be predicted includes: The user-inputted search keywords are obtained as the first keyword, and a pre-trained retrieval enhancement generation model is used to perform information retrieval on the search keywords to obtain preliminary search results. Based on the product information and / or merchant information in the preliminary search results, at least one product keyword is matched in the product keyword database as the second keyword; After obtaining the relevance prediction results of the first keyword and the second keyword, the method further includes: obtaining the search results corresponding to the search keyword based on the relevance prediction results of the search keyword and the product keyword, and sending the search results to the client.

10. A semantic relevance prediction device, characterized in that, The device includes: A keyword acquisition module is used to acquire a first keyword and a second keyword to be predicted, and input the first keyword and the second keyword into the semantic relevance prediction model as described in any one of claims 1 to 4; The relevance prediction module is used to predict the relevance of the first keyword and the second keyword through the online prompting model in the semantic relevance prediction model, and to obtain the reason for the relevance prediction and the relevance prediction result. The thinking chain reasoning module is used to generate thinking chain prompts based on the relevance prediction reasons and the relevance prediction results, and input the thinking chain prompts into the student model in the semantic relevance prediction model; The relevance prediction module is used by the student model to perform relevance analysis on the first keyword and the second keyword based on the thought chain prompt information, and to obtain the relevance prediction results of the first keyword and the second keyword.

11. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 5 to 9.

12. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 5 to 9.

Images