Semantic understanding model optimization method and system based on search click log

Abstract

The invention discloses a semantic understanding model optimization method and system based on a search click log, and mainly relates to the technical field of natural language processing. Comprising the steps of obtaining a historical search click log set, executing commodity information and query text extraction, and obtaining a text pair set; traversing the text pair set to perform dropout enhancement, and constructing a positive sample pair set and a negative sample pair set; constructing an initial semantic understanding model; determining a base class text pair set and a long-tail difficult-to-divide text pair set; and introducing a staged fine tuning mechanism, and optimizing the initial semantic understanding model in combination with the base class text pair set and the long-tail difficult-to-separate text pair set to obtain the semantic understanding model. The method has the advantages that the technical problems that in the prior art, semantic understanding depends on manual work or weak supervision labels, the distinguishing ability for low-frequency, complex and long-tail intentions is poor, and the generalization ability is insufficient are solved, and the technical effect of improving the intention understanding and semantic processing accuracy is achieved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a method and system for optimizing semantic understanding models based on search click logs. Background Technology

[0002] In modern search engine systems, understanding user query intent and accurately matching relevant content are crucial for improving retrieval efficiency and user experience. Existing search semantic understanding methods typically rely on supervised training with large-scale labeled data or employ unsupervised representation learning approaches, such as SimCSE and Sentence-BERT. These methods are based on contextual sentence enhancement, constructing positive and negative sample pairs for comparative learning, thereby improving semantic representation capabilities.

[0003] However, due to the scarcity of publicly available labeled data and the long-tail distribution of online log data, existing methods perform well in covering mainstream high-frequency semantic scenarios, but have poor generalization ability for low-frequency and long-tail scenarios, which can easily lead to problems such as "missed intent detection" and "confusion of similar words" in search recall and ranking.

[0004] Existing technologies suffer from problems such as relying on manual or weakly supervised labeling for semantic understanding, poor ability to distinguish low-frequency, complex, and long-tail intentions, and insufficient generalization ability. Summary of the Invention

[0005] This application provides a method and system for optimizing semantic understanding models based on search click logs, which addresses the technical problems in existing technologies where semantic understanding relies on manual or weakly supervised labeling, resulting in poor ability to distinguish low-frequency, complex, and long-tail intentions and insufficient generalization ability.

[0006] In view of the above problems, this application provides a method and system for optimizing semantic understanding models based on search click logs.

[0007] The first aspect of this application provides a method for optimizing a semantic understanding model based on search click logs, the method comprising: A historical search click log set is obtained, and product information and query text extraction are performed to obtain a text pair set. Each text pair includes the query text, a list of query product information, and the product information actually clicked by the user. Dropout enhancement is performed on the text pair set to construct a positive sample pair set and a negative sample pair set. Unsupervised pre-training is performed using GTE-Large based on the positive and negative sample pair sets, and vertical fine-tuning is performed using Simcseloss to construct an initial semantic understanding model. Long-tail hard-to-distinguish identification is performed on the text pair set to determine the base class text pair set and the long-tail hard-to-distinguish text pair set. A staged fine-tuning mechanism is introduced to optimize the initial semantic understanding model in combination with the base class text pair set and the long-tail hard-to-distinguish text pair set to obtain the final semantic understanding model.

[0008] In one embodiment, traversing the text pair set to perform dropout enhancement and constructing a positive sample pair set and a negative sample pair set includes: extracting multiple text pairs from the text pair set according to a preset batch size, and constructing multiple training batches respectively; inputting the multiple training batches into the semantic coding model respectively, and enabling a random deactivation mechanism during the encoding process, performing forward encoding operations at least twice on the same text pair to obtain multiple semantic representation sets; defining the different semantic representations obtained by the same text pair in the multiple semantic representation sets through multiple Dropout encodings as the positive sample pair set; and defining the semantic representations generated by different text pairs within the same training batch as the negative sample pair set.

[0009] In one embodiment, an initial semantic understanding model is constructed by unsupervised pre-training of GTE-Large with the set of positive and negative sample pairs, and vertical fine-tuning using Simcseloss. This includes: semantically encoding the text in the set of positive and negative sample pairs using a GTE-Large pre-trained text encoding model, mapping the input text to a high-dimensional semantic vector representation. The semantic encoding process includes pooling the hidden states output by the encoding model and normalizing the resulting semantic vectors; training the semantic vectors using Simcseloss to ensure that the semantic vectors corresponding to the same positive sample pair remain similar in the semantic space, and that the semantic vectors corresponding to different negative sample pairs remain distinguishable in the semantic space; and updating the parameters of the GTE-Large pre-trained text encoding model through backpropagation to construct the initial semantic understanding model.

[0010] In one embodiment, performing long-tail hard-to-separate identification on the text pair set to determine the base class text pair set and the long-tail hard-to-separate text pair set includes: performing two-dimensional long-tail weighted identification based on the text pair set using query text and actual user-clicked product information to determine a first class of long-tail hard-to-separate text pair sets; performing hard-to-separate analysis on the query product information list based on the text pair set to determine a second class of long-tail hard-to-separate text pair sets; performing a union operation on the first class of long-tail hard-to-separate text pair sets and the second class of long-tail hard-to-separate text pair sets to obtain the long-tail hard-to-separate text pair set; removing the long-tail hard-to-separate text pair set from the text pair set, and adding the remaining text pairs to the base class text pair set.

[0011] In one embodiment, based on the text pair set, a difficult-to-segment analysis of the query product information list is performed to determine a second set of long-tail difficult-to-segment text pairs. This includes: traversing each text pair in the text pair set to calculate the pairwise semantic similarity of the product information within the query product information list, and calculating the mean of the calculation results to determine a set of mean semantic similarity values ​​for text pairs; traversing the set of mean semantic similarity values ​​for text pairs to perform mean drift filtering to determine the mean semantic similarity values ​​for filtered text pairs; comparing the mean semantic similarity values ​​for filtered text pairs with a preset mean semantic similarity threshold to determine a difficult-to-segment filtering standard; and adding text pairs whose semantic similarity values ​​in the set of mean semantic similarity values ​​are greater than or equal to the difficult-to-segment filtering standard to the second set of long-tail difficult-to-segment text pairs.

[0012] In one embodiment, comparing the mean semantic similarity of the selected text pairs with a preset mean semantic similarity threshold to determine the difficult-to-distinguish screening criterion includes: determining whether the mean semantic similarity of the selected text pairs is less than the preset mean semantic similarity threshold; if so, the mean semantic similarity of the selected text pairs is used as the difficult-to-distinguish screening criterion; if not, the preset mean semantic similarity threshold is used as the difficult-to-distinguish screening criterion.

[0013] In one embodiment, a phased fine-tuning mechanism is introduced to optimize the initial semantic understanding model by combining the base class text pair set and the long-tail hard-to-separate text pair set, thereby obtaining a semantic understanding model. This includes: introducing a cue vector mechanism into the semantic encoding module of the initial semantic understanding model to construct mutually independent base class cue vectors and long-tail hard-to-separate cue vectors; using the base class text pair set to perform base class fine-tuning on the initial semantic understanding model, freezing the long-tail hard-to-separate cue vectors during the fine-tuning process, and updating only the base class cue vectors and model parameters related to their semantic encoding process to obtain updated base class cue vectors and base class fine-tuned semantic understanding model parameters; based on the long-tail hard-to-separate text pair set... A training triplet set is constructed from the text pairs. Based on this training triplet set, the initial semantic understanding model is fine-tuned using long-tail triplets. During this fine-tuning process, the base class cue vector is frozen, and only the long-tail hard-to-separate cue vector and its semantic encoding-related model parameters are updated. This yields updated long-tail hard-to-separate cue vectors and long-tail triplet fine-tuned semantic understanding model parameters. After completing the base class cue fine-tuning and long-tail triplet fine-tuning, the initial semantic understanding model is optimized based on the updated base class cue vectors and base class fine-tuned semantic understanding model parameters, as well as the updated long-tail hard-to-separate cue vectors and long-tail triplet fine-tuned semantic understanding model parameters, to obtain an optimized semantic understanding model.

[0014] In one embodiment, after completing the base class cue fine-tuning and long-tail triplet fine-tuning, the initial semantic understanding model is optimized based on the updated base class cue vector and the base class fine-tuned semantic understanding model parameters, as well as the updated long-tail hard-to-distinguish cue vector and the long-tail triplet fine-tuned semantic understanding model parameters, to obtain an optimized semantic understanding model. This includes: using the updated base class cue vector and the base class fine-tuned semantic understanding model parameters as the base class parameter state; using the updated long-tail hard-to-distinguish cue vector and the long-tail triplet fine-tuned semantic understanding model parameters as the long-tail parameter state; and uniformly loading the base class parameter state and the long-tail parameter state into the initial semantic understanding model to take effect together, thereby obtaining the optimized semantic understanding model.

[0015] In one embodiment, the query text in each long-tail hard-to-divide text pair in the long-tail hard-to-divide text pair set is used as the anchor sample, the product information actually clicked by the user is used as the positive sample, and the product information in the query product information list other than the product information actually clicked by the user is used as the negative sample.

[0016] A second aspect of this application provides a semantic understanding model optimization system based on search click logs, the system comprising: The module for obtaining text pairs is used to acquire a set of historical search click logs, extract product information and query text, and obtain a set of text pairs. Each text pair includes the query text, a list of query product information, and information about the product actually clicked by the user. The module for enhancing text pairs is used to traverse the set of text pairs and perform dropout enhancement to construct a set of positive sample pairs and a set of negative sample pairs. The module for initializing the model is used to perform unsupervised pre-training using GTE-Large based on the set of positive and negative sample pairs, and to perform vertical fine-tuning using Simcseloss to build an initial semantic understanding model. The module for identifying long-tailed difficulties is used to identify long-tailed difficulties in the set of text pairs to determine the set of base class text pairs and the set of long-tailed difficult text pairs. The module for optimizing the model is used to introduce a phased fine-tuning mechanism and optimize the initial semantic understanding model in combination with the set of base class text pairs and the set of long-tailed difficult text pairs to obtain a semantic understanding model.

[0017] One or more technical solutions provided in this application have at least the following technical effects or advantages: This application obtains a set of historical search click logs, extracts product information and query text to obtain a set of text pairs. Each text pair includes the query text, a list of query product information, and information about the product actually clicked by the user. Then, dropout enhancement is applied to the text pair set to construct a positive sample pair set and a negative sample pair set. Next, GTE-Large is used for unsupervised pre-training based on the positive and negative sample pair sets, and Simcseloss is used for vertical fine-tuning to build an initial semantic understanding model. Then, long-tail hard-to-distinguish text pair identification is performed on the text pair set to determine the base class text pair set and the long-tail hard-to-distinguish text pair set. A staged fine-tuning mechanism is introduced, and the initial semantic understanding model is optimized by combining the base class text pair set and the long-tail hard-to-distinguish text pair set to obtain the final semantic understanding model. This significantly improves the overall semantic understanding capability of the semantic understanding model in search scenarios, especially optimizing low-frequency long-tail query scenarios. Attached Figure Description

[0018] Appendix Figure 1 This is a schematic diagram of the semantic understanding model optimization method based on search click logs provided in an embodiment of the present invention.

[0019] Appendix Figure 2 This is a schematic diagram of the semantic understanding model optimization system based on search click logs provided in an embodiment of the present invention.

[0020] The labels shown in the attached diagram: Text pair set acquisition module 11, text pair enhancement module 12, model initialization module 13, long-tail hard-to-distinguish recognition module 14, model optimization module 15. Detailed Implementation

[0021] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims. It should be noted that the terms "comprising" and "having" are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices.

[0022] Example 1, as shown in the appendix Figure 1 As shown, this application provides a method for optimizing a semantic understanding model based on search click logs, wherein the method includes: Step S100: Obtain the historical search click log set, perform product information and query text extraction, and obtain a text pair set, wherein each text pair includes the query text, the query product information list, and the product information actually clicked by the user; In one embodiment, historical search click logs refer to user behavior data continuously recorded during the operation of the search system, including the query text entered by the user, the product information displayed in the search results, and the user's actual click behavior. The query text represents the user's search intent, product information typically includes text fields such as product title, product attributes, and product description, the query product information list refers to the set of candidate products returned by the search system for the same query text, and the user's actual click on product information indicates the product object ultimately selected by the user from that candidate set. The text pair set refers to a data unit formed by combining the query text and its corresponding product information according to a preset structure, with the query text as the core, and is used for subsequent semantic modeling and training processing.

[0023] Specifically, based on time windows, such as the most recent week, three weeks, or one month, a collection of historical search click logs is retrieved from the search system's log storage module. From each log entry in this collection, the user-input query text and the list of product information returned by the search system for that query are extracted, further identifying the product information actually clicked by the user. Based on this, a set of text pairs is constructed for each query, where each text pair includes the query text, the corresponding list of candidate products, and the product information actually clicked by the user.

[0024] By constructing a set of text pairs, the system accurately reflects the matching relationship between user search behavior and products, providing a reliable data foundation for subsequent semantic processing, comparative learning, and identification of long-tailed, difficult-to-distinguish samples. This effectively avoids reliance on manual annotation and leverages implicit supervisory signals from search click logs. For example, a text pair might contain the query text "Men's-Black-Suit-Slim Fit," and the query product list might include: Men's Slim Fit Black Suit-Business Formal Wear; Men's Suit Set-Black-Classic Style; Casual Suit-Men's-Black-Loose Fit. The user would actually click on the product information "Men's Slim Fit Black Suit-Business Formal Wear."

[0025] Step S200: Iterate through the text pair set and perform dropout enhancement to construct a positive sample pair set and a negative sample pair set; Furthermore, by traversing the text pair set and performing dropout enhancement to construct a positive sample pair set and a negative sample pair set, step S200 of this embodiment further includes: Multiple text pairs are extracted from the text pair set according to a preset batch size, and multiple training batches are constructed respectively; The multiple training batches are input into the semantic coding model respectively, and a random deactivation mechanism is enabled during the coding process. At least two forward coding operations are performed on the same text pair to obtain multiple semantic representation sets. The different semantic representations of the same text pair obtained by multiple Dropout encodings in the multiple semantic representation sets are defined as the positive sample pair set; Within the same training batch, the semantic representations generated from different text pairs are defined as a set of negative sample pairs.

[0026] In one embodiment, a positive sample pair refers to a combination of text representations that should maintain a high degree of semantic consistency, specifically the semantic representations generated by the same text pair under different Dropout conditions. A negative sample pair refers to a combination of text representations that should be semantically distinct, specifically the combination relationship between the corresponding semantic representations of different text pairs within the same training batch. A training batch refers to the basic training unit that groups the set of text pairs according to a preset batch size during model training. The preset batch size is a batch size for a single training run pre-defined by those skilled in the art, such as 32, 64, etc.

[0027] For example, when the preset batch size is 4, it means that the number of text pairs trained in a single run is 4. Random samples are taken from the text pair set to form a batch containing text pairs A, B, C, and D. Then, this batch of text is input into the semantic encoding model. Dropout is enabled during encoding in the semantic encoding model, with a dropout rate of 0.1, and the same batch of data undergoes two (or more) independent forward propagations. Preferably, the semantic encoding model uses a lightweight version of the existing text encoding model, such as GTE-Base or a lightweight version of Sentence-BERT. Since this is only used for batch construction of positive and negative samples, a complex model structure is not required. Using a lightweight model can improve batch encoding speed and is compatible with the subsequent semantic encoding logic of GTE-Large, thereby avoiding semantic representation bias caused by cross-model architecture.

[0028] Due to the randomness of Dropout, during the first forward propagation of text A, some neurons are randomly dropped, resulting in vector A1; during the second forward propagation, another group of neurons is randomly dropped, resulting in vector A2. Vectors A1 and A2 originate from the same text and are highly semantically consistent, but due to slight differences in their internal representations, they constitute a positive sample pair. Similarly, we obtain B1 / B2, C1 / C2, and D1 / D2. Therefore, the set of positive sample pairs is {(A1, A2), (B1, B2), (C1, C2), (D1, D2)}.

[0029] Meanwhile, within the contrastive learning framework, any vector from different original texts within the same batch is considered a negative sample pair, such as A1 and B1, A1 and C2, B2 and D1, which together constitute a set of negative sample pairs. This constructs unsupervised training signals, generating the necessary anchor points—positive sample pairs and a massive number of negative sample pairs—for subsequent SimCSE unsupervised pre-training. Thus, the model can learn semantic similarity without any manual annotation, representing a key technological starting point for solving the problem of scarce labeled data.

[0030] Step S300: Use GTE-Large to perform unsupervised pre-training based on the positive sample pair set and the negative sample pair set, and use Simcseloss for vertical fine-tuning to build an initial semantic understanding model; Furthermore, using GTE-Large for unsupervised pre-training based on the positive sample pair set and the negative sample pair set, and using Simcseloss for vertical fine-tuning, an initial semantic understanding model is constructed. Step S300 in this embodiment further includes: The GTE-Large pre-trained text encoding model is used to perform semantic encoding on the text in the positive sample pair set and the negative sample pair set, mapping the input text into a high-dimensional semantic vector representation. The semantic encoding process includes pooling the hidden state output by the encoding model and normalizing the resulting semantic vector. Simcseloss is used to train semantic vectors so that the semantic vectors corresponding to the same positive sample pair remain similar in the semantic space, and the semantic vectors corresponding to different negative sample pairs remain distinguishable in the semantic space. The parameters of the GTE-Large pre-trained text encoding model are updated through backpropagation to construct an initial semantic understanding model.

[0031] In one embodiment, the GTE-Large pre-trained text encoding model, which is available to those skilled in the art, is invoked to semantically encode the text in the positive and negative sample pairs. During the encoding process, the GTE-Large Transformer encoder outputs multi-dimensional hidden states, with each layer outputting 1024-dimensional features. A total of 24 layers result in 24×1024-dimensional hidden states. Mean pooling is used to average all hidden states to obtain a 1024-dimensional initial semantic vector. Then, L2 normalization is applied to adjust the vector magnitude to 1. For example, after encoding, pooling, and normalization, the two semantic representations of a thin and light laptop in a positive sample pair yield vectors E and F. After processing, the semantic vector of a gaming laptop in a negative sample pair yields vector G.

[0032] Subsequently, using the Simcseloss technique with vector E as the anchor, the cosine similarity between E and the positive sample vector F is maximized (approaching 1), while the cosine similarity between E and the negative sample vector G is minimized (approaching 0). If the initial calculated similarity between E and F is 0.6 and between E and G is 0.4, the loss function will generate a corresponding loss value of 1.61. The Simcseloss unsupervised loss function is... , Let i be the vector of the i-th sample after the first semantic representation processing. Let be the cosine similarity between the vector of the i-th sample after the first semantic representation processing and the vector of the i-th sample after the j-th semantic representation processing. This is a temperature coefficient used to adjust the smoothness of the similarity distribution, typically set to 0.05~0.1. Finally, through backpropagation, the gradient of the loss value with respect to the GTE-Large model parameters, such as the Transformer layer weights and the word embedding matrix, is calculated. The parameters are iteratively updated along the gradient descent direction (the learning rate is set to 2e-5). After 100 rounds of training, the similarity of positive sample pairs is improved to above 0.9, and the similarity of negative sample pairs is reduced to below 0.1. Finally, an initial semantic understanding model with basic semantic matching capabilities is constructed.

[0033] Step S400: Perform long-tail hard-to-distinguish identification on the text pair set to determine the base class text pair set and the long-tail hard-to-distinguish text pair set; Furthermore, the text pair set is subjected to long-tail hard-to-distinguish identification to determine the base class text pair set and the long-tail hard-to-distinguish text pair set. In this embodiment, step S400 further includes: Based on the text pair set, a two-dimensional long-tail weighted recognition of query text and actual clicked product information by the user is performed to determine the first type of long-tail hard-to-distinguish text pair set; Based on the text pair set, a difficult-to-segment analysis of the query product information list is performed to determine the second type of long-tail difficult-to-segment text pair set; Perform a union operation on the first type of long-tailed hard-to-separate text pair set and the second type of long-tailed hard-to-separate text pair set to obtain the long-tailed hard-to-separate text pair set. Remove the long-tailed, hard-to-distinguish text pairs from the text pair set, and add the remaining text pairs to the base class text pair set.

[0034] Furthermore, based on the text pair set, a difficult-to-segment analysis of the query product information list is performed to determine the second type of long-tail difficult-to-segment text pair set. In this embodiment, step S400 further includes: Iterate through each text pair in the text pair set to calculate the semantic similarity of each pair of product information in the product information list, and calculate the mean of the calculation results to determine the set of mean semantic similarity of text pairs. The text is traversed and the semantic similarity mean set is filtered by mean shift to determine the semantic similarity mean of the selected text pairs; The mean semantic similarity of the selected text pairs is compared with a preset mean semantic similarity threshold to determine the difficulty-to-distinguish screening criteria; Text pairs whose semantic similarity is greater than or equal to the difficulty screening criteria are added to the second long-tail difficult-to-distinguish text pair set.

[0035] Furthermore, by comparing the mean semantic similarity of the selected text pairs with a preset mean semantic similarity threshold to determine the difficulty-to-distinguish screening criteria, step S400 of this embodiment further includes: Determine whether the average semantic similarity of the selected text pairs is less than the preset average semantic similarity threshold. If so, use the average semantic similarity of the selected text pairs as the screening criterion for difficulty in distinguishing them. If not, the preset semantic similarity mean threshold will be used as the screening criterion for difficulty in distinguishing between different semantic similarities.

[0036] Preferably, for each text pair in the text pair set, low-frequency long-tail identification is performed on the query text and the actual clicked product information from two dimensions: query text frequency and product click frequency. A weighted scoring formula is used to identify the low-frequency long-tail coefficient: score = 0.6 × query text low-frequency long-tail frequency + 0.4 × product information low-frequency long-tail frequency. When the low-frequency long-tail coefficient is less than or equal to a threshold preset by those skilled in the art, the corresponding text pair is added to the first type of long-tail difficult-to-distinguish text pair set.

[0037] Furthermore, by analyzing the difficulty of distinguishing the information in the list of query product information within each text pair set, a second type of long-tail difficult-to-distinguish text pair set is determined. Preferably, a first text pair and the list of query product information within the first text pair are extracted from the text pair set. The query product information in the list of query product information is then subjected to pairwise semantic similarity identification using a cosine similarity function to obtain the mean semantic similarity of the first text, which is then added to the set of mean semantic similarity of text pairs. Here, "first" refers to any element in the set, regardless of its order.

[0038] Since the set of average semantic similarity values ​​of text pairs reflects the degree of similarity of product information within the text pairs in the semantic understanding scenario, or the degree of difficulty in distinguishing them, because the higher the similarity, the higher the difficulty in semantic understanding and the higher the degree of difficulty in distinguishing them, by performing mean drift filtering on the set of average semantic similarity values ​​of the text pairs, the most common semantic similarity situation in the set is determined, thereby determining the general degree of semantic difficulty in understanding the text pairs.

[0039] Specifically, the mode is extracted from the set of semantic similarity mean values ​​of the text pairs. The extracted result is used as the initial mean drift screening center. Then, according to a preset distance bandwidth, the initial mean drift screening center is randomly iterated within the set of semantic similarity mean values ​​of the text pairs to determine the iterative mean drift screening center. This is because although the iterative mean drift screening center is the mode, it does not necessarily mean that there is a large amount of data surrounding it. There may be some data points that are small in number but surrounded by a large amount of data; in such cases, the mode alone cannot be used as a representative indicator.

[0040] The difference between the iterative mean drift filter center and the initial mean drift filter center is a preset distance bandwidth. The number of text pairs with semantic similarity mean values ​​within the preset distance bandwidth between the initial mean drift filter center and the iterative mean drift filter center is counted, thus obtaining the number of neighbors of the initial mean drift filter center and the number of neighbors of the iterative mean drift filter center.

[0041] When the number of neighbors of the initial mean drift screening center is greater than the number of neighbors of the iterative mean drift screening center, the direction from the iterative mean drift screening center to the initial mean drift screening center is taken as the first iteration direction. The initial mean drift screening center is iterated again according to the first iteration direction. If, after one iteration, the number of neighbors of the initial mean drift screening center is still greater than the number of neighbors of the iterative mean drift screening center obtained after the iteration, then the semantic similarity mean of the text pair corresponding to the initial mean drift screening center is taken as the semantic similarity mean of the selected text pair. If, after one iteration, the number of neighbors of the iterative mean drift screening center obtained by the iteration is greater than the number of neighbors of the initial mean drift screening center, then the iteration continues along the first iteration direction until the number of neighbors in the next iteration is less than or equal to the number of neighbors in the previous iteration. Then, the semantic similarity mean of the text pair corresponding to the iterative mean drift screening center obtained in the previous iteration is taken as the semantic similarity mean of the selected text pair.

[0042] When the number of neighbors of the initial mean drift screening center is less than or equal to the number of neighbors of the iterative mean drift screening center, the direction from the initial mean drift screening center to the iterative mean drift screening center is taken as the second iteration direction. The iterative mean drift screening center is iterated again according to the second iteration direction until the number of neighbors in the next iteration is less than or equal to the number of neighbors in the previous iteration. The semantic similarity mean of the text pair corresponding to the iterative mean drift screening center obtained in the previous iteration is taken as the semantic similarity mean of the screened text pair.

[0043] In the general case of determining the average semantic similarity of product information within a list of text pairs, that is, after filtering the average semantic similarity of text pairs, it is compared with a preset semantic similarity threshold set by those skilled in the art, such as 70%. This is to avoid a large deviation between the similarity situation in the current application scenario and the value set by those skilled in the art based on experience, which could lead to omissions or over-segmentation of long-tail, difficult-to-distinguish text pairs.

[0044] Determine whether the average semantic similarity of the selected text pairs is less than a preset threshold for average semantic similarity. If so, it indicates that the general semantic similarity of the text pairs is lower than the threshold set by those skilled in the art. In this case, the average semantic similarity of the selected text pairs should be used as the screening criterion for difficulty in distinguishing them. If not, the preset threshold for average semantic similarity should be used as the screening criterion for difficulty in distinguishing them.

[0045] By comparing the actual screening mean derived from the sample distribution with a manually preset threshold, the smaller of the two is taken as the final difficult-to-distinguish screening criterion. Using this as a boundary, text pairs within the list whose semantic similarity mean meets the standard are classified into the second category of long-tail difficult-to-distinguish sets. This approach combines the actual data distribution with domain experience to achieve accurate screening of difficult-to-distinguish samples. The semantic similarity mean of the selected text pairs is the difficult-to-distinguish boundary mined from the actual set of text pairs, conforming to the current data distribution. It reflects the actual semantic similarity of the queried product lists in the logs. The preset semantic similarity mean threshold is a fixed standard pre-set based on search domain experience, avoiding distortion of the screening criteria due to anomalous data distribution in a single instance. Taking the smaller of the two as the final difficult-to-distinguish screening criterion ensures that the screening criteria align with the actual data while maintaining the domain experience's minimum difficulty threshold through the preset threshold, preventing a large number of easily distinguishable samples from being misclassified as difficult samples due to an overly lenient standard.

[0046] Step S500: Introduce a phased fine-tuning mechanism to optimize the initial semantic understanding model by combining the base class text pair set and the long-tail hard-to-separate text pair set, and obtain the semantic understanding model.

[0047] Furthermore, a phased fine-tuning mechanism is introduced to optimize the initial semantic understanding model by combining the base class text pair set and the long-tail hard-to-separate text pair set, thereby obtaining a semantic understanding model. In this embodiment, step S500 further includes: A cue vector mechanism is introduced into the semantic encoding module of the initial semantic understanding model to construct mutually independent base class cue vectors and long-tailed cue vectors; The initial semantic understanding model is fine-tuned using the base class text pair set. During the fine-tuning process, the long-tailed hard-to-distinguish cue vector is frozen, and only the base class cue vector and the model parameters related to its semantic encoding process are updated to obtain the updated base class cue vector and the base class fine-tuned semantic understanding model parameters. Based on the set of long-tail hard-to-separate text pairs, a training triple set is constructed, and the initial semantic understanding model is fine-tuned using long-tail triples according to the training triple set. During the fine-tuning process, the base class cue vector is frozen, and only the long-tail hard-to-separate cue vector and the model parameters related to its semantic encoding process are updated to obtain the updated long-tail hard-to-separate cue vector and the long-tail triple fine-tuned semantic understanding model parameters. After completing the base class hint fine-tuning and long-tail triplet fine-tuning, the initial semantic understanding model is optimized based on the updated base class hint vector and base class fine-tuned semantic understanding model parameters, as well as the updated long-tail hard-to-distinguish hint vector and long-tail triplet fine-tuned semantic understanding model parameters, to obtain the optimized semantic understanding model.

[0048] Furthermore, the query text in each long-tail hard-to-separate text pair in the long-tail hard-to-separate text pair set is used as the anchor sample, the product information actually clicked by the user is used as the positive sample, and the product information in the query product information list other than the product information actually clicked by the user is used as the negative sample.

[0049] In one embodiment, the cue vector mechanism introduces a set of learnable vectors into the semantic encoding module. These vectors guide the model to focus on specific types of samples, base classes, or semantic features that are difficult to distinguish between long tails. Different types of samples correspond to independent cue vectors, avoiding mutual interference between feature learning. During the training and fine-tuning process, parameters are frozen, that is, the values ​​of specified parameters are fixed during the model fine-tuning process, so that they do not participate in backpropagation and parameter updates.

[0050] First, a cue vector mechanism is introduced into the semantic encoding module of the initial semantic understanding model, namely the Transformer layer of GTE-Large, to construct two sets of independent cue vectors with a dimension of 1024. The base class cue vector is initialized with a normal distribution N(0,0.02), and the long-tailed cue vector adopts the same initialization rule as the base class cue vector.

[0051] Then, base class fine-tuning is performed. The set of base class text pairs is input into the initial semantic understanding model, the long-tail hard-to-distinguish cue vector is frozen, and only the base class cue vector and the parameters of the top two Transformer layers of GTE-Large are updated (the learning rate is set to 1e-5). After 80 rounds of training, the base class cue vector is updated from the initial value [0.01,0.02,...,0.03] to [0.15,0.08,...,0.12]. At the same time, the parameters of the base class fine-tuned semantic understanding model are obtained. This stage allows the model to solidify its semantic matching ability for high-frequency and easy-to-distinguish samples.

[0052] Subsequently, a long-tail triplet set was constructed, using the query text in the long-tail hard-to-distinguish text pairs as anchor samples, the actual clicked product information as positive samples, and unclicked products in the query product list as negative samples, generating 1000 training triplet sets. Based on this set, long-tail triplet fine-tuning was carried out. The base class cue vector was frozen, and only the long-tail hard-to-distinguish cue vector and the parameters of the top 3 layers of the GTE-Large Transformer were updated (the learning rate was set to 2e-5 to adapt to the learning of hard-to-distinguish samples). After 150 rounds of training iterations, the long-tail hard-to-distinguish cue vector was updated to [0.21, 0.19, ..., 0.15], obtaining the fine-tuned long-tail triplet fine-tuned semantic understanding model parameters. At this stage, the model focuses on learning the semantic discrimination ability of long-tail hard-to-distinguish samples.

[0053] For example, the specific data constructed by the triple is as follows: anchor sample (query text): niche brand outdoor camping folding chair lightweight aluminum alloy; positive sample (product information actually clicked by the user): XX brand camping folding chair aluminum alloy frame lightweight 1.2kg; negative sample (product not clicked in the query product list): ordinary outdoor folding chair iron frame weight 3kg.

[0054] Finally, the updated base class hint vector and base class fine-tuning parameters, along with the long-tail hard-to-distinguish hint vector and long-tail triplet fine-tuning parameters, are loaded into the initial model so that both sets of parameters work together to obtain the optimized semantic understanding model.

[0055] By using a phased fine-tuning and independent cue vector design, the problem of mutual suppression of feature learning between base class samples and long-tail hard-to-separate samples is solved. This ensures the model's matching accuracy for high-frequency, easily separable scenarios while significantly improving the semantic understanding ability for low-frequency, long-tail, and hard-to-separate scenarios. This is the core part of the model optimization in the whole solution, directly building on the sample segmentation results mentioned above, and ultimately achieving an overall performance improvement of the semantic understanding model in search scenarios.

[0056] Furthermore, after completing the base class cue fine-tuning and long-tail triplet fine-tuning, the initial semantic understanding model is optimized based on the updated base class cue vector and base class fine-tuned semantic understanding model parameters, as well as the updated long-tail hard-to-distinguish cue vector and long-tail triplet fine-tuned semantic understanding model parameters, to obtain an optimized semantic understanding model. Step S500 in this embodiment further includes: The updated base class cue vector and the base class fine-tuned semantic understanding model parameters are used as the base class parameter state; The long-tailed parameter state is used to update the long-tailed hard-to-distinguish cue vector and the long-tailed triplet to fine-tune the semantic understanding model parameters. The base class parameter state and the long-tail parameter state are loaded into the initial semantic understanding model and take effect together to obtain the optimized semantic understanding model.

[0057] Preferably, the results of the two fine-tuning processes are first encapsulated into parameter states. The updated base class hint vector (e.g., a 1024-dimensional vector [0.15, 0.08, ..., 0.12]) obtained after base class fine-tuning and the fine-tuning parameters (including weight matrices, bias terms, etc.) of the top two Transformers of GTE-Large are integrated into the base class parameter state, and the parameter configuration for model adaptation to high-frequency easily distinguishable samples is recorded in this state. The updated long-tail difficult-to-distinguish hint vector (e.g., a 1024-dimensional vector [0.21, 0.19, ..., 0.15]) obtained after long-tail triplet fine-tuning and the fine-tuning parameters of the top three Transformers of GTE-Large are integrated into the long-tail parameter state, and the parameter configuration for model adaptation to low-frequency difficult-to-distinguish samples is recorded. Subsequently, a unified parameter loading operation is performed, and the two sets of parameter states are synchronously imported into the initial semantic understanding model. During loading, through the model parameter mapping mechanism, the base class hint vector and the long-tail difficult-to-distinguish hint vector exist independently and work collaboratively in the semantic encoding module, and the corresponding parameters adopt the values ​​of the two sets of states respectively.

[0058] For example, when processing the high-frequency query "Apple 15 phone", the base class hint vector guides the model to call the encoding logic of the base class parameter state, improving matching accuracy; when processing the long-tail query "niche retro film camera", the long-tail difficult-to-distinguish hint vector guides the model to call the encoding logic of the long-tail parameter state, strengthening semantic discrimination ability. Preferably, when calling the two types of hint vectors, the model first distinguishes the text pairs to be identified based on the same principle as identifying the long-tail difficult-to-distinguish text pairs to determine the base class text pairs and the long-tail difficult-to-distinguish text pairs, and then calls the corresponding hint vector.

[0059] This solves the problem of parameter fusion after phased fine-tuning, avoids parameter overlay caused by single-stage fine-tuning, achieves the collaborative preservation of the adaptability of base class samples and long-tail hard-to-distinguish samples, and ultimately allows the optimized model to take into account the semantic understanding effect of both types of scenarios, completes the closed loop of the entire model optimization chain, and directly achieves the goal of improving the semantic matching accuracy in search scenarios, especially in low-frequency long-tail scenarios.

[0060] Example 2, based on the same inventive concept as the semantic understanding model optimization method based on search click logs in the foregoing examples, as shown in the appendix. Figure 2 As shown, this application provides a semantic understanding model optimization system based on search click logs. The system and method embodiments in this application are based on the same inventive concept. The system includes: The text pair set acquisition module 11 is used to acquire a historical search click log set, perform product information and query text extraction, and obtain a text pair set, wherein each text pair includes the query text, a list of query product information, and the product information actually clicked by the user; Text pair enhancement module 12 is used to traverse the text pair set to perform dropout enhancement and construct a positive sample pair set and a negative sample pair set; The model initialization module 13 is used to perform unsupervised pre-training based on the positive sample pair set and the negative sample pair set using GTE-Large, and to perform vertical fine-tuning using Simcseloss to build an initial semantic understanding model. The long-tail hard-to-distinguish identification module 14 is used to perform long-tail hard-to-distinguish identification on the text pair set and determine the base class text pair set and the long-tail hard-to-distinguish text pair set. The model optimization module 15 is used to introduce a phased fine-tuning mechanism to optimize the initial semantic understanding model by combining the base class text pair set and the long-tail hard-to-separate text pair set, thereby obtaining a semantic understanding model.

[0061] Furthermore, the text enhancement module 12 performs the following steps: Multiple text pairs are extracted from the text pair set according to a preset batch size, and multiple training batches are constructed respectively; The multiple training batches are input into the semantic coding model respectively, and a random deactivation mechanism is enabled during the coding process. At least two forward coding operations are performed on the same text pair to obtain multiple semantic representation sets. The different semantic representations of the same text pair obtained by multiple Dropout encodings in the multiple semantic representation sets are defined as the positive sample pair set; Within the same training batch, the semantic representations generated from different text pairs are defined as a set of negative sample pairs.

[0062] Furthermore, the model initialization module 13 is used to perform the following steps: The GTE-Large pre-trained text encoding model is used to perform semantic encoding on the text in the positive sample pair set and the negative sample pair set, mapping the input text into a high-dimensional semantic vector representation. The semantic encoding process includes pooling the hidden state output by the encoding model and normalizing the resulting semantic vector. Simcseloss is used to train semantic vectors so that the semantic vectors corresponding to the same positive sample pair remain similar in the semantic space, and the semantic vectors corresponding to different negative sample pairs remain distinguishable in the semantic space. The parameters of the GTE-Large pre-trained text encoding model are updated through backpropagation to construct an initial semantic understanding model.

[0063] Furthermore, the long-tail hard-to-distinguish identification module 14 is used to perform the following steps: Based on the text pair set, a two-dimensional long-tail weighted recognition of query text and actual clicked product information by the user is performed to determine the first type of long-tail hard-to-distinguish text pair set; Based on the text pair set, a difficult-to-segment analysis of the query product information list is performed to determine the second type of long-tail difficult-to-segment text pair set; Perform a union operation on the first type of long-tailed hard-to-separate text pair set and the second type of long-tailed hard-to-separate text pair set to obtain the long-tailed hard-to-separate text pair set. Remove the long-tailed, hard-to-distinguish text pairs from the text pair set, and add the remaining text pairs to the base class text pair set.

[0064] Furthermore, the long-tail hard-to-distinguish identification module 14 is used to perform the following steps: Iterate through each text pair in the text pair set to calculate the semantic similarity of each pair of product information in the product information list, and calculate the mean of the calculation results to determine the set of mean semantic similarity of text pairs. The text is traversed and the semantic similarity mean set is filtered by mean shift to determine the semantic similarity mean of the selected text pairs; The mean semantic similarity of the selected text pairs is compared with a preset mean semantic similarity threshold to determine the difficulty-to-distinguish screening criteria; Text pairs whose semantic similarity is greater than or equal to the difficulty screening criteria are added to the second long-tail difficult-to-distinguish text pair set.

[0065] Furthermore, the long-tail hard-to-distinguish identification module 14 is used to perform the following steps: Determine whether the average semantic similarity of the selected text pairs is less than the preset average semantic similarity threshold. If so, use the average semantic similarity of the selected text pairs as the screening criterion for difficulty in distinguishing them. If not, the preset semantic similarity mean threshold will be used as the screening criterion for difficulty in distinguishing between different semantic similarities.

[0066] Furthermore, the model optimization module 15 is used to perform the following steps: A cue vector mechanism is introduced into the semantic encoding module of the initial semantic understanding model to construct mutually independent base class cue vectors and long-tailed cue vectors; The initial semantic understanding model is fine-tuned using the base class text pair set. During the fine-tuning process, the long-tailed hard-to-distinguish cue vector is frozen, and only the base class cue vector and the model parameters related to its semantic encoding process are updated to obtain the updated base class cue vector and the base class fine-tuned semantic understanding model parameters. Based on the set of long-tail hard-to-separate text pairs, a training triple set is constructed, and the initial semantic understanding model is fine-tuned using long-tail triples according to the training triple set. During the fine-tuning process, the base class cue vector is frozen, and only the long-tail hard-to-separate cue vector and the model parameters related to its semantic encoding process are updated to obtain the updated long-tail hard-to-separate cue vector and the long-tail triple fine-tuned semantic understanding model parameters. After completing the base class hint fine-tuning and long-tail triplet fine-tuning, the initial semantic understanding model is optimized based on the updated base class hint vector and base class fine-tuned semantic understanding model parameters, as well as the updated long-tail hard-to-distinguish hint vector and long-tail triplet fine-tuned semantic understanding model parameters, to obtain the optimized semantic understanding model.

[0067] Furthermore, the model optimization module 15 is used to perform the following steps: The updated base class cue vector and the base class fine-tuned semantic understanding model parameters are used as the base class parameter state; The long-tailed parameter state is used to update the long-tailed hard-to-distinguish cue vector and the long-tailed triplet to fine-tune the semantic understanding model parameters. The base class parameter state and the long-tail parameter state are loaded into the initial semantic understanding model and take effect together to obtain the optimized semantic understanding model.

[0068] Furthermore, the query text in each long-tail hard-to-separate text pair in the long-tail hard-to-separate text pair set is used as the anchor sample, the product information actually clicked by the user is used as the positive sample, and the product information in the query product information list other than the product information actually clicked by the user is used as the negative sample.

[0069] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0070] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0071] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A method for optimizing a semantic understanding model based on search click logs, characterized in that, The method includes: Obtain a collection of historical search click logs, perform product information and query text extraction to obtain a collection of text pairs, where each text pair includes the query text, a list of query product information, and the product information actually clicked by the user; The text pair set is traversed and dropout enhancement is performed to construct a positive sample pair set and a negative sample pair set. The initial semantic understanding model is constructed by using GTE-Large for unsupervised pre-training based on the positive sample pair set and the negative sample pair set, and then using Simcseloss for vertical fine-tuning. Perform long-tail hard-to-distinguish identification on the text pair set to determine the base class text pair set and the long-tail hard-to-distinguish text pair set; A phased fine-tuning mechanism is introduced to optimize the initial semantic understanding model by combining the base class text pair set and the long-tail hard-to-separate text pair set, thereby obtaining a semantic understanding model.

2. The semantic understanding model optimization method based on search click logs as described in claim 1, characterized in that, Iterate through the text pair set and perform dropout enhancement to construct a positive sample pair set and a negative sample pair set, including: Multiple text pairs are extracted from the text pair set according to a preset batch size, and multiple training batches are constructed respectively; The multiple training batches are input into the semantic coding model respectively, and a random deactivation mechanism is enabled during the coding process. At least two forward coding operations are performed on the same text pair to obtain multiple semantic representation sets. The different semantic representations of the same text pair obtained by multiple Dropout encodings in the multiple semantic representation sets are defined as the positive sample pair set; Within the same training batch, the semantic representations generated from different text pairs are defined as a set of negative sample pairs.

3. The semantic understanding model optimization method based on search click logs as described in claim 1, characterized in that, An initial semantic understanding model is constructed by unsupervised pre-training using GTE-Large on the positive and negative sample pairs, followed by vertical fine-tuning using Simcseloss, including: The GTE-Large pre-trained text encoding model is used to perform semantic encoding on the text in the positive sample pair set and the negative sample pair set, mapping the input text into a high-dimensional semantic vector representation. The semantic encoding process includes pooling the hidden state output by the encoding model and normalizing the resulting semantic vector. Simcseloss is used to train semantic vectors so that the semantic vectors corresponding to the same positive sample pair remain similar in the semantic space, and the semantic vectors corresponding to different negative sample pairs remain distinguishable in the semantic space. The parameters of the GTE-Large pre-trained text encoding model are updated through backpropagation to construct an initial semantic understanding model.

4. The semantic understanding model optimization method based on search click logs as described in claim 1, characterized in that, Perform long-tail hard-to-separate identification on the text pair set to determine the base class text pair set and the long-tail hard-to-separate text pair set, including: Based on the text pair set, a two-dimensional long-tail weighted recognition of query text and actual clicked product information by the user is performed to determine the first type of long-tail hard-to-distinguish text pair set; Based on the text pair set, a difficult-to-segment analysis of the query product information list is performed to determine the second type of long-tail difficult-to-segment text pair set; Perform a union operation on the first type of long-tailed hard-to-separate text pair set and the second type of long-tailed hard-to-separate text pair set to obtain the long-tailed hard-to-separate text pair set. Remove the long-tailed, hard-to-distinguish text pairs from the text pair set, and add the remaining text pairs to the base class text pair set.

5. The semantic understanding model optimization method based on search click logs as described in claim 4, characterized in that, Based on the text pair set, a difficulty analysis of querying product information lists was performed to identify the second type of long-tail difficult-to-segment text pair set, including: Iterate through each text pair in the text pair set to calculate the semantic similarity of each pair of product information in the product information list, and calculate the mean of the calculation results to determine the set of mean semantic similarity of text pairs. The text is traversed and the semantic similarity mean set is filtered by mean shift to determine the semantic similarity mean of the selected text pairs; The mean semantic similarity of the selected text pairs is compared with a preset mean semantic similarity threshold to determine the difficulty-to-distinguish screening criteria; Text pairs whose semantic similarity is greater than or equal to the difficulty screening criteria are added to the second long-tail difficult-to-distinguish text pair set.

6. The semantic understanding model optimization method based on search click logs as described in claim 5, characterized in that, The mean semantic similarity of the selected text pairs is compared with a preset mean semantic similarity threshold to determine the difficulty-to-distinguish screening criteria, including: Determine whether the average semantic similarity of the selected text pairs is less than the preset average semantic similarity threshold. If so, use the average semantic similarity of the selected text pairs as the screening criterion for difficulty in distinguishing them. If not, the preset semantic similarity mean threshold will be used as the screening criterion for difficulty in distinguishing between different semantic similarities.

7. The semantic understanding model optimization method based on search click logs as described in claim 1, characterized in that, A phased fine-tuning mechanism is introduced to optimize the initial semantic understanding model by combining the base class text pair set and the long-tail hard-to-separate text pair set, thereby obtaining a semantic understanding model, including: A cue vector mechanism is introduced into the semantic encoding module of the initial semantic understanding model to construct mutually independent base class cue vectors and long-tailed cue vectors; The initial semantic understanding model is fine-tuned using the base class text pair set. During the fine-tuning process, the long-tailed hard-to-distinguish cue vector is frozen, and only the base class cue vector and the model parameters related to its semantic encoding process are updated to obtain the updated base class cue vector and the base class fine-tuned semantic understanding model parameters. Based on the set of long-tail hard-to-separate text pairs, a training triple set is constructed, and the initial semantic understanding model is fine-tuned using long-tail triples according to the training triple set. During the fine-tuning process, the base class cue vector is frozen, and only the long-tail hard-to-separate cue vector and the model parameters related to its semantic encoding process are updated to obtain the updated long-tail hard-to-separate cue vector and the long-tail triple fine-tuned semantic understanding model parameters. After completing the base class hint fine-tuning and long-tail triplet fine-tuning, the initial semantic understanding model is optimized based on the updated base class hint vector and base class fine-tuned semantic understanding model parameters, as well as the updated long-tail hard-to-distinguish hint vector and long-tail triplet fine-tuned semantic understanding model parameters, to obtain the optimized semantic understanding model.

8. The semantic understanding model optimization method based on search click logs as described in claim 7, characterized in that, After completing the base class cue fine-tuning and long-tail triplet fine-tuning, the initial semantic understanding model is optimized based on the updated base class cue vector and the base class fine-tuned semantic understanding model parameters, as well as the updated long-tail hard-to-distinguish cue vector and long-tail triplet fine-tuned semantic understanding model parameters, to obtain an optimized semantic understanding model, including: The updated base class cue vector and the base class fine-tuned semantic understanding model parameters are used as the base class parameter state; The long-tailed parameter state is used to update the long-tailed hard-to-distinguish cue vector and the long-tailed triplet to fine-tune the semantic understanding model parameters. The base class parameter state and the long-tail parameter state are loaded into the initial semantic understanding model and take effect together to obtain the optimized semantic understanding model.

9. The semantic understanding model optimization method based on search click logs as described in claim 8, characterized in that, The query text in each long-tail hard-to-separate text pair in the set of long-tail hard-to-separate text pairs is used as the anchor sample, the product information actually clicked by the user is used as the positive sample, and the product information in the query product information list other than the product information actually clicked by the user is used as the negative sample.

10. A semantic understanding model optimization system based on search click logs, characterized in that, The system is used to implement the semantic understanding model optimization method based on search click logs as described in any one of claims 1-9, and the system comprises: The text pair set acquisition module is used to acquire a historical search click log set, perform product information and query text extraction, and obtain a text pair set, wherein each text pair includes the query text, a list of query product information, and the product information actually clicked by the user; The text pair enhancement module is used to traverse the text pair set and perform dropout enhancement to construct a positive sample pair set and a negative sample pair set. The model initialization module is used to perform unsupervised pre-training based on the positive sample pair set and the negative sample pair set using GTE-Large, and to perform vertical fine-tuning using Simcseloss to build an initial semantic understanding model. The long-tail hard-to-distinguish recognition module is used to perform long-tail hard-to-distinguish recognition on the text pair set and determine the base class text pair set and the long-tail hard-to-distinguish text pair set. The model optimization module is used to introduce a phased fine-tuning mechanism, which combines the base class text pair set and the long-tail hard-to-separate text pair set to optimize the initial semantic understanding model and obtain the semantic understanding model.

Images