Search result classification model training method, device, medium and equipment

Abstract

The present application discloses a training method, device, medium and equipment for a search result classification model, which relates to the field of artificial intelligence. The method includes: obtaining a training data set, the training data set includes search query terms and search results corresponding to the search query terms; determining the similarity between the search query terms and the search results based on a target text matching algorithm; determining a classification label for a sample based on a preset classification interval and the similarity, the sample including the search query terms and the search results; fine-tuning the pre-trained language model based on the sample and the corresponding classification label to obtain a search result classification model. The search result classification model obtained based on the method provided by the present application can make the obtained search results more matched with the search query terms in the similarity dimension, improve the search speed, and meet the user's search needs.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a training method, device, medium and equipment for a search result classification model. Background Art

[0002] Artificial Intelligence (AI) is a comprehensive technology in computer science. By studying the design principles and implementation methods of various intelligent machines, it enables machines to have the functions of perception, reasoning and decision-making. Artificial Intelligence technology is a comprehensive discipline that covers a wide range of fields, such as natural language processing, machine learning, deep learning and other major directions. With the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0003] With the development of word vector embedding representation technology, text pre-training models are increasingly becoming the basic modules for many natural language processing tasks and their upstream application scenarios. For example, in search engines, the search query terms entered by users and the content of candidate search results are embedded through pre-trained language models, and the correlation between the search query terms and candidate search results is determined based on the similarity between the vectors. At present, this language model can better capture the relationship between syntactic and semantic vocabulary, but it does not meet the requirement of matching the similarity between search query terms and search results in search scenarios. Summary of the invention

[0004] In order to meet the demand for a better match between search results and search query terms, the present application provides a method, device, medium and equipment for training a search result classification model. The technical solution is as follows:

[0005] In a first aspect, the present application provides a method for training a search result classification model, the method comprising:

[0006] Acquire a training data set, the training data set including a search query term and search results corresponding to the search query term;

[0007] Determining the similarity between the search query and the search result based on a target text matching algorithm;

[0008] Determining a classification label of a sample according to a preset classification interval and the similarity, the sample including the search query term and the search result;

[0009] The pre-trained language model is fine-tuned according to the samples and the corresponding classification labels to obtain a search result classification model.

[0010] In a second aspect, the present application provides a training device for a search result classification model, the device comprising:

[0011] A data acquisition module, configured to acquire a training data set, wherein the training data set includes a search query term and search results corresponding to the search query term;

[0012] A similarity calculation module, used to determine the similarity between the search query and the search result based on a target text matching algorithm;

[0013] A classification label determination module, used to determine a classification label of a sample according to a preset classification interval and the similarity, wherein the sample includes the search query term and the search result;

[0014] The model fine-tuning module is used to fine-tune the pre-trained language model according to the sample and the corresponding classification label to obtain a search result classification model.

[0015] In a third aspect, the present application provides a computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement a training method for a search result classification model as described in the first aspect.

[0016] In a fourth aspect, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement a training method for a search result classification model as described in the first aspect.

[0017] In a fifth aspect, the present application provides a computer program product, which includes computer instructions, and when the computer instructions are executed by a processor, implements a training method for a search result classification model as described in the first aspect.

[0018] The training method, device, medium and equipment of the search result classification model provided by this application have the following technical effects:

[0019] The solution provided in this application calculates the similarity between the search query and the search results through a text matching algorithm, and determines the classification label of the sample based on the similarity and the preset classification interval, the sample includes the search query and the corresponding search result; then the existing pre-trained language model is fine-tuned through the sample and the classification label of the sample to obtain the search result classification model, and the search result classification model can be applied to the search application, so that the obtained search results are more matched with the search query in the similarity dimension, further meeting the user's search needs and improving the user experience. Specifically, the BM25 formula is rewritten in the text matching algorithm, factors such as morpheme coverage are added, and the calculation of morpheme weights and morpheme similarity with search results is improved, so that the text matching algorithm pays more attention to the similarity between the search query and the search results in content to meet the model application requirements. In addition, compared to training the model from scratch, the training method of fine-tuning the pre-trained language model can reduce the amount of training data required, save the model training cost, and improve the model training efficiency.

[0020] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 It is a schematic diagram of an implementation environment of a semantic model training method provided in an embodiment of the present application;

[0023] Figure 2 It is a flowchart of a semantic model training method provided in an embodiment of the present application;

[0024] Figure 3 It is a schematic diagram of a process for determining the similarity between a search query term and a search result provided by an embodiment of the present application;

[0025] Figure 4 is a flow chart of determining the similarity between a morpheme and a search result provided in an embodiment of the present application;

[0026] Figure 5 is a schematic diagram of a flow chart of determining the weight of a morpheme provided in an embodiment of the present application;

[0027] Figure 6It is another flow chart of determining the similarity between a search query term and a search result provided by an embodiment of the present application;

[0028] Figure 7 It is a schematic diagram of a process of fine-tuning a language model provided in an embodiment of the present application;

[0029] Figure 8 It is a schematic diagram of a flow chart of determining text features provided by an embodiment of the present application;

[0030] Fig. 9 is another flow chart of fine-tuning a language model provided in an embodiment of the present application;

[0031] Fig.10 is a schematic diagram of a training device for a search result classification model provided in an embodiment of the present application;

[0032] Fig.11 It is a schematic diagram of the hardware structure of a device for implementing a training method for a search result classification model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is an interdisciplinary subject that covers a wide range of fields, including both hardware-level technology and software-level technology. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics.

[0034] The solution provided in the embodiments of the present application involves technologies such as deep learning (DL) of artificial intelligence and natural language processing (NLP).

[0035] Deep learning (DL) is a major research direction in the field of machine learning (ML). It is introduced into machine learning to make it closer to its original goal - artificial intelligence. Deep learning is the inherent laws and representation levels of learning sample data. The information obtained in the learning process is of great help in the interpretation of data such as text, images and sounds. Its ultimate goal is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images and sounds. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition that far exceed previous related technologies. Deep learning has achieved many results in search technology, data mining, machine learning, machine translation, natural language processing, multimedia learning, speech, recommendation and personalization technology, and other related fields. Deep learning enables machines to imitate human activities such as seeing, hearing and thinking, solves many complex pattern recognition problems, and makes great progress in artificial intelligence related technologies.

[0036] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0037] The solution provided in the embodiment of the present application can be deployed in the cloud, which also involves cloud technology, etc.

[0038] Cloud technology: refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data computing, storage, processing and sharing. It can also be understood as a general term for network technology, information technology, integration technology, management platform technology and application technology based on the cloud computing business model application, which can form a resource pool and be used on demand, flexible and convenient. The back-end services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the back-end system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, so cloud technology needs to be supported by cloud computing. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use. As a cloud computing basic capability provider, a cloud computing resource pool platform will be established, referred to as a cloud platform, generally referred to as Infrastructure as a Service (IaaS), and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (which can be virtualized machines, including operating systems), storage devices, and network devices.

[0039] In order to meet the needs of matching the similarity between search results and search query terms, embodiments of the present application provide a training method, device, medium and equipment for a search result classification model. The technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Examples of the embodiments are shown in the drawings, where the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] In order to facilitate understanding of the technical solutions described in the embodiments of the present application and the technical effects produced therefrom, the embodiments of the present application explain the relevant professional terms involved:

[0042] BERT: Bidirectional Encoder Representation from Transformers, bidirectional encoding representation based on transformer; it is a pre-trained language representation model.

[0043] Word2vec: Word to vector, is a group of related models used to generate word vectors, such as the Bag-of-words model and the Skip-gram model. These models are shallow, two-layer neural networks that are trained to reconstruct linguistic word texts. After training, the word2vec model can be used to map each word to a vector, which can be used to represent the relationship between words.

[0044] TF-IDF: Term Frequency–Inverse Document Frequency, is a commonly used weighting technique for information retrieval and data mining. TF (Term Frequency) is the term frequency, and IDF (Inverse Document Frequency) is the inverse document frequency index.

[0045] BM25: (BM, Best Matching), originated from the probability correlation model, is an improved text matching algorithm based on TF-IDF and is known as the next generation of TF-IDF.

[0046] Fine-tuning: It can also be called fine-tuning, which means adding a small number of task-specific parameters to the already trained language model. For example, for classification problems, a softmax network is added to the language model, and then retraining is done on the new corpus to complete fine-tuning.

[0047] See also Figure 1 , which is a schematic diagram of an implementation environment of a training method for a search result classification model provided in an embodiment of the present application, such as Figure 1 As shown, the implementation environment may include at least a client 01 and a server 02 .

[0048] Specifically, the client 01 may include devices such as smart phones, desktop computers, tablet computers, laptop computers, vehicle terminals, digital assistants, smart wearable devices, monitoring devices, and voice interaction devices, and may also include software running in the device, such as web pages provided to users by some service providers, or applications provided to users by these service providers. Specifically, the client 01 may be used to collect the required training data set for the search application, and the training data set may include search query terms and search results corresponding to the search query terms, which may be obtained from historical search logs.

[0049] Specifically, the server 02 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 02 may include a network communication unit, a processor, and a memory, etc. The terminal and the server may be directly or indirectly connected via wired or wireless communication, and this application does not limit this. Specifically, the server 02 may be used to respond to a request from the client 01, obtain a training data set, and calculate the similarity between the search query term and the search result based on a preset target text matching algorithm, and classify the search results according to the similarity, thereby determining the classification label of the sample, which is a combination of the search query term and the search result, and finally fine-tuning the pre-trained language model through the sample and the classification label of the sample to obtain a search result classification model that meets the similarity requirements in the search scenario.

[0050] The embodiments of the present application can also be implemented in combination with cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software and network in a wide area network or a local area network to realize the calculation, storage, processing and sharing of data. It can also be understood as a general term for network technology, information technology, integration technology, management platform technology and application technology based on the cloud computing business model application. Cloud technology needs to be supported by cloud computing. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". Specifically, the server 02 and the database are located in the cloud, and the server 02 can be a physical machine or a virtualized machine.

[0051] The following introduces a training method for a search result classification model provided by this application. Figure 2 This is a flowchart of a method for training a search result classification model provided in an embodiment of the present application. The present application provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment) according to the method shown in the embodiment or the accompanying drawings. Please refer to Figure 2 A method for training a search result classification model provided in an embodiment of the present application may include the following steps:

[0052] S210: Acquire a training data set, where the training data set includes a search query term and search results corresponding to the search query term.

[0053] It is understandable that there may be multiple search results corresponding to the search query, and the multiple search results constitute the search result set corresponding to the search query. In the embodiment of the present application, the search query and a corresponding search result are used as a training sample, and the data used for training are all historical data.

[0054] S230: Determine the similarity between the search query term and the search result based on a target text matching algorithm.

[0055] It is understandable that the current unsupervised pre-trained language model is essentially a word embedding technology based on neural networks, which can better capture the relationship between syntax and semantic vocabulary. The pre-trained model learns the vector representation of words through context, which is actually to obtain which words often appear in the relevant context with the current word. For example, the two words "Beijing" and "Shanghai" often appear together in the context of massive corpus, so the vector similarity expressed by their pre-trained model will be marked high. There is no problem in understanding from a semantic perspective, but in the search scenario, if a user searches for "Beijing", the search results related to "Shanghai" are only related but not similar, that is, related ≠ similar. From the above analysis, the existing pre-trained language model emphasizes the contextual relevance of words, and is not suitable for search scenarios that emphasize the similarity matching of query (search query) and doc (document, or extended to search results).

[0056] In the embodiment of the present application, the classification label of the training sample is determined by calculating the similarity between the search query and the search results, so as to adjust the pre-trained language model to meet the needs of the search scenario. Among them, the so-called similarity is not the same as the relevance. The similarity in this application emphasizes the similarity of the content itself. For example, if the search query is "Beijing", the search results that need to be displayed are all information related to Beijing.

[0057] In an embodiment of the present application, the target text matching algorithm is used to calculate the similarity between the search term and the search result in text content. Specifically, the target text matching algorithm can be any one of the traditional character matching rules, BM25 algorithm, Jaccard similarity algorithm, cosine similarity algorithm, etc.

[0058] Exemplarily, traditional character matching rules can be used to calculate the similarity between the search query term and the search result. Specifically, if the search result is a web page, article, etc., the similarity between the search query term and the text content of the search result can be calculated, and the text content can be a title or text; if the search result is a media resource such as a picture, audio, video, etc., the similarity between the search query term and the title of the search result can be calculated.

[0059] For example, considering that in the scenario of calculating the similarity between search query words and titles, the title length is often short, and the weights of the inverse document frequency values ​​of different hit words are different, the BM25 algorithm can be used to calculate the similarity. The BM25 algorithm is usually used for search relevance scoring. Its main idea is to perform morpheme analysis on the query (search query word) and generate morpheme q i , then for the search result D, calculate each morpheme q i The correlation score with D, finally, q iThe weighted sum of the correlation scores with D is performed to obtain the correlation score between Query and D. Specifically, the commonly used BM25 formula is shown in formula (1):

[0060]

[0061] In formula (1), Q represents query, i.e., the search query term input by the user; S represents document, which is the title of the current document in this application. In other application scenarios such as video search and image search, S can also represent the title of the search result; IDF(q i ) represents the i-th word q in Q i The inverse document frequency value of is used as q in formula (1) i The weight corresponding to the relevance score of D; TF(qi) represents the i-th word q in query Q i word frequency; length(S) represents the length of the current document title, avglength(S) represents the average length of the corresponding multiple document titles; the parameter b is mainly used to adjust the impact of the text title length on the similarity, k is the adjustment factor for word frequency, such as k can be set to 2 and b can be set to 0.75.

[0062] Based on the above formula (1), the BM25 algorithm is improved by adding factors such as morpheme coverage, and improving the calculation of morpheme weight and the similarity between morphemes and search results. The morpheme coverage can represent the degree of overlap in content, so that the calculation results can better reflect similarity rather than correlation.

[0063] In one embodiment of the present application, specifically, Figure 3 As shown, step S230 may include the following steps:

[0064] S310: Perform morpheme analysis on the search query to determine one or more target morphemes in the search query.

[0065] Specifically, the morpheme analysis of the search query is generally performed by word segmentation to obtain a sequence of target morphemes q1, q2, ... q n In the embodiment of the present application, the target morpheme is also referred to as a hit word.

[0066] In an exemplary embodiment, for English text, spaces can be used to separate text. For Chinese text, there are two main methods: dictionary-based word segmentation algorithm and statistical-based word segmentation algorithm. The dictionary-based word segmentation algorithm is essentially a string matching algorithm. Based on a certain matching strategy, the Chinese text to be matched is matched with a dictionary. If the match is successful, the word segmentation can be performed according to the matching result. The statistical-based word segmentation algorithm is essentially a sequence labeling problem. The word is labeled according to its position in the text based on statistical probability to obtain the sequence labeling result of the Chinese text, and then the Chinese text is segmented according to the sequence labeling result.

[0067] S330: Determine the target similarity between each of the target morphemes and the search results based on a target text matching algorithm.

[0068] In a possible implementation, Figure 4 As shown, in the case where the search query word corresponds to multiple search results, step S330 may include the following steps:

[0069] S331: Determine the word frequency of the target morpheme in multiple search results.

[0070] Specifically, multiple search results can be used as a search result set, and the word frequency represents the number of times the target morpheme appears in the title of each search result in the search result set. In addition, if the similarity calculation is performed for the target morpheme and the text content of the search result (such as a document), the word frequency can be the frequency of the target morpheme appearing in a single search result. The calculation formula for the word frequency can be defined according to the specific application category or business demand adaptability, which will not be elaborated here.

[0071] S333: Determine a first word quantity of the search query term and a second word quantity of the title of the search result; the search result is any one of the multiple search results.

[0072] Specifically, the number of target morphemes in the search query is taken as the first word quantity, and the title of the search result is also segmented, and the number of words obtained is taken as the second word quantity.

[0073] S335: Obtain a target similarity between the target morpheme and the search result according to the word frequency, the first word quantity, and the second word quantity.

[0074] Exemplarily, the target similarity calculation between the target morpheme qi and the search result S can be shown as formula (2):

[0075]

[0076] Among them, TF(q i) represents the frequency of the target morpheme qi in multiple search results, Length(Q) represents the number of words in the query after word segmentation, that is, the first word quantity, Length(S) represents the number of title words in the search result S, that is, the second word quantity, the parameter b is introduced to adjust the effect of the title length of S on the target similarity, and the parameter k is introduced to adjust the word frequency TF(q i ) on the target similarity, such as setting k to 2 and b to 0.75.

[0077] S350: Determine a normalized morpheme weight of each target morpheme.

[0078] In a feasible implementation, the inverse document frequency value of the target morpheme is used to calculate its weight, and the weight is normalized so that the value can fall within a relative range. Specifically, when the search query word corresponds to multiple search results, such as Figure 5 As shown, step S350 may include the following steps:

[0079] S351: Determine the inverse document frequency value of each of the target morphemes in the multiple search results.

[0080] Specifically, taking the search result as a document as an example, the calculation formula of IDF can be shown as formula (3):

[0081]

[0082] Where N represents the number of search results in the search result set, n(q i ) is the number of search results in which the target morpheme appears in the title.

[0083] Optionally, the calculation formula of IDF can also be shown as formula (4):

[0084]

[0085] Among them, 0.5 in the numerator and denominator of the fraction is mainly used for smoothing.

[0086] S353: According to the inverse document frequency value of each target morpheme, a sum of the inverse document frequency values ​​of the target morphemes is obtained.

[0087] S355: Obtain a normalized morpheme weight of each target morpheme according to the inverse document frequency value of each target morpheme and the sum of the inverse document frequency values.

[0088] Specifically, the inverse document frequency value of each target morpheme is divided by the sum of the inverse document frequency values, and the obtained value is used as the normalized morpheme weight corresponding to each target morpheme.

[0089] S370: Determine the similarity between the search query and the search result according to the normalized morpheme weight of each target morpheme and the target similarity between each target morpheme and the search result.

[0090] In a feasible implementation, in the calculation of the similarity between a morpheme and a search result, avglength(S) in formula (1) is replaced by the word count of the search query, and the coverage of the morpheme in the search query and the title of the search result is increased, so that the calculation result emphasizes the similarity in content. Specifically, Figure 6 As shown, step S370 may include the following steps:

[0091] S371: Determine a first proportion of the one or more target morphemes in the search query.

[0092] As mentioned above, the target morpheme can also be called a hit word.

[0093] S373: Determine a second proportion of the one or more target morphemes in the title of the search result.

[0094] S375: Determine the similarity between the search query and the search result according to the first proportion, the second proportion, and the target similarity and normalized morpheme weight corresponding to each target morpheme.

[0095] For example, taking the search result as a document as an example, the improved BM25 algorithm is shown in formula (5):

[0096]

[0097] In formula (5), Q represents query, i.e., the search query term input by the user; S represents document, which is the title of the current document in this application. In other application scenarios such as video search and image search, S can also represent the title of the search result; Query_Cover(Q, S) represents the ratio of the number of hit words to the number of all words in the query, i.e., the first ratio; Sen_Cover(Q, S) represents the ratio of the number of hit words to the number of all words in the title, i.e., the second ratio; TF(q i ) represents the frequency of the hit word qi in the search result collection; IDF(q i ) represents the IDF value (i.e., inverse document frequency) of the target morpheme (i.e., the hit word) qi in the search result collection; Length(S) represents the number of words in the title, i.e., the second word quantity; Length(Q) represents the number of words in the query, i.e., the first word quantity; represents the similarity score between qi and S, Indicates the normalized morpheme weight of qi. In addition, the parameter b is mainly used to adjust the effect of text title length on similarity, and k is the adjustment factor for word frequency.

[0098] Formula (5) can be used to calculate the similarity score between the query and the current document title. Generally speaking, the score is between 0 and 1. The smaller the score, the less similar it is, and the closer to 1, the more similar it is.

[0099] S250: Determine a classification label of a sample according to a preset classification interval and the similarity, wherein the sample includes the search query term and the search result.

[0100] In one embodiment of the present application, based on the aforementioned similarity score in the range of 0 to 1, corresponding interval division rules can be set according to manual experience or business needs. For example, the similarity can be divided into three levels to facilitate the construction of a subsequent pre-trained classifier model, such as:

[0101] [0~0.3): not very similar;

[0102] [0.3~0.6): indicates general similarity;

[0103] [0.6~1]: Indicates very similar.

[0104] S270: Fine-tune the pre-trained language model according to the sample and the corresponding classification label to obtain a search result classification model.

[0105] In the embodiments of the present application, the pre-trained language model refers to a general language model trained with a large amount of sample data. In order to make the model meet specific business needs, a small amount of sample data related to the specific business is often used for retraining, so as to make appropriate adjustments to the model parameters. Specifically, the sample is input into the pre-trained language model, the loss function is calculated according to the model output result and the classification label corresponding to the sample, and then the parameters of the pre-trained language model are appropriately adjusted according to the loss function to obtain the final search result classification model.

[0106] In an embodiment of the present application, a search result classification model can be applied to a search engine to accurately calculate the similarity between the search query term currently input by the user and the candidate search results corresponding to the search query term in the database, thereby classifying the candidate search results, and providing a class of candidate search results with high similarity as the final search results to the user, thereby further meeting the user's actual search needs and improving the user's search experience.

[0107] In one embodiment of the present application, the last layer of the classification network in the pre-trained language model is fine-tuned to achieve a classification function that is more suitable for specific services.

[0108] In a feasible implementation manner, specifically, as Figure 7 As shown, the steps may include:

[0109] S710: Extract features of the search query words in the sample based on the first language sub-model in the language model to obtain a first text feature.

[0110] S730: Extract features of search results in the sample based on the second language sub-model in the language model to obtain second text features.

[0111] The first language sub-model and the second language sub-model may both adopt a BERT model or a Word2Vec model to extract text features.

[0112] S750: Obtain a third text feature according to the first text feature and the second text feature.

[0113] In a feasible implementation, the text features obtained after feature extraction are expressed as vectors in terms of data type. Specifically, Figure 8 As shown, step S750 may include the following steps:

[0114] S751: Obtain text difference features according to the first text features and the second text features.

[0115] Specifically, the first language sub-model performs feature extraction on the search query term and outputs a first feature vector, which represents the first text feature of the search query. The second language sub-model performs feature extraction on the search results and outputs a second feature vector, which represents the second text feature of the search results. The first feature vector and the second feature vector are the same in dimension, so the first feature vector and the second feature vector can be subtracted to obtain a feature difference vector, which is used to represent the text difference feature. In terms of vector dimension, the feature difference vector is also the same as the first feature vector and the second feature vector.

[0116] S753: Concatenate the first text feature, the second text feature, and the text difference feature to obtain a third text feature.

[0117] Specifically, the first feature vector, the second feature vector and the feature difference vector are concatenated (ie, the number of dimensions is increased) and merged into a third feature vector representing the third text feature.

[0118] S770: Input the third text feature into the output layer of the language model to obtain a predicted classification label corresponding to the third text feature.

[0119] S790: Fine-tune the language model according to the predicted classification label and the corresponding classification label to obtain a search result classification model.

[0120] For example, Fig. 9 As shown, the original pre-trained language model can use BERT or word2vec model, and the two sentences of search query and document title are initialized and pre-trained vector representation respectively, and then the classification pooling layer (classification pooling, which can be referred to as CLS pooling) output by BERT is taken to obtain the corresponding first feature vector u and second feature vector v, and then u, v and |uv| are concatenated into the third feature vector vector (uv represents the difference between the two vectors u and v), and then given to the output layer (i.e., the classifier softmax classifier) ​​for classification. The classification label here can be the classification label under the above three categories. By fine-tuning the model (mainly the classifier) ​​with the predicted classification label and the real classification label, the sentences with higher text similarity can be represented in the model as vectors that are closer, while the vectors of sentences with lower text similarity are farther apart. Optionally, the BERT model includes many layers of transformers. In addition to extracting vectors from the CLS pooling layer, the vectors output by the first layer transformer and the last layer transformer can also be averaged to form u and v.

[0121] The present application also provides a training device 1000 for a search result classification model. Fig.10 As shown, the device 1000 may include:

[0122] A data acquisition module 1010 is used to acquire a training data set, wherein the training data set includes a search query term and search results corresponding to the search query term;

[0123] A similarity calculation module 1020, configured to determine the similarity between the search query and the search result based on a target text matching algorithm;

[0124] A classification label determination module 1030, configured to determine a classification label of a sample according to a preset classification interval and the similarity, wherein the sample includes the search query term and the search result;

[0125] The model fine-tuning module 1040 is used to fine-tune the pre-trained language model according to the samples and the corresponding classification labels to obtain a search result classification model.

[0126] In one embodiment of the present application, the similarity calculation module 1020 may include:

[0127] A morpheme parsing unit, configured to perform morpheme parsing on the search query to determine one or more target morphemes in the search query;

[0128] A morpheme similarity calculation unit, used to determine the target similarity between each target morpheme and the search result based on a target text matching algorithm;

[0129] A morpheme weight determination unit, used to determine a normalized morpheme weight of each target morpheme;

[0130] The similarity calculation unit is used to determine the similarity between the search query and the search result according to the normalized morpheme weight of each target morpheme and the target similarity between each target morpheme and the search result.

[0131] In one embodiment of the present application, the morpheme similarity calculation unit may include:

[0132] A word frequency determination subunit, configured to determine the word frequency of the target morpheme in the search result set when the search result set corresponding to the search query word includes one or more search results;

[0133] A word quantity determination subunit, used to determine a first word quantity of the search query word and a second word quantity of the title of the search result;

[0134] The target similarity calculation subunit is used to obtain the target similarity between the target morpheme and the search result according to the word frequency, the first word quantity and the second word quantity.

[0135] In one embodiment of the present application, the morpheme weight determination unit may include:

[0136] an inverse document frequency value determining subunit, configured to determine an inverse document frequency value of each target morpheme in the search result set when the search result set corresponding to the search query term includes one or more search results;

[0137] A summing subunit, configured to obtain the sum of the inverse document frequency values ​​of each target morpheme according to the inverse document frequency value of each target morpheme;

[0138] The normalized morpheme weight determination subunit is used to obtain the normalized morpheme weight of each target morpheme according to the inverse document frequency value of each target morpheme and the sum of the inverse document frequency values.

[0139] In one embodiment of the present application, the similarity calculation unit may include:

[0140] A first coverage determination subunit, configured to determine a first proportion of the one or more target morphemes in the search query;

[0141] A second coverage determination subunit, configured to determine a second proportion of the one or more target morphemes in the title of the search result;

[0142] The similarity calculation subunit is used to determine the similarity between the search query and the search result according to the first proportion, the second proportion, and the target similarity and normalized morpheme weight corresponding to each target morpheme.

[0143] In one embodiment of the present application, the model fine-tuning module 1040 may include:

[0144] A first feature extraction unit, configured to extract features of the search query term in the sample based on a first language sub-model in the language model to obtain a first text feature;

[0145] A second feature extraction unit, configured to extract features of the search results in the sample based on a second language sub-model in the language model to obtain a second text feature;

[0146] A third feature determination unit, configured to obtain a third text feature according to the first text feature and the second text feature;

[0147] A model prediction unit, used for inputting the third text feature into an output layer in the language model to obtain a predicted classification label corresponding to the third text feature;

[0148] A fine-tuning unit is used to fine-tune the language model according to the predicted classification label and the corresponding classification label to obtain a search result classification model.

[0149] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0150] An embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a training method for a search result classification model as provided in the above method embodiment.

[0151] Fig.11 A schematic diagram of the hardware structure of a device for implementing a method for training a search result classification model provided in an embodiment of the present application is shown. The device may participate in or include the apparatus or system provided in an embodiment of the present application. Fig.11 As shown, the device 10 may include one or more (1002a, 1002b, ..., 1002n are used to illustrate) processors 1002 (the processor 1002 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Fig.11 The structure shown is for illustration only and does not limit the structure of the electronic device. Fig.11 More or fewer components as shown, or with Fig.11 Different configurations are shown.

[0152] It should be noted that the one or more processors 1002 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the device 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0153] The memory 1004 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the methods described in the embodiments of the present application. The processor 1002 executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, that is, to implement the above-mentioned training method for a search result classification model. The memory 1004 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include a memory remotely arranged relative to the processor 1002, and these remote memories may be connected to the device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0154] The transmission device 1006 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the device 10. In one example, the transmission device 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 1006 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0155] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of device 10 (or mobile device).

[0156] An embodiment of the present application also provides a computer-readable storage medium, which can be set in a server to store at least one instruction or at least one program related to a training method for a search result classification model in a method embodiment, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a training method for a search result classification model provided in the above method embodiment.

[0157] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0158] The embodiment of the present invention also provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes a method for training a search result classification model provided in the above various optional implementations.

[0159] It can be seen from the embodiments of the training method, device, medium and equipment of the search result classification model provided by the present application above that

[0160] The solution provided in this application calculates the similarity between the search query and the search results through a text matching algorithm, and determines the classification label of the sample based on the similarity and the preset classification interval, the sample includes the search query and the corresponding search result; then the existing pre-trained language model is fine-tuned through the sample and the classification label of the sample to obtain the search result classification model, and the search result classification model can be applied to the search application, so that the obtained search results are more matched with the search query in the similarity dimension, further meeting the user's search needs and improving the user experience. Specifically, the BM25 formula is rewritten in the text matching algorithm, factors such as morpheme coverage are added, and the calculation of morpheme weights and morpheme similarity with search results is improved, so that the text matching algorithm pays more attention to the similarity between the search query and the search results in content to meet the model application requirements. In addition, compared with training from scratch, the training method of fine-tuning the pre-trained language model can reduce the amount of training data required, save model training costs, and improve model training efficiency.

[0161] It should be noted that fine-tuning is a training method, which means that based on the model trained with a large amount of training data (the model at this time is general and generally open source), the model is trained again with a part of the training data related to a specific business to make it more suitable for the needs of the specific business. Therefore, generally only a second round of training is required. Experiments have shown that only a small amount of specific training data is needed to make the model performance meet the needs of a specific business; and reducing the amount of training data can shorten the training time and reduce the cost of model training, thereby improving the efficiency of model training.

[0162] The above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the attached claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0163] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0164] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0165] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for training a search result classification model, characterized in that: The method comprises: Acquire a training data set, the training data set including a search query term and search results corresponding to the search query term; Determining the similarity between the search query and the search result based on a target text matching algorithm, wherein the search query includes one or more target morphemes; Determining a classification label of a sample according to a preset classification interval and the similarity, the sample including the search query term and the search result; Fine-tune the pre-trained language model according to the sample and the corresponding classification label to obtain a search result classification model; Wherein, the target text matching algorithm is expressed as: Q represents the search query; S represents the search result; Query_Cover(Q, S) is a first ratio, indicating the coverage of the one or more target morphemes in the search query; Sen_Cover(Q, S) is a second ratio, indicating the coverage of the one or more target morphemes in the title of the search result; TF(q i ) represents the frequency of the target morpheme qi in the multiple search results; IDF(q i ) represents the inverse document frequency value of the target morpheme qi in the plurality of search results; Length(Q) is the first word quantity, indicating the number of words in the search query; Length(S) is the second word quantity, indicating the number of words in the title of the search result; represents the target similarity between the target morpheme qi and the search result; represents the normalized morpheme weight of the target morpheme qi; Parameter b is used to adjust the effect of the length of the title of the search result on the similarity, and k is an adjustment factor for word frequency.

2. The method according to claim 1, characterized in that The determining the similarity between the search query and the search result based on the target text matching algorithm includes: Performing morpheme analysis on the search query to determine one or more target morphemes in the search query; Determining a target similarity between each of the target morphemes and the search result; determining a normalized morpheme weight for each of the target morphemes; The similarity between the search query and the search result is determined based on the coverage of the one or more target morphemes in the search query and the corresponding search result, the normalized morpheme weight of each target morpheme, and the target similarity between each target morpheme and the search result.

3. The method according to claim 2, characterized in that In the case where the search query word corresponds to a plurality of search results, determining the target similarity between each of the target morphemes and the search results includes: Determining the frequency of the target morpheme in the plurality of search results; Determining a first word quantity of the search query term and a second word quantity of the title of the search result; the search result is any one of the multiple search results; A target similarity between the target morpheme and the search result is obtained according to the word frequency, the first word quantity and the second word quantity.

4. The method according to claim 3, characterized in that The determining of the normalized morpheme weight of each target morpheme comprises: Determining an inverse document frequency value of each of the target morphemes in the plurality of search results; According to the inverse document frequency value of each target morpheme, a sum of the inverse document frequency values ​​of each target morpheme is obtained; The normalized morpheme weight of each target morpheme is obtained according to the inverse document frequency value of each target morpheme and the sum of the inverse document frequency values.

5. The method according to claim 2, characterized in that: The determining the similarity between the search query and the search result according to the coverage of the one or more target morphemes in the search query and the corresponding search result, the normalized morpheme weight of each target morpheme, and the target similarity between each target morpheme and the search result includes: Determining a first proportion of the one or more target morphemes in the search query; the first proportion represents a coverage rate of the one or more target morphemes in the search query; Determining a second proportion of the one or more target morphemes in the title of the search result; the second proportion represents a coverage rate of the one or more target morphemes in the corresponding search result; The similarity between the search query and the search result is determined according to the first proportion, the second proportion, the target similarity corresponding to each target morpheme, and the normalized morpheme weight.

6. The method according to claim 1, characterized in that The step of fine-tuning the pre-trained language model according to the sample and the corresponding classification label to obtain a search result classification model includes: Extracting features of the search query words in the sample based on the first language sub-model in the language model to obtain a first text feature; Extracting features of search results in the sample based on a second language sub-model in the language model to obtain a second text feature; Obtaining a third text feature according to the first text feature and the second text feature; Inputting the third text feature into the output layer of the language model to obtain a predicted classification label corresponding to the third text feature; The language model is fine-tuned according to the predicted classification label and the corresponding classification label to obtain a search result classification model.

7. The method according to claim 6, characterized in that The text feature is represented by a vector, and obtaining the third text feature according to the first text feature and the second text feature includes: Obtaining a text difference feature according to the first text feature and the second text feature; The first text feature, the second text feature and the text difference feature are concatenated to obtain a third text feature.

8. A training device for a search result classification model, characterized in that: The device comprises: A data acquisition module, configured to acquire a training data set, wherein the training data set includes a search query term and search results corresponding to the search query term; A similarity calculation module, configured to determine the similarity between the search query and the search result based on a target text matching algorithm, wherein the search query includes one or more target morphemes; the target text matching algorithm is a text matching algorithm based on the coverage of the one or more target morphemes in the search query and the corresponding search result; A classification label determination module, used to determine a classification label of a sample according to a preset classification interval and the similarity, wherein the sample includes the search query term and the search result; A model fine-tuning module, used to fine-tune the pre-trained language model according to the sample and the corresponding classification label to obtain a search result classification model; Wherein, the target text matching algorithm is expressed as: Q represents the search query; S represents the search result; Query_Cover(Q, S) is a first ratio, indicating the coverage of the one or more target morphemes in the search query; Sen_Cover(Q, S) is a second ratio, indicating the coverage of the one or more target morphemes in the title of the search result; TF(q i ) represents the frequency of the target morpheme qi in the multiple search results; IDF(q i ) represents the inverse document frequency value of the target morpheme qi in the plurality of search results; Length(Q) is the first word quantity, indicating the number of words in the search query; Length(S) is the second word quantity, indicating the number of words in the title of the search result; represents the target similarity between the target morpheme qi and the search result; represents the normalized morpheme weight of the target morpheme qi; Parameter b is used to adjust the effect of the length of the title of the search result on the similarity, and k is an adjustment factor for word frequency.

9. The device according to claim 8, characterized in that The similarity calculation module comprises: A morpheme parsing unit, configured to perform morpheme parsing on the search query to determine one or more target morphemes in the search query; A morpheme similarity calculation unit, used to determine a target similarity between each of the target morphemes and the search result; A morpheme weight determination unit, used to determine a normalized morpheme weight of each target morpheme; A similarity calculation unit is used to determine the similarity between the search query and the search result based on the coverage of the one or more target morphemes in the search query and the corresponding search result, the normalized morpheme weight of each target morpheme, and the target similarity between each target morpheme and the search result.

10. The device according to claim 9, characterized in that The morpheme similarity calculation unit comprises: A word frequency determination subunit, configured to determine the word frequency of the target morpheme in the search result set when the search result set corresponding to the search query word includes one or more search results; A word quantity determination subunit, used to determine a first word quantity of the search query word and a second word quantity of the title of the search result; The target similarity calculation subunit is used to obtain the target similarity between the target morpheme and the search result according to the word frequency, the first word quantity and the second word quantity.

11. The device according to claim 10, characterized in that The morpheme weight determination unit comprises: an inverse document frequency value determining subunit, configured to determine an inverse document frequency value of each target morpheme in the search result set when the search result set corresponding to the search query term includes one or more search results; A summing subunit, configured to obtain the sum of the inverse document frequency values ​​of each target morpheme according to the inverse document frequency value of each target morpheme; The normalized morpheme weight determination subunit is used to obtain the normalized morpheme weight of each target morpheme according to the inverse document frequency value of each target morpheme and the sum of the inverse document frequency values.

12. The device according to claim 9, characterized in that The similarity calculation unit comprises: A first coverage determination subunit is used to determine a first proportion of the one or more target morphemes in the search query; the first proportion represents the coverage of the one or more target morphemes in the search query; A second coverage determination subunit is used to determine a second proportion of the one or more target morphemes in the title of the search result; the second proportion represents the coverage of the one or more target morphemes in the corresponding search result; The similarity calculation subunit is used to determine the similarity between the search query and the search result according to the first proportion, the second proportion, and the target similarity and normalized morpheme weight corresponding to each target morpheme.

13. The device according to claim 8, characterized in that The model fine-tuning module includes: A first feature extraction unit, configured to extract features of the search query term in the sample based on a first language sub-model in the language model to obtain a first text feature; A second feature extraction unit, configured to extract features of the search results in the sample based on a second language sub-model in the language model to obtain a second text feature; A third feature determination unit, configured to obtain a third text feature according to the first text feature and the second text feature; A model prediction unit, configured to input the third text feature into an output layer of the language model to obtain a predicted classification label corresponding to the third text feature; A fine-tuning unit is used to fine-tune the language model according to the predicted classification label and the corresponding classification label to obtain a search result classification model.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a training method for a search result classification model as described in any one of claims 1 to 7.

15. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement a training method for a search result classification model as described in any one of claims 1 to 7.

16. A computer program product, comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, a method for training a search result classification model as described in any one of claims 1 to 7 is implemented.

Images