Method, apparatus, electronic device and storage medium for training a semantic search model

By integrating multiple semantic search models through a distilled dataset and a double tower structure, the method addresses the limitations of single-model semantic search, enhancing accuracy and robustness.

JP7581612B2Active Publication Date: 2024-11-13BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023037698
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2022-06-30
Filing Date
2023-03-10
Publication Date
2024-11-13
Estimated Expiration
2043-03-10

Smart Images

  • Figure 0007581612000001
    Figure 0007581612000001
  • Figure 0007581612000002
    Figure 0007581612000002
  • Figure 0007581612000003
    Figure 0007581612000003
Patent Text Reader

Abstract

To provide a semantic retrieval model training method, an apparatus, an electronic device, and a storage medium.SOLUTION: A method includes: obtaining target query statement types corresponding to each of at least two original semantic retrieval models which are query statement types that are most accurate in processing various types of query statements; based on at least two original semantic retrieval models, the target query sentence type corresponding to each original semantic retrieval model, and a pre-constructed corpus, obtaining a distilled data set; and training the target semantic retrieval model based on the distilled data set.EFFECT: The trained target semantic retrieval model can fuse retrieval capabilities of the at least two original semantic retrieval models, and semantic retrieval accuracy can be improved.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to the field of computer technology, specifically to the field of artificial intelligence technologies such as machine learning and natural language processing, and in particular to methods, apparatus, electronic devices, and storage media for training semantic search models. [Background technology]

[0002] In the information age, people want to quickly find the information they need from a huge number of books, web pages, and documents. Recalling candidates from a large amount of data, and then re-sorting the recalled data to score their confidence, has become the mainstream pattern of information retrieval today.

[0003] Among them, in the recall stage of the search task, there are usually two different methods: sparse vector search and dense vector search. In the sparse vector search method, the query sentence and the candidate linguistic materials are encoded into a sparse vector, and the dimension of this vector is generally roughly the same as that of the dictionary. This method mainly relies on the degree of character matching to calculate the similarity. Common algorithms include BM25, but it is impossible to learn a semantic search model that corresponds to the sparse vector search. This method has high transferability and is not limited to a specific field. In the dense vector search method, the query sentence and the candidate linguistic materials are encoded into two vectors in the semantic space by the corresponding semantic search model, and the similarity is calculated based on the vector to recall the correlation result. In this method, it is necessary to train the semantic search model with training data, and the matching degree can be judged using semantic information, but the transferability is poor. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a method, apparatus, electronic device, and storage medium for training a semantic search model. [Means for solving the problem]

[0005] According to one aspect of the present disclosure, there is provided a method for training a semantic search model, the method including: obtaining a target query sentence type corresponding to each of at least two original semantic search models, the target query sentence type being a query sentence type for which the original semantic search models are most accurate when processing various types of query sentences; obtaining a distilled dataset based on the at least two original semantic search models, the target query sentence types corresponding to each of the original semantic search models, and a pre-constructed corpus; and training a target semantic search model based on the distilled dataset.

[0006] According to another aspect of the present disclosure, there is provided an apparatus for training a semantic search model, the apparatus including: a type acquisition module for acquiring a target query sentence type corresponding to each of at least two original semantic search models, the target query sentence type being a query sentence type for which the original semantic search models are most accurate when processing various types of query sentences; a data acquisition module for acquiring a distilled dataset based on the at least two original semantic search models and a pre-established corpus; and a training module for training a target semantic search model based on the distilled dataset.

[0007] According to yet another aspect of the present disclosure, there is provided an electronic device comprising at least one processor and a memory communicatively connected to the at least one processor, the memory storing commands executable by the at least one processor, the commands, when executed by the at least one processor, causing the at least one processor to perform a method of any of the aspects and possible embodiments described above.

[0008] According to yet another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any of the above aspects and possible embodiments.

[0009] According to yet another aspect of the present disclosure there is provided a computer program product comprising a computer program which, when executed by a processor, performs the method of any of the above aspects and possible embodiments.

[0010] According to the technology disclosed herein, the search capabilities of at least two original semantic search models can be integrated into a trained target semantic search model, thereby overcoming the shortcomings of a single semantic search model and improving the accuracy of semantic search.

[0011] It should be understood that the contents described in this section are not intended to identify key or essential features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood from the following specification. [Brief description of the drawings]

[0012] The drawings are for better understanding of the technical solution and are not intended to limit the present disclosure. [Figure 1] FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure. [Diagram 3] FIG. 13 is a schematic diagram according to a third embodiment of the present disclosure. [Figure 4] FIG. 1 is a configuration diagram of a method for training a semantic search model according to an embodiment of the present invention. [Diagram 5] FIG. 13 is a schematic diagram according to a fourth embodiment of the present disclosure. [Figure 6] FIG. 13 is a schematic diagram according to a fifth embodiment of the present disclosure. [Figure 7] FIG. 1 is a block diagram of an electronic device for implementing the method of an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, exemplary embodiments of the present disclosure will be described based on the drawings. For ease of understanding, various details of the embodiments of the present disclosure are included, and they should be regarded as merely examples. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of simplicity, the following description omits the description of well-known functions and structures.

[0014] Apparently, the described embodiments are only some of the embodiments of the present disclosure, but not all of the embodiments. All other embodiments obtained based on the embodiments of the present disclosure under the premise that a person skilled in the art does not perform creative labor, belong to the scope of protection of the present disclosure.

[0015] It should be noted that the terminal device according to the embodiment of the present disclosure may include, but is not limited to, smart devices such as mobile phones, personal digital assistants (PDAs), wireless handheld devices, and tablet computers. The display device may include, but is not limited to, devices having a display function such as personal computers and televisions.

[0016] Furthermore, the term "and / or" in this specification is simply a relational relationship describing related objects, and means that three relations may exist. For example, A and / or B can mean three situations: A exists alone, A and B exist simultaneously, and B exists alone. Also, the character " / " in this specification generally means that the related objects before and after it are in an "or" relationship.

[0017] In the prior art, the sparse vector-based search method and the dense vector-based search method are generally used alone. The sparse vector-based search method can only model text matching, and lacks semantic understanding of the content, making it less effective. On the other hand, the dense vector-only search method misses some text matching information. In other words, if either of the above search methods is used alone, the accuracy of semantic search will be poor.

[0018] 1 is a schematic diagram according to a first embodiment of the present disclosure. As shown in FIG. 1, this embodiment provides a method for training a semantic search model, which specifically includes the following steps:

[0019] In S101, a target query sentence type corresponding to each of at least two original semantic search models is obtained.

[0020] Here, the target query sentence type corresponding to the original semantic search model is the query sentence type with the highest accuracy when the original semantic search model processes various types of query sentences.

[0021] In S102, a distilled dataset is obtained based on at least two original semantic search models, a target query sentence type corresponding to each original semantic search model, and a pre-constructed corpus.

[0022] In S103, a target semantic search model is trained based on the distilled dataset.

[0023] The execution body of the method for training a semantic search model in this embodiment may be a device for training a semantic search model, which may be an electronic entity or an application that adopts software integration and runs on a computing device when in use to realize the training of a semantic search model.

[0024] The at least two semantic search models of the present embodiment may include a sparse vector-based semantic search model and a dense vector-based semantic search model. Here, the sparse vector-based semantic search model may encode the literal semantic vector of any linguistic material, such as a query sentence and / or a candidate linguistic material, based on a dictionary. Meanwhile, the dense vector-based semantic search model realizes encoding the semantic vector of the query sentence and the candidate linguistic material at the semantic level based on a pre-trained neural network model. For example, the dense vector-based semantic search model of the present embodiment may be realized based on RocketQAv2, ColBERT, Phrase-BERT, or COIL model, etc.

[0025] That is, the at least two original semantic search models of this embodiment can include at least two of the BM25 model, RocketQAv2, ColBERT, Phrase-BERT, and COIL model, etc. In addition, the original semantic search models used in this embodiment are all known or trained.

[0026] Since each original semantic search model processes different types of query sentences, the accuracy is different accordingly. For query sentence types that the original semantic search model is good at processing, the accuracy of the processing is high. On the other hand, for query sentence types that the original semantic search model is not good at processing, the accuracy of the processing is low. Based on this, in the present embodiment, when selecting a target query sentence type corresponding to each original semantic search model, the corresponding target query sentence type can be obtained based on the accuracy of each original semantic search model processing a query sentence corresponding to each query sentence type. For example, the most accurate query sentence type can be selected as the target query sentence type. Optionally, in the present embodiment, one, two or more target query sentence types corresponding to each original semantic search model can be obtained according to actual needs. For example, when only one target query sentence type is obtained, more target query sentence types may be obtained because the language material of the selectable target query sentence is insufficient. This may be considered as the target query sentence type that the original semantic search model is good at. Thereby, when obtaining distilled data based on the target query sentence type, the characteristics of the corresponding original semantic search model can be more embodied. Furthermore, when training a target semantic model based on such distilled data, the target semantic search model can learn the characteristics of the original semantic search model.

[0027] The target query sentence type in this embodiment may be an address type, an answer class, or a resource class, or in practical applications, the query sentence type may be divided according to fields, scenarios, etc. to obtain the corresponding target query sentence type.

[0028] In this embodiment, a distilled dataset can be obtained based on at least two original semantic search models, the target query sentence types corresponding to each original semantic search model, and a pre-constructed corpus. The distilled dataset is screened based on at least two original semantic search models, so that the characteristics of each original semantic search model are compatible. Furthermore, training is performed on a target semantic search model based on the distilled dataset, so that the characteristics of each original semantic search model are integrated into the target semantic search model, which overcomes the shortcomings of a single semantic search model being inaccurate, and can perform more accurate semantic search and improve the accuracy of recall results.

[0029] The semantic search model training method of this embodiment obtains a distilled dataset based on at least two original semantic search models and a pre-constructed corpus, and then trains a target semantic search model based on the distilled dataset, thereby integrating the search capabilities of the at least two original semantic search models into the trained target semantic search model, thereby overcoming the shortcomings of a single semantic search model, and improving the accuracy of semantic search.

[0030] Figure 2 is a schematic diagram according to the second embodiment of the present disclosure. The training method for the semantic search model of this embodiment is described in more detail based on the technical solution of the embodiment shown in Figure 1 above. As shown in Figure 2, the training method for the semantic search model of this embodiment specifically includes the following steps:

[0031] In S201, a target query sentence type corresponding to each of at least two original semantic search models is obtained.

[0032] In S202, a target query sentence corresponding to each original semantic search model is obtained based on a target query sentence type corresponding to each original semantic search model.

[0033] In S203, a predetermined number of recall language materials are recalled from the corpus based on each original semantic search model and the target query sentence corresponding to each original semantic search model.

[0034] In S204, a distilled dataset is generated based on the target query sentences corresponding to each original semantic search model and the corresponding recalled predetermined number of recall linguistic materials.

[0035] In order to make the distilled data in the generated distilled dataset more accurately reflect the characteristics of each original semantic search model, in this embodiment, as many query sentences as possible that each original semantic search model is good at are selected as target queries to perform language material recall. Based on the target queries of each semantic search model and the recall of the corresponding recall language materials, the search ability and search characteristics of the corresponding semantic search model can be reflected. Therefore, in this embodiment, a distilled dataset is generated based on the target query sentences corresponding to each original semantic search model and a predetermined number of recall language materials that have been recalled.

[0036] In this embodiment, in step S201, obtaining the target query sentence type corresponding to each original semantic search model may specifically include any of the following ways:

[0037] The first method obtains target query sentence types corresponding to each original semantic search model based on a pre-constructed test set corresponding to each type of query sentence.

[0038] In the first method, a corresponding test set can be pre-constructed based on different types of query sentences. For example, the different types of query sentences can include address-seeking type, answer-seeking type, or resource-seeking type. Alternatively, the types of query sentences can be divided according to fields or scenarios.

[0039] In this method, a test set corresponding to each type of query sentence can be used to detect the accuracy rate of each original semantic search model. If the accuracy rate is greater than a preset accuracy rate threshold, such as 90%, 95% or other proportional value, the query sentence type can be the target query sentence type corresponding to the original semantic search model. In addition, various query sentence types can be sorted in order of accuracy to obtain the most accurate query sentence type. In this embodiment, the target query sentence type corresponding to each original semantic search model can be one, two or more, and is not limited here.

[0040] The second method obtains a target query sentence type corresponding to each original semantic search model based on the attributes of each original semantic search model.

[0041] In this implementation, the attributes of the original semantic search model can be predefined. Specifically, the attributes of the original semantic search model can be arranged according to the type, field or scenario of the training set when training the model. The attributes can indicate the type, field or scenario of the training data used when the original semantic search model is trained, and indicate that the original semantic search model is good at processing query sentences in the type, field or scenario, and has the highest accuracy in processing query sentences in the type, field or scenario compared with processing query sentences of other types. Here, the type of the training set can also include a type for requesting an address, a type for requesting an answer, or a type for requesting a resource. Based on this, the attributes of the original semantic search model can be the target query sentence type corresponding to the original semantic search model.

[0042] In this embodiment, no matter which of the above methods is adopted, the target query sentence type corresponding to each original semantic search model can be accurately obtained.

[0043] Next, a target query sentence corresponding to each original semantic search model is obtained based on the target query sentence type corresponding to each original semantic search model. For example, various types of query sentences can be collected and stored in a query sentence corpus based on the user's historical behavior data. When used, any corresponding query sentence can be obtained from the query sentence corpus based on the target query sentence type as the target query sentence. Alternatively, the target query sentence can be obtained in another manner. For example, any one query sentence of the corresponding type can be obtained from the user's log as the target query sentence directly based on the target query sentence type.

[0044] In this embodiment, when generating a distilled dataset, the above steps S202 and S203 may be repeated as described above according to the size of the distilled dataset to be generated, to obtain each target query sentence corresponding to each original semantic search model and a predetermined number of recall linguistic materials recalled from the corpus for each target query sentence. That is, for one target query sentence and one original semantic search model, a predetermined number of recall linguistic materials can be recalled from the corpus. The predetermined number in this embodiment can be set according to actual needs. For example, it may be 100, 80, 50, 20, or other recall linguistic materials.

[0045] In this embodiment, the target query sentence corresponding to each original semantic search model and a predetermined number of recalled recall linguistic materials can be added to the distilled dataset as distilled data as they are. Since the recall linguistic materials are all obtained according to the target query sentence based on the original semantic search model, they are distilled data as positive samples in the distilled dataset. Therefore, it is necessary to mark this distilled data as a positive sample by arranging the correlation degree between the target query sentence and the recall linguistic materials as 1, and it may be indicated that the probability that the recall linguistic materials will be recalled when the target query sentence is searched is 1.

[0046] Conversely, when constructing distilled data of a negative sample in the distilled dataset, non-recallable negative sample language material can be constructed by referring to the target query sentence corresponding to the original semantic search model and a predetermined number of recalled recall language materials, and the distilled data can be marked as a negative sample by arranging the correlation between the target query sentence and the negative sample language material to be 0, thereby indicating that the probability that the negative sample language material will be recalled when the target query sentence is searched is 0.

[0047] Steps S201 to S204 are an embodiment of step S101 in the embodiment shown in FIG.

[0048] In S205, a target semantic search model is trained based on the distilled dataset.

[0049] The training method of the semantic search model of this embodiment obtains the target query sentence corresponding to each original semantic search model and a predetermined number of recalled linguistic materials recalled accordingly to generate a distilled dataset, and further trains the target semantic search model based on the distilled dataset. The target query sentence corresponding to each original semantic search model obtained and the predetermined number of recalled linguistic materials recalled accordingly can fully express the ability and performance of each original semantic search model, so that the generated distilled dataset can retain the characteristics of each original semantic search model. Furthermore, when training the target semantic search model based on the distilled dataset, the search ability of at least two original semantic search models can be integrated into the trained target semantic search model, which overcomes the shortcomings of a single semantic search model and effectively improves the accuracy of semantic search.

[0050] 3 is a schematic diagram according to the third embodiment of the present disclosure. The training method for the semantic search model of this embodiment is described in more detail based on the technical solution of the embodiment shown in FIG. 1. As shown in FIG. 3, the training method for the semantic search model of this embodiment specifically includes the following steps:

[0051] In S301, obtain target query sentence types corresponding to at least two original semantic search models respectively based on a pre-defined test set corresponding to each type of query sentence.

[0052] For a specific implementation method, the description of the embodiment shown in FIG. 2 above can be referred to, and therefore a detailed description will not be given here.

[0053] In S302, a target query sentence corresponding to each original semantic search model is obtained based on a target query sentence type corresponding to each original semantic search model.

[0054] In S303, a predetermined number of recall language materials are recalled from the corpus based on each original semantic search model and the target query sentence corresponding to each original semantic search model.

[0055] In S304, a pre-trained fine sorting model is used to screen the target query sentences corresponding to each original semantic search model and a predetermined number of recalled linguistic materials to generate a distilled dataset.

[0056] Steps S301 to S304 are an embodiment of step S101 in the embodiment shown in FIG.

[0057] Unlike the embodiment shown in FIG. 2 above, in this embodiment, when generating a distilled data set by screening, a fine sorting model is further used to screen the target query sentences and a predetermined number of recalled language materials corresponding to each original semantic search model to generate a distilled data set. For example, FIG. 4 is a configuration diagram of a training method for a semantic search model in this embodiment. Accordingly, the configuration diagram of the embodiment shown in FIG. 2 may be the screening of the recall result excluding the fine sorting model in FIG. 4.

[0058] The original semantic search model and the target semantic search model using the neural network structure of this embodiment both use a double-tower structure model to realize the encoding of the query sentence query and the candidate linguistic materials, and further calculate the vector similarity between the two based on the encoding result. The fine sort model of this embodiment is obtained by pre-training using annotation data. The fine sort model can model the interaction information between the query sentence query and the candidate linguistic materials, and its ability is superior to that of the corresponding double-tower structure model. Therefore, the quality of the distilled data in the distilled dataset can be effectively improved by using the fine sort model to screen and filter the recall results of each original semantic search model.

[0059] In one embodiment of the present disclosure, step S304, using a pre-trained fine sorting model to screen the target query sentences corresponding to each original semantic search model and the recalled predetermined number of recall linguistic materials to generate a distilled dataset, can specifically include the following steps:

[0060] (a1) Screening positive sample data based on the fine sorting model, the target query sentence corresponding to each original semantic search model, and a predetermined number of recalled linguistic materials.

[0061] (b1) Screening negative sample data based on the fine sorting model, the target query sentence corresponding to each original semantic search model, and a predetermined number of recalled linguistic materials.

[0062] (c1) The positive sample data and the negative sample data are stored in a distilled dataset.

[0063] For example, in this embodiment, the positive sample data and the negative sample data can be separately screened based on the fine sorting model and collected as distilled data to form a distilled data set. Specifically, the ratio of the positive sample data to the negative sample data in the distilled data set can be set according to actual needs, for example, it can be 1:1, 1:2, 1:3, 1:4, or other ratios, and is not limited here. It is sufficient to screen the corresponding sample data according to the number of desired samples.

[0064] In this embodiment, when the distilled data set is generated, the positive sample data and the negative sample data may not be labeled in the method of integrating samples or in the distilled data. For example, each distilled data may include one query sentence, at least two recall language materials, and a ranking of the correlation degree between the at least two recall language materials and the query sentence. This type of distilled data may be soft-labeled data. By training the target semantic search model with this type of distilled data, the target semantic search model can learn a score ranking of the correlation degree between different recall language materials and the same query sentence. In this embodiment, in step S304, the target query sentence corresponding to each original semantic search model and a predetermined number of recall language materials that have been recalled are screened using a pre-trained fine sorting model to generate a distilled data set in the following three cases.

[0065] The first case is direct integration. In this case, it is necessary to obtain positive sample data and negative sample data according to the above-mentioned method based on each original semantic search model, and then directly integrate the positive sample data and negative sample data generated by at least two original semantic search models to obtain a final distilled dataset. That is, the distilled dataset obtained in this case sufficiently contains the positive sample data and negative sample data distilled for each original semantic search model.

[0066] For example, step (a1) in this case may accordingly include the following steps:

[0067] (a2) for a target query sentence corresponding to each original semantic search model, a fine sort model is used to calculate a correlation score between the target query sentence and each of a predetermined number of recall linguistic materials recalled accordingly;

[0068] (b2) Removing recall language materials having correlation scores less than a preset threshold from the first N of the predetermined number of recall language materials, where N is a positive integer greater than 1.

[0069] (c2) Constructing positive sample data based on the target query sentence and each of the remaining recall linguistic materials among the first N in the predetermined number of recall linguistic materials.

[0070] For example, the positive sample data constructed at this point may include the target query sentence, and the first N recall language materials among a predetermined number of recall language materials that have a correlation score equal to or greater than a preset threshold. In order to train the target semantic search model using these samples as positive samples, the labels of the correlation scores of the positive sample data at this point are rearranged to 1, so that the target semantic search model can learn the ability to recall the recall language materials in the corresponding positive sample data based on the target query sentence.

[0071] For example, accordingly, in this case step (b1) may include the following steps:

[0072] (a3) For each target query sentence corresponding to the original semantic search model, select a recall language material having a correlation score smaller than a preset threshold from among the (N+1)th and subsequent recall language materials among a predetermined number of recall language materials, where N is a positive integer greater than 1.

[0073] (b3) Constructing negative sample data based on the target query sentence and recall language materials selected from the (N+1)th and subsequent recall language materials among a predetermined number of recall language materials and having a correlation score smaller than a preset threshold value.

[0074] Similarly, the negative sample data constructed at this point may include the target query sentence and the recall language materials from the N+1th and subsequent recall language materials in the predetermined number of recall language materials whose correlation scores are smaller than a preset threshold. In order to train the target semantic search model using this sample as a negative sample, the label of the correlation score of the negative sample data at this point is rearranged to 0, so that the target semantic search model can learn the ability not to recall the recall language materials in the corresponding negative sample data based on this target query sentence.

[0075] For example, when the predetermined number is 100, N is 20. For the first 100 recall language materials recalled by each original semantic search model for any one target query sentence, the quality of the positive sample can be improved by taking the recall language materials with a correlation score smaller than a preset threshold among the first 20. Meanwhile, the negative sample data can be constructed by taking the recall language materials with a correlation score smaller than a preset threshold among the 20th to 100th recall language materials. The preset threshold can be empirically set, such as 0.1, 0.2, or other numerical values. By adopting this method, the quality of the distilled data in the distilled dataset can be effectively improved.

[0076] The second case is cross-integration. In this case, the positive sample data generated by each original semantic search model can be taken and stored in a distilled dataset. Meanwhile, the negative sample data can be screened from all the recall results by at least two original semantic search models. That is, the distilled dataset obtained in this case contains a sufficient amount of positive sample data distilled by each original semantic search model, but the negative sample data may be distilled from only a part of the original semantic search models.

[0077] For example, according to this case, in step (b1), screening the negative sample data based on the fine sorting model, the target query sentence corresponding to each original semantic search model, and the recalled predetermined number of recall linguistic materials can specifically include the following steps:

[0078] (a4) for each target query sentence corresponding to the original semantic search model, a fine sort model is used to calculate a correlation score between the corresponding target query sentence and each recall linguistic material in the predetermined number of recall linguistic materials recalled accordingly;

[0079] (b4) Based on the correlation score between each target query sentence and each recall language material, negative sample data is screened from all recall language materials by at least two original semantic search models according to a preset method.

[0080] For example, a required number of negative sample data can be screened in ascending order of correlation score. Alternatively, all recall language materials whose correlation scores are smaller than a preset threshold can be directly obtained. Alternatively, negative sample data can be screened in other ways, but this is not limited thereto. In this way, by referring to all recall language materials according to at least two original semantic search models, negative sample data with higher quality can be selected, and the quality of the distilled dataset can be further improved.

[0081] The third case is soft-label data integration. In this case, unlike the above two cases, the target query sentences corresponding to at least two original semantic search models and a predetermined number of recalled linguistic materials corresponding thereto are integrated without distinguishing between positive and negative samples, while keeping the original scores or rankings of the original semantic search models. In this case, when training the target semantic search model using the distilled dataset, the target semantic search model is trained to learn the scores or rankings for different sample data by the original semantic search model.

[0082] In practical applications, one can choose any of the above three cases to generate a distilled data set according to need. In any case, an accurate, reasonable and effective distilled data set can be obtained.

[0083] In this embodiment, when generating a distilled data set, the above method adopts the target query sentence type that each original semantic search model is good at as much as possible to generate distilled data of a related type, thereby making the most of the advantages of each original semantic search model to generate advantageous distilled data. For example, since the RocketQAv2 model may be better for problem-based queries, the distribution of query sentences when generating a distilled data set in this step is adjusted, and the proportion of problem-based query sentences handled by the RocketQAv2 model is increased to obtain more effective distilled data. The same is true for other original semantic search models, and in order to obtain more effective distilled data, a type of query sentence that is good at processing is selected, but this will not be described in detail here.

[0084] In S305, a target semantic search model is trained based on the distilled dataset.

[0085] The target semantic search model in this embodiment is a double tower structure model.

[0086] For different distilled dataset generation methods, different training methods can be used. For example, for the first and second cases of generating distilled datasets, i.e., the direct integration and cross integration generation methods, the hard label method can be used for training, and a common comparative learning training method can be used to train using cross entropy loss and introducing intra-lot negative sampling. For the soft label data integration method, i.e., the third case of generating the distilled dataset, the marginMSE method can be used for training to learn the scoring between sample pairs. These two methods can be flexibly selected according to the training effect.

[0087] The double-column target semantic search model trained using the integrated distilled data can integrate the characteristics of multiple teachers, i.e., the original semantic search models, and control the data distribution of different teachers during the distilled data generation process, and only retain the advantageous and highly different parts, thereby making the most of the generated weakly labeled training data.

[0088] After training on weakly labeled data, the effect of the double-tower target semantic search model can exceed the results of training on annotated data, and has stronger generalization ability, reducing over-fitting to the annotated data, and partially resolving the robustness problem caused by data deviation.

[0089] In the present embodiment, the method for training a semantic search model uses a fine sorting model to obtain the target query sentences of each original semantic search model and a certain number of recalled language materials corresponding to the target query sentences to generate a distilled data set, and further effectively improves the quality of the distilled data set. Furthermore, the target semantic search model is trained based on the distilled data set, so that the accuracy of the trained target semantic search model can be improved.

[0090] The training method for a semantic search model in this embodiment can integrate the capabilities of at least two original semantic search models into a target semantic search model with a double-tower structure, so as to effectively improve the search ability of the target semantic search model.

[0091] In the training method for the semantic search model in this embodiment, at least two original semantic search models include a sparse vector search model, which allows the target semantic search model to learn a certain degree of sparse vector search ability, improves the resolution ability of lexical fine sort matching scenarios, improves the generalization ability of the target semantic search model, and has better zero-sample domain transfer performance.

[0092] In addition, for the existing double-tower structure semantic search model, the embodiment of the present disclosure maintains the characteristics of rapid deployment and high search efficiency of the double-tower model without changing the structure of the double-tower model, and can be widely applied to large-scale search scenarios.

[0093] The semantic search model training method of this embodiment can be flexibly expanded when applied using a scheme that integrates the capabilities of multiple types of original semantic search models. When a model or system with higher capabilities in some respect emerges, this scheme can be adopted to effectively absorb the advantages of that system and improve the performance of the target semantic search model.

[0094] Fig. 5 is a schematic diagram according to a fourth embodiment of the present invention. As shown in Fig. 5, this embodiment provides a semantic search model training device 500, which includes a type acquisition module 501 for acquiring a target query sentence type corresponding to each of at least two original semantic search models, which is a query sentence type that the original semantic search models have the highest accuracy in processing various types of query sentences, a data acquisition module 502 for acquiring a distilled dataset based on at least two original semantic search models, the target query sentence types corresponding to each original semantic search model, and a corpus previously constructed, and a training module 503 for training a target semantic search model based on the distilled dataset.

[0095] The implementation principle and technical effect of the semantic search model training apparatus 500 of this embodiment, which uses the above modules to realize the training of a semantic search model, are the same as those of the above-mentioned related method embodiment. For details, please refer to the description of the above-mentioned related method embodiment, and will not be described in detail here.

[0096] Fig. 6 is a schematic diagram according to a fifth embodiment of the present disclosure. This embodiment provides a training apparatus 600 for a semantic search model, which further describes the technical solution of the embodiment shown in Fig. 5 above in detail. As shown in Fig. 6, this embodiment provides a training apparatus 600 for a semantic search model, which includes a type acquisition module 601, a data acquisition module 602, and a training module 603, which are modules with the same names and functions as those shown in Fig. 5.

[0097] Here, the data acquisition module 602 includes a sentence acquisition unit 6021 that acquires target query sentences corresponding to each original semantic search model based on a target query sentence type corresponding to each original semantic search model, a linguistic material acquisition unit 6022 that recalls a predetermined number of recall linguistic materials from the corpus based on each original semantic search model and the target query sentences corresponding to each original semantic search model, and a generation unit 6023 that generates a distilled dataset based on the target query sentences corresponding to each original semantic search model and the predetermined number of recall linguistic materials recalled accordingly.

[0098] Furthermore, in one embodiment of the present disclosure, the type acquisition module 601 acquires a target query sentence type corresponding to each original semantic search model based on a pre-set test set corresponding to each type of query sentence.

[0099] Furthermore, in one embodiment of the present disclosure, the type acquisition module 601 acquires a target query sentence type corresponding to each original semantic search model based on the attributes of each original semantic search model.

[0100] Further, in one embodiment of the present disclosure, the generation unit 6023 uses a pre-trained fine sorting model to screen the target query sentences corresponding to each original semantic search model and a predetermined number of recalled linguistic materials recalled accordingly to generate a distilled dataset.

[0101] Further, in one embodiment of the present disclosure, the generation unit 6023 screens positive sample data based on the fine sorting model, the target query sentence corresponding to each original semantic search model, and a predetermined number of recall linguistic materials recalled accordingly, and screens negative sample data based on the fine sorting model, the target query sentence corresponding to each original semantic search model, and a predetermined number of recall linguistic materials recalled accordingly, and stores the positive sample data and the negative sample data in a distilled dataset.

[0102] Furthermore, in one embodiment of the present disclosure, the generation unit 6023 uses a fine sorting model to calculate a correlation score between the target query sentence and each of a predetermined number of recall linguistic materials recalled accordingly, deletes recall linguistic materials having a correlation score smaller than a preset threshold from the first N (N is a positive integer greater than 1) of the predetermined number of recall linguistic materials, and constructs positive sample data based on the target query sentence and each of the remaining recall linguistic materials among the first N of the predetermined number of recall linguistic materials.

[0103] Furthermore, in one embodiment of the present disclosure, for a target query sentence corresponding to each original semantic search model, the generation unit 6023 selects recall language materials whose correlation scores are smaller than a predetermined threshold from among the N+1th (N is a positive integer greater than 1) or later recall language materials among a predetermined number of recall language materials, and constructs negative sample data based on the target query sentence and the recall language materials whose correlation scores are smaller than a predetermined threshold and selected from the N+1th or later recall language materials among the predetermined number of recall language materials.

[0104] Furthermore, in one embodiment of the present disclosure, the generation unit 6023 uses a fine sorting model to calculate, for a target query sentence corresponding to each original semantic search model, a correlation score between the corresponding target query sentence and each of a predetermined number of recall language materials recalled accordingly, and screens negative sample data from all the recall language materials by at least two original semantic search models in a predetermined manner based on the correlation scores of each target query sentence and each recall language material.

[0105] The implementation principle and technical effect of the semantic search model training apparatus 600 of this embodiment is the same as that of the embodiment of the related method described above, which uses the above modules to realize the training of a semantic search model. For details, please refer to the description of the embodiment of the related method described above, and will not be described in detail here.

[0106] In the technical solution disclosed herein, the acquisition, storage, application, etc. of personal information of users involved are in accordance with the provisions of relevant laws and regulations and are not contrary to public order and morals.

[0107] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0108] 7 shows a schematic block diagram of an exemplary electronic device 700 capable of implementing embodiments of the present disclosure. The electronic device may represent various forms of digital computers, such as laptops, desktop computers, workbenches, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as PDAs, mobile phones, smartphones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present disclosure as described and / or claimed herein.

[0109] 7, the device 700 includes a computing means 701 capable of performing various appropriate operations and processes according to a computer program stored in a read only memory (ROM) 702 or loaded from a storage means 708 into a random access memory (RAM) 703. The RAM 703 may store various programs and data necessary for the operation of the device 700. The computing means 701, the ROM 702, and the RAM 703 are connected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0110] Several components of the device 700 are connected to the I / O interface 705, including input means 706, e.g. a keyboard, mouse, etc., output means 707, e.g. various types of displays, speakers, etc., storage means 708, e.g. magnetic disks, optical disks, etc., and communication means 709, e.g. a network card, modem, wireless communication transceiver, etc. The communication means 709 enables the device 700 to exchange information / data with other devices via computer networks, e.g. the Internet, and / or various telecommunication networks.

[0111] The computing means 701 may be various general-purpose and / or dedicated processing components having processing and computing capabilities. Some examples of the computing means 701 include, but are not limited to, a central processing unit (CPU), a graphic processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that execute machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing means 701 executes various methods and processes described above, such as the methods of the present disclosure. For example, in some embodiments, the methods of the present disclosure may be implemented as a computer software program physically embodied in a machine-readable medium, such as the storage means 708. In some embodiments, some or all of the computer program may be loaded and / or installed in the device 700 via the ROM 702 and / or the communication means 709. When the computer program is loaded into the RAM 703 and executed by the computing means 701, one or more steps of the above methods of the present disclosure may be executed. Alternatively, in other embodiments, the computing means 701 may be configured in any other suitable manner (eg, via firmware) to perform the methods of the present disclosure.

[0112] Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), field programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include being embodied in one or more computer programs that may be executed and / or interpreted on a programmable system that includes at least one programmable processor. The programmable processor may be a special purpose or general purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and may transfer data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] Program codes for implementing the methods of the present disclosure can be written using any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus such that, when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are performed. The program codes can be executed entirely on the machine, partially on the machine, as a stand-alone package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0114] In the context of this disclosure, a machine-readable medium is a tangible medium that can contain or store a program used by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More specific examples of machine-readable storage media include one or more line-based electrical connections, portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0115] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer that includes a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) for providing input to the computer by the user. Other types of devices may also be used to provide interaction with a user. For example, feedback provided to the user may be any form of sensing feedback (e.g., visual feedback, auditory feedback, or haptic feedback), and input from the user may be received in any form (including sound input, speech input, or tactile input).

[0116] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network ("LAN"), a wide area network ("WAN"), and an internetwork.

[0117] The computer system may include clients and servers. The clients and servers are generally remote from each other and typically interact through a communication network. The relationship of clients and servers arises by virtue of computer programs running on the corresponding computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server in combination with a blockchain.

[0118] It should be understood that steps can be rearranged, added, or removed using the various types of flows shown above. For example, each step described in the present disclosure can be performed in a parallel order or a sequential order, or can be performed in a different order, and is not limited thereto, as long as the desired results of the technical proposal disclosed in the present disclosure can be achieved.

[0119] The above specific embodiments do not constitute limitations on the scope of protection of the present disclosure. Those skilled in the art should understand that various modifications, combinations, partial combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present disclosure should be included within the scope of protection of the present disclosure.

Claims

1. A computer-implemented method for training a semantic search model, comprising: Obtaining a target query sentence type corresponding to each of at least two original semantic search models, the target query sentence type being the most accurate query sentence type when the original semantic search models process various types of query sentences; Obtaining a distilled dataset based on at least two original semantic search models, a target query sentence type corresponding to each of the original semantic search models, and a pre-constructed corpus; training a target semantic search model based on the distilled dataset; How to train a semantic search model.

2. Obtaining a target query sentence type corresponding to each of the at least two original semantic search models includes: Obtaining the target query sentence type corresponding to each of the original semantic search models based on a pre-defined test set corresponding to various types of query sentences; The method for training a semantic search model according to claim 1 .

3. Obtaining a target query sentence type corresponding to each of the at least two original semantic search models includes: obtaining the target query sentence type corresponding to each of the original semantic search models based on attributes of each of the original semantic search models; The method for training a semantic search model according to claim 1 .

4. Obtaining a distilled dataset based on at least two original semantic search models, a target query sentence type corresponding to each of the original semantic search models, and a pre-constructed corpus, Obtaining a target query sentence corresponding to each of the original semantic search models according to a target query sentence type corresponding to each of the original semantic search models; Recalling a predetermined number of recall language materials from the corpus based on each of the original semantic search models and a target query sentence corresponding to each of the original semantic search models; generating the distilled dataset based on target query sentences corresponding to each of the original semantic search models and the predetermined number of recall linguistic materials recalled accordingly; A method for training a semantic search model according to any one of claims 1 to 3.

5. generating the distilled dataset based on the target query sentences corresponding to each of the original semantic search models and the predetermined number of recall linguistic materials recalled accordingly, using a pre-trained fine sorting model to screen the target query sentences corresponding to each of the original semantic search models and the predetermined number of recalled linguistic materials recalled accordingly to generate the distilled dataset; The method for training a semantic search model according to claim 4.

6. Using a pre-trained fine sorting model to screen the target query sentences corresponding to each of the original semantic search models and the predetermined number of recall linguistic materials recalled accordingly to generate the distilled dataset, Screening positive sample data based on the fine sorting model, the target query sentence corresponding to each of the original semantic search models, and the predetermined number of recall linguistic materials recalled accordingly; Screening negative sample data based on the fine sorting model, the target query sentence corresponding to each of the original semantic search models, and the predetermined number of recall linguistic materials recalled accordingly; storing the positive sample data and the negative sample data in the distilled data set. The method for training a semantic search model according to claim 5.

7. Screening positive sample data based on the fine sorting model, the target query sentence corresponding to each of the original semantic search models, and the predetermined number of recall linguistic materials recalled accordingly, For each target query sentence corresponding to the original semantic search model, calculating a correlation score between the target query sentence and each of the predetermined number of recall linguistic materials recalled correspondingly using the fine sort model; removing recall language materials having the correlation score smaller than a preset threshold from the first N (N is a positive integer greater than 1) of the predetermined number of recall language materials; constructing the positive sample data based on the target query sentence and each remaining recall linguistic material among the first N in the predetermined number of recall linguistic materials; The method for training a semantic search model according to claim 6.

8. Screening negative sample data based on the fine sorting model, the target query sentence corresponding to each of the original semantic search models, and the predetermined number of recall linguistic materials recalled accordingly, For each target query sentence corresponding to the original semantic search model, select recall language materials having a correlation score smaller than a preset threshold from among the N+1th (N is a positive integer greater than 1) or later recall language materials in the predetermined number of recall language materials; and constructing negative sample data based on the target query sentence and recall language materials selected from the N+1th or later recall language materials in the predetermined number of recall language materials, the correlation score of which is less than a preset threshold value. The method for training a semantic search model according to claim 6.

9. Screening negative sample data based on the fine sorting model, the target query sentence corresponding to each of the original semantic search models, and the predetermined number of recall linguistic materials recalled accordingly, For each target query sentence corresponding to the original semantic search model, calculating a correlation score between the corresponding target query sentence and each of the predetermined number of recall linguistic materials recalled correspondingly using the fine sort model; and screening the negative sample data from all recall language materials according to the at least two original semantic search models based on a correlation score between each of the target query sentences and each of the recall language materials in a preset manner. The method for training a semantic search model according to claim 6.

10. An apparatus for training a semantic search model, comprising: a type acquisition module for acquiring a target query sentence type corresponding to each of at least two original semantic search models, the target query sentence type being the most accurate query sentence type when the original semantic search model processes various types of query sentences; A data acquisition module for acquiring a distilled dataset based on at least two original semantic search models, a target query sentence type corresponding to each of the original semantic search models, and a pre-constructed corpus; a training module for training a target semantic search model based on the distilled dataset; A training device for semantic retrieval models.

11. The type acquisition module includes: Obtaining the target query sentence type corresponding to each of the original semantic search models based on a pre-defined test set corresponding to various types of query sentences; An apparatus for training a semantic search model according to claim 10.

12. The type acquisition module includes: obtaining the target query sentence type corresponding to each of the original semantic search models based on the attributes of each of the original semantic search models; An apparatus for training a semantic search model according to claim 10.

13. The data acquisition module includes: a sentence acquisition unit that acquires a target query sentence corresponding to each of the original semantic search models according to a target query sentence type corresponding to each of the original semantic search models; a language material acquisition unit that recalls a predetermined number of recall language materials from the corpus based on each of the original semantic search models and a target query sentence corresponding to each of the original semantic search models; and a generator for generating the distilled dataset based on a target query sentence corresponding to each of the original semantic search models and the predetermined number of recall linguistic materials recalled accordingly. An apparatus for training a semantic search model according to any one of claims 10 to 12.

14. The generation unit is Using a pre-trained fine sorting model, screen the target query sentences corresponding to each of the original semantic search models and the predetermined number of recalled linguistic materials recalled accordingly to generate the distilled dataset. An apparatus for training a semantic search model according to claim 13.

15. The generation unit is Screening positive sample data based on the fine sorting model, the target query sentence corresponding to each of the original semantic search models, and the predetermined number of recall linguistic materials recalled accordingly; Screening negative sample data based on the fine sorting model, the target query sentence corresponding to each of the original semantic search models, and the predetermined number of recall linguistic materials recalled accordingly; storing the positive sample data and the negative sample data in the distilled data set; An apparatus for training a semantic search model according to claim 14.

16. The generation unit is For each target query sentence corresponding to the original semantic search model, calculate a correlation score between the target query sentence and each of the predetermined number of recall linguistic materials recalled correspondingly using the fine sort model; Removing recall language materials having correlation scores smaller than a preset threshold from the first N (N is a positive integer greater than 1) of the predetermined number of recall language materials; constructing the positive sample data based on the target query sentence and each remaining recall linguistic material among the first N of the predetermined number of recall linguistic materials; An apparatus for training a semantic search model according to claim 15.

17. The generation unit is For each target query sentence corresponding to the original semantic search model, select recall language materials having a correlation score smaller than a preset threshold from among the N+1th (N is a positive integer greater than 1) or later recall language materials in the predetermined number of recall language materials; constructing negative sample data based on the target query sentence and recall language materials selected from the N+1th or later recall language materials in the predetermined number of recall language materials, the correlation score of which is less than a preset threshold value; An apparatus for training a semantic search model according to claim 15.

18. The generation unit is For each target query sentence corresponding to the original semantic search model, calculate a correlation score between the corresponding target query sentence and each of the predetermined number of recall linguistic materials recalled accordingly using the fine sort model; Screening the negative sample data from all the recall language materials according to the at least two original semantic search models based on a correlation score between each of the target query sentences and each of the recall language materials in a preset manner; An apparatus for training a semantic search model according to claim 15.

19. At least one processor; a memory communicatively coupled to the at least one processor; An electronic device having stored in the memory instructions executable by the at least one processor, the commands, when executed by the at least one processor, causing the at least one processor to perform the method for training a semantic search model described in any one of claims 1 to 3.

20. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method for training a semantic search model according to any one of claims 1 to 3.

21. A computer program product which, when executed by a processor, implements the method for training a semantic search model according to any one of claims 1 to 3.

Citation Information

Patent Citations

  • Semantic matching model training method and device, semantic matching model matching method and device, equipment and storage medium

    CN112307048A

  • Learning device, learning method, and program

    JP2022002023A