Method and apparatus for training semantic feature vector generation model and semantic search

Abstract

Embodiments of the present specification provide a method and device for training a semantic feature vector generation model and semantic search. In the method for training the semantic feature vector generation model, text extraction is performed on a sample reference text of an obtained training sample to obtain corresponding first and second partial texts; the sample query text of the obtained training sample and the sample reference text and the corresponding first and second partial texts are provided to a current semantic feature vector generation model to obtain corresponding semantic feature vectors; the global matching information and the local matching information of the sample query text and the sample reference text are determined based on the similarity between the semantic feature vectors corresponding to the sample query text and the sample reference text, the first partial text and the second partial text respectively; and the loss value of the loss function determined according to the global matching information and the local matching information is used to adjust the parameters of the current semantic feature vector generation model.

Description

Technical Field

[0001] The embodiments in this specification generally relate to the field of Natural Language Processing (NLP) technology, and in particular to methods for training semantic feature vector generation models, semantic similarity determination methods, semantic search methods, and apparatus. Background Technology

[0002] With the rapid development of artificial intelligence technology, natural language processing (NLP) technology is finding increasingly widespread applications. For example, in the field of semantic search, to find texts semantically similar to the input text from a large dataset, various text vectorization methods are typically used to convert the entire text into semantic feature vectors, and then the similarity between these vectors is calculated to determine the similarity between the texts. Existing text vectorization methods often employ optimization pre-training or fine-tuning techniques to improve the accuracy of semantic representation. However, since these methods primarily target general text representation, their matching performance is unsatisfactory in scenarios involving fuzzy semantic matching specific to certain needs. Therefore, maximizing the representational power of text semantic feature vectors has become one of the effective means to further improve the determination of text similarity and the effectiveness of semantic search. Summary of the Invention

[0003] In view of the above, embodiments of this specification provide a method for training a semantic feature vector generation model, a semantic similarity determination method, a semantic search method, and an apparatus. Using this method and apparatus, the representational capability of text semantic feature vectors can be improved, thereby helping to determine semantic similarity more accurately, and particularly contributing to the effective improvement of the recall rate of semantic search methods for text semantic fuzzy matching scenarios with specific needs.

[0004] According to one aspect of an embodiment of this specification, a method for training a semantic feature vector generation model is provided, comprising: acquiring a training sample set, wherein the training sample set includes sample query text and sample reference text; performing text extraction on the sample reference text to obtain a first part of text and a second part of text corresponding to the sample reference text; and performing the following training steps: providing the training samples and the corresponding first part of text and second part of text in the training sample set to a current semantic feature vector generation model to obtain semantic feature vectors corresponding to the sample query text, the sample reference text, the first part of text, and the second part of text, respectively; determining global matching information and local matching information between the sample query text and the sample reference text based on the similarity between the semantic feature vectors corresponding to the sample query text and the sample reference text, the first part of text, and the second part of text, respectively; determining the loss value of a loss function based on the global matching information and the local matching information; and adjusting the parameters of the current semantic feature vector generation model according to the loss value and continuing to execute the training steps when the training termination condition is not met.

[0005] According to another aspect of the embodiments of this specification, a semantic similarity determination method is provided, comprising: providing a query text and a reference text to a semantic feature vector generation model respectively to obtain semantic feature vectors corresponding to the query text and the reference text respectively, wherein the semantic feature vector of the reference text is fused with information of a portion of the text extracted based on the reference text; and determining the semantic similarity between the query text and the reference text based on the semantic feature vectors corresponding to the query text and the reference text respectively.

[0006] According to another aspect of the embodiments of this specification, a semantic similarity determination method is provided, comprising: extracting text from a reference text to obtain a first part of text and a second part of text corresponding to the reference text; providing a query text, the reference text, the first part of text and the second part of text corresponding to the reference text to a semantic feature vector generation model to obtain semantic feature vectors corresponding to the query text, the reference text, the first part of text and the second part of text corresponding to the reference text, respectively; determining global matching information and local matching information between the query text and the reference text based on the similarity between the semantic feature vector corresponding to the query text and the semantic feature vectors corresponding to the first part of text and the second part of text corresponding to the reference text, respectively; and determining the semantic similarity between the query text and the reference text based on the determined global matching information and local matching information.

[0007] According to another aspect of the embodiments of this specification, a semantic search method is provided, comprising: receiving query text provided by a user; determining the semantic similarity between the query text and each reference text in a reference text set according to the semantic similarity determination method as described above; determining a semantic search result from the reference text set based on the semantic similarity between the query text and each reference text; and providing the semantic search result to the user.

[0008] According to another aspect of the embodiments of this specification, an apparatus for training a semantic feature vector generation model is provided, comprising: a sample acquisition unit configured to acquire a training sample set, wherein the training sample set includes a sample query text and a sample reference text; a text extraction unit configured to extract text from the sample reference text to obtain a first part of text and a second part of text corresponding to the sample reference text; and a training unit configured to provide the training samples in the training sample set and the corresponding first part of text and second part of text to a current semantic feature vector generation model to obtain semantic feature vectors corresponding to the sample query text, the sample reference text, the first part of text, and the second part of text, respectively; determining global matching information and local matching information between the sample query text and the sample reference text based on the similarity between the semantic feature vectors corresponding to the sample query text and the sample reference text, the first part of text, and the second part of text, respectively; determining a loss value of a loss function based on the global matching information and the local matching information; and adjusting the parameters of the current semantic feature vector generation model and continuing the training steps based on the loss value when the training termination condition is not met.

[0009] According to another aspect of the embodiments of this specification, a semantic similarity determination apparatus is provided, comprising: a vector generation unit configured to provide a query text and a reference text to a semantic feature vector generation model respectively, to obtain semantic feature vectors corresponding to the query text and the reference text respectively, wherein the semantic feature vector of the reference text incorporates information of a portion of the text extracted based on the reference text; and a similarity determination unit configured to determine the semantic similarity between the query text and the reference text based on the semantic feature vectors corresponding to the query text and the reference text respectively.

[0010] According to another aspect of the embodiments of this specification, a semantic similarity determination apparatus is provided, comprising: an extraction unit configured to extract text from a reference text to obtain a first part of text and a second part of text corresponding to the reference text; a vectorization unit configured to provide a query text, the reference text, the first part of text and the second part of text corresponding to the reference text to a semantic feature vector generation model to obtain semantic feature vectors corresponding to the query text, the reference text, the first part of text and the second part of text corresponding to the reference text; a matching information determination unit configured to determine global matching information and local matching information between the query text and the reference text based on the similarity between the semantic feature vector corresponding to the query text and the semantic feature vectors corresponding to the first part of text and the second part of text corresponding to the reference text; and a semantic similarity determination unit configured to determine the semantic similarity between the query text and the reference text based on the determined global matching information and local matching information.

[0011] According to another aspect of the embodiments of this specification, a semantic search apparatus is provided, comprising: a receiving unit configured to receive query text provided by a user; similarity determination units configured to determine the semantic similarity between the query text and each reference text in a reference text set according to the semantic similarity determination method described above; a semantic search unit configured to determine semantic search results from the reference text set based on the semantic similarity between the query text and each reference text; and a result providing unit configured to provide the semantic search results to the user.

[0012] According to another aspect of the embodiments of this specification, an apparatus for training a semantic feature vector generation model is provided, comprising: at least one processor, a memory coupled to the at least one processor, and a computer program stored in the memory, wherein the at least one processor executes the computer program to implement the method for training a semantic feature vector generation model as described above.

[0013] According to another aspect of the embodiments of this specification, a semantic similarity determination apparatus is provided, comprising: at least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, wherein the at least one processor executes the computer program to implement the semantic similarity determination method as described above.

[0014] According to another aspect of the embodiments of this specification, a semantic similarity determination apparatus is provided, comprising: at least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, wherein the at least one processor executes the computer program to implement the semantic similarity determination method as described above.

[0015] According to another aspect of the embodiments of this specification, a semantic search apparatus is provided, comprising: at least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, wherein the at least one processor executes the computer program to implement the semantic search method as described above.

[0016] According to another aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements the method for training a semantic feature vector generation model, a semantic similarity determination method, a semantic similarity determination method, or a semantic search method as described above.

[0017] According to another aspect of the embodiments of this specification, a computer program product is provided, including a computer program that is executed by a processor to implement the method for training a semantic feature vector generation model, a semantic similarity determination method, a semantic similarity determination method, or a semantic search method as described above. Attached Figure Description

[0018] A further understanding of the nature and advantages of this specification can be achieved by referring to the following figures. In the figures, similar components or features may have the same reference numerals.

[0019] Figure 1 An exemplary architecture of a method, semantic similarity determination method, semantic search method, and apparatus for training a semantic feature vector generation model according to embodiments of this specification is shown.

[0020] Figure 2 A flowchart illustrating an example of a method for training a semantic feature vector generation model according to an embodiment of this specification is shown.

[0021] Figure 3 A flowchart illustrating an example of a process for determining global and local matching information between a sample query text and a sample reference text according to an embodiment of this specification is provided.

[0022] Figure 4 A flowchart illustrating an example of a process for determining partial matching information between sample query text and sample reference text according to an embodiment of this specification is provided.

[0023] Figure 5 A schematic diagram illustrating an example of a method for training a semantic feature vector generation model according to an embodiment of this specification is shown.

[0024] Figure 6 A flowchart illustrating an example of a semantic similarity determination method according to an embodiment of this specification is shown.

[0025] Figure 7 A flowchart illustrating yet another example of a semantic similarity determination method according to an embodiment of this specification is shown.

[0026] Figure 8 A schematic diagram illustrating yet another example of a semantic similarity determination method according to an embodiment of this specification is shown.

[0027] Figure 9 A flowchart illustrating an example of a semantic search method according to an embodiment of this specification is shown.

[0028] Figure 10 A block diagram of an example apparatus for training a semantic feature vector generation model according to an embodiment of this specification is shown.

[0029] Figure 11 A block diagram of an example of a semantic similarity determination apparatus according to embodiments of this specification is shown.

[0030] Figure 12 A block diagram of yet another example of a semantic similarity determination apparatus according to embodiments of this specification is shown.

[0031] Figure 13 A block diagram illustrating an example of a semantic search apparatus according to an embodiment of this specification is shown.

[0032] Figure 14 A block diagram of an example apparatus for training a semantic feature vector generation model according to an embodiment of this specification is shown.

[0033] Figure 15 A block diagram of an example of a semantic similarity determination apparatus according to embodiments of this specification is shown.

[0034] Figure 16 A block diagram of yet another example of a semantic similarity determination apparatus according to embodiments of this specification is shown.

[0035] Figure 17 A block diagram illustrating an example of a semantic search apparatus according to an embodiment of this specification is shown. Detailed Implementation

[0036] The subject matter described herein will be discussed below with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of the embodiments described herein. Various processes or components may be omitted, substituted, or added as needed in the various examples. Furthermore, features described in some examples may be combined in other examples.

[0037] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless explicitly indicated by the context, the definition of a term shall remain consistent throughout the specification.

[0038] In this specification, the term "semantic feature vector" can refer to any vector that can be used to represent semantic information. For example, it can be a word embedding obtained in various ways.

[0039] In this specification, the term "semantic feature vector generation model" can be understood as a mapping that converts text into semantic feature vectors. Typically, various artificial neural networks (ANNs) can be used to achieve this functionality. For example, semantic feature vector generation models can employ recurrent neural networks (RNNs), long short-term memory (LSTM) networks, Transformers-based encoders, and pre-trained models such as BERT (Bidirectional Encoder Representation from Transformers), XLNet, and ALBERT.

[0040] The following describes in detail, with reference to the accompanying drawings, the method for training a semantic feature vector generation model, the semantic similarity determination method, the semantic search method, and the apparatus according to embodiments of this specification.

[0041] Figure 1An exemplary architecture 100 for a method, semantic similarity determination method, semantic search method, and apparatus for training a semantic feature vector generation model according to embodiments of this specification is shown.

[0042] exist Figure 1 In this context, network 110 is used to interconnect terminal device 120 and application server 130.

[0043] Network 110 can be any type of network capable of interconnecting network entities. Network 110 can be a single network or a combination of various networks. In terms of coverage, network 110 can be a local area network (LAN), a wide area network (WAN), etc. In terms of the carrying medium, network 110 can be a wired network, a wireless network, etc. In terms of data switching technology, network 110 can be a circuit-switched network, a packet-switched network, etc.

[0044] Terminal device 120 can be any type of electronic computing device capable of connecting to network 110, accessing servers or websites on network 110, processing data or signals, etc. For example, terminal device 120 can be a desktop computer, laptop computer, tablet computer, smartphone, etc. Although in Figure 1 Only one terminal device is shown in the diagram, but it should be understood that a different number of terminal devices may be connected to network 110.

[0045] In one implementation, terminal device 120 can be used by a user. Terminal device 120 may include an application client (e.g., application client 121) that can provide various services to the user. In some cases, application client 121 may interact with application server 130. For example, application client 121 may transmit user-inputted messages to application server 130 and receive responses associated with those messages from application server 130. However, it should be understood that in other cases, application client 121 may also generate responses to user-inputted messages locally, rather than interacting with application server 130. In this document, "message" can refer to any input information, such as query text 1211 input by the user.

[0046] Application server 130 can be connected to reference text database 133. Reference text database 133 may include multiple reference texts. Optionally, reference text database 133 may also include semantic feature vectors corresponding to each of the multiple reference texts. In one example, the semantic feature vectors corresponding to each reference text can be obtained by providing each reference text separately to semantic feature vector generation model 132. Optionally, application server 130 can also obtain semantic feature vector generation model 132 trained on training sample set 131 through various methods. For example, application server 130 can train semantic feature vector generation model 132 locally or obtain a trained semantic feature vector generation model 132 from other electronic devices with communication connections.

[0047] It should be understood that Figure 1 All network entities shown are exemplary, and any other network entities may be involved in Architecture 100 depending on the specific application requirements.

[0048] Figure 2 A flowchart of a method 200 for training a semantic feature vector generation model according to an embodiment of this specification is shown.

[0049] like Figure 2 As shown, at 210, the training sample set is obtained.

[0050] In this embodiment, the training sample set may include sample text pairs. Each sample text pair may include a sample query text and a sample reference text. The content and format of the sample reference text can be flexibly set according to the actual application scenario and are not limited here. In one example, the sample query text and the sample reference text can be any two texts used to determine semantic similarity. In another example, the sample query text can be text entered by the user through the input terminal in the history; the sample reference text can be text in the history used to match the sample query text.

[0051] In step 220, text extraction is performed on the sample reference text to obtain the first part of the text and the second part of the text corresponding to the sample reference text.

[0052] In this embodiment, text extraction of the sample reference text can be performed in various ways. The first part of the text and the second part of the text mentioned above can be text fragments extracted from the sample reference text. As an example, the sample reference text can be information text. Text extraction of the sample reference text can be performed based on the position of the text (e.g., at the title) or specific words (e.g., "abstract", "keywords") to obtain the first part of the text (e.g., title text) and the second part of the text (e.g., abstract text, keywords, etc.) corresponding to each piece of information. The sample reference text can also be processed, for example, by using automatic text summarization technology to generate paragraph summaries for the text fragments of each piece of information (e.g., each text paragraph, each 10 lines of text, each page of text, etc.). The paragraph summaries are then used as the first part of the text or the second part of the text. It is understood that the sample reference text can also be a collection of papers, a Chinese encyclopedia, a product introduction, etc., and is not limited here.

[0053] In some optional implementations of this embodiment, the first part of the text may include the title of the sample reference text. The second part of the text may include the keywords of the sample reference text. It is understood that the title and keywords of the text may be extracted directly from the text of the sample reference text, or they may be generated based on the sample reference text using automatic title generation technology or keyword extraction technology.

[0054] Optionally, the semantic feature vector generation model trained by the above-mentioned method for training the semantic feature vector generation model can be applied to service search. The service search can include searching for service-related keywords such as ordering food, shopping, express delivery, weather, and government services. The first part of the text can include the name of the service. For example, "XX takeout," "XX food," "XX shopping," "XX express delivery," "XX weather," "XX government service platform," etc. The second part of the text can include keywords related to the functions involved in the service. For example, "ordering food," "reviews," "product categories," "tracking express delivery," "sending express delivery," "picking up express delivery," "15-day forecast," "leaving comments and suggestions," "all-in-one card," "public rental housing," "social security," "housing provident fund," etc. It can be understood that in the service search domain, the above-mentioned raw corpus can be, for example, the text content contained in the pages of various services (such as mini-programs, subscription accounts, etc.).

[0055] In step 230, the training samples in the training sample set, along with the corresponding first and second part texts, are provided to the current semantic feature vector generation model to obtain semantic feature vectors corresponding to the sample query text, the sample reference text, the first part text, and the second part text, respectively.

[0056] In this embodiment, the current semantic feature vector generation model can vary depending on the training stage. For example, the current semantic feature vector generation model can be a semantic feature vector generation model with initialized parameters, a pre-trained semantic feature vector generation model, or a semantic feature vector generation model after several parameter adjustments.

[0057] In one example, the semantic feature vector generation model described above can be a bidirectional encoder based on Transformers. For instance, an encoder consisting of 12 Transformers layers can be pre-trained using corpus related to the content of the training sample set. Then, the pre-trained encoder can be used to encode the query text, the reference text, the first part of the text, and the second part of the text to obtain the semantic feature vectors corresponding to the query text, the reference text, the first part of the text, and the second part of the text, respectively.

[0058] In 240, the global matching information and local matching information between the sample query text and the sample reference text are determined based on the similarity between the semantic feature vectors corresponding to the sample query text and the sample reference text, the first part of the text and the second part of the text, respectively.

[0059] In this embodiment, the aforementioned global matching information can be used to characterize the degree of matching between the overall sample query text and the overall sample reference text. The aforementioned local matching information can be used to characterize the degree of matching between the overall sample query text and a part of the sample reference text. The aforementioned part of the sample reference text can be represented by at least one of the aforementioned first part of text and second part of text. The similarity representation between the aforementioned semantic feature vectors can include, but is not limited to, at least one of the following: Euclidean distance, cosine similarity, and cosine distance.

[0060] Figure 3 This diagram illustrates an example of a process 300 for determining global and local matching information between a sample query text and a sample reference text according to an embodiment of this specification.

[0061] like Figure 3 As shown in 310, the global matching information between the sample query text and the sample reference text is determined based on the similarity between the semantic feature vector corresponding to the sample reference text and the semantic feature vector corresponding to the sample query text.

[0062] In this embodiment, the similarity between the semantic feature vector corresponding to the sample reference text and the semantic feature vector corresponding to the sample query text can be determined as the global matching information between the sample query text and the sample reference text. In one example, when the training process uses multiple training samples in each iteration, the same number of global matching information as the number of sample pairs composed of the sample query text and the sample reference text can be obtained. Optionally, the representative value of the similarity corresponding to each sample pair composed of the sample query text and the sample reference text can also be determined as the aforementioned global matching information. The aforementioned representative value can be, for example, the average, median, maximum, minimum, etc.

[0063] In step 320, the local matching information between the sample query text and the sample reference text is determined based on the similarity between at least one of the semantic feature vectors corresponding to the first part of the text and the second part of the text and the semantic feature vector corresponding to the sample query text.

[0064] In this embodiment, the similarity between at least one of the semantic feature vectors corresponding to the first part of the text and the second part of the text and the semantic feature vector corresponding to the sample query text can be determined as the local matching information between the sample query text and the sample reference text. In one example, the similarity between the semantic feature vector corresponding to the first part of the text and the semantic feature vector corresponding to the sample query text can be determined as the first similarity. The similarity between the semantic feature vector corresponding to the second part of the text and the semantic feature vector corresponding to the sample query text can be determined as the second similarity. Then, the average of the first similarity and the second similarity can be determined as the local matching information between the sample query text and the sample reference text. In another example, when the training process uses multiple training samples in each iteration, the same number of local matching information as the number of sample pairs composed of the sample query text and the sample reference text can be obtained. Optionally, the representative value of the average similarity of each sample pair composed of the sample query text and the sample reference text can also be determined as the aforementioned local matching information. The aforementioned representative value can be, for example, the average, median, maximum, minimum, etc.

[0065] Based on the foregoing, this solution can combine the first part of the text and the second part of the text to determine the local matching information between the sample query text and the sample reference text.

[0066] Figure 4 This diagram illustrates an example of a process 400 for determining partial matching information between sample query text and sample reference text according to an embodiment of this specification.

[0067] like Figure 4As shown in Figure 410, the semantic similarity between the semantic feature vector corresponding to the sample query text and the semantic feature vectors corresponding to the first part of the text and the second part of the text is determined.

[0068] In this embodiment, the similarity between the semantic feature vector corresponding to the first part of the text and the semantic feature vector corresponding to the sample query text can be determined as the first similarity. The similarity between the semantic feature vector corresponding to the second part of the text and the semantic feature vector corresponding to the sample query text can be determined as the second similarity.

[0069] In step 420, local matching information is determined based on the greater semantic similarity between the semantic feature vector corresponding to the determined sample query text and the semantic feature vectors corresponding to the first part of the text and the second part of the text.

[0070] In this embodiment, the larger value between the first similarity and the second similarity can be determined as the local matching information between the sample query text and the sample reference text. In one example, when the training process uses multiple training samples in each iteration, the same number of local matching information as the number of sample pairs consisting of the sample query text and the sample reference text can be obtained. Optionally, the representative value of the larger similarity value corresponding to each sample pair consisting of the sample query text and the sample reference text can also be determined as the aforementioned local matching information. The aforementioned representative value can be, for example, the average, median, maximum, minimum, etc.

[0071] Based on the foregoing, this scheme can combine text segments in the first and second parts of the text that are more similar to the sample query text to determine the local matching information between the sample query text and the sample reference text, thereby helping to improve the fuzzy matching rate of the semantic feature vectors obtained by using the semantic feature vector generation model.

[0072] Back Figure 2 At 250, the loss value of the loss function is determined based on global matching information and local matching information.

[0073] In this embodiment, the loss value of the loss function can be determined based on global and local matching information in various ways. In one example, the global and local matching information can be fused (e.g., weighted summation) to obtain comprehensive matching information. The training sample set may also include matching information labels. These labels can indicate the semantic similarity between the query text and the reference text. Therefore, the loss value can be determined based on the difference between the comprehensive matching information and the corresponding matching information labels. The loss function may include, for example, the Cross Entropy Loss function, the Mean Square Error loss function, etc.

[0074] In some optional implementations of this embodiment, the loss function may include a mutual information loss term to characterize the consistency between global matching information and local matching information. In one example, the mutual information loss term may be, for example, the absolute value of the difference between global matching information and local matching information.

[0075] Optionally, the mutual information loss term used to characterize the consistency between global and local matching information may include a mutual information loss term used to characterize the difference between the distributions of global and local matching information. In one example, each iteration of the training process can use multiple training samples, thus obtaining global and local matching information corresponding to each training sample. It is evident that multiple global matching information sets can form a distribution of global matching information. Similarly, the distribution of local matching information can also be obtained. Therefore, the aforementioned loss term can be used to characterize the difference between the distributions of global and local matching information. As an example, the aforementioned mutual information loss term can be represented by Kullback-Leibler divergence (KL divergence), Jensen-Shannon divergence, etc.

[0076] For example, the mutual information loss term L mentioned above dc It can be: L dc =D KL [P(y|S q S i )||Q(y)]. Wherein, the above D KL It can be used to characterize KL divergence. The above P(y|S q S i The above can be used to characterize the distribution of local matching information. The above Q(y) can be used to characterize the distribution of global matching information. The above S... q S iThese can be used to characterize the semantic feature vector corresponding to the sample query text and the semantic feature vector corresponding to the sample reference text, respectively.

[0077] It should be noted that by using the mutual information loss term mentioned above, we can achieve the desired local matching result as close as possible to the global matching result.

[0078] Optionally, the loss function may further include a supervised loss term to characterize the difference between the global matching information and the corresponding matching information label. In one example, the supervised loss term may, for example, include cross-entropy loss, mean squared error loss, Info NCE loss, etc. In one example, the supervised loss term may be: Among them, the above S q ·S i+ This can be used to characterize the global matching information corresponding to positive sample pairs. The aforementioned positive sample pairs can be sample pairs whose matching information labels indicate that the query text matches the reference text. The aforementioned S... q S i+ These can be used to represent the semantic feature vectors corresponding to the query text and the reference text in a positive sample pair, respectively. The above *k* can be used to represent the number of training samples used to calculate the supervised loss in this iteration. The above *S*... j It can be used to characterize the semantic feature vector corresponding to any sample reference text of the training samples used to calculate the supervised loss of this iteration process.

[0079] Furthermore, the aforementioned loss function could be, for example, L1 = L dc +L s Among them, L dc and L s The meaning of can be found in the preceding description.

[0080] It should be noted that since the results of local matching lack labels from supervised data for calculating the loss, the maximum value is used to approximate the logits of global matching. Furthermore, in methods such as the BERT model that use token-based vectorization, the semantic feature vector of the complete sample reference text can be derived from the semantic feature vectors of the first and second parts of the text extracted from the sample reference text. Therefore, the results of local matching can be integrated into global matching.

[0081] At 260, determine whether the training termination condition is met.

[0082] In this embodiment, the training termination condition may include, for example, reaching a preset number of iterations, reaching a preset training duration, or the loss value converging.

[0083] At 270, if the training termination condition is not met, adjust the parameters of the current semantic feature vector generation model based on the loss value and continue to execute the above training steps.

[0084] In this embodiment, when the training termination condition is not met, the gradient can be calculated based on the loss value, and the parameters of the current semantic feature vector generation model can be adjusted through the backpropagation algorithm; and the training steps described above can be continued at position 230.

[0085] In some optional implementations of this embodiment, at 280, the current semantic feature vector generation model is determined as the semantic feature vector generation model that has been trained when the training termination condition is met.

[0086] Figure 5 A schematic diagram illustrating an example of a method 500 for training a semantic feature vector generation model according to embodiments of this specification is shown. Figure 5 In the example, a training sample set including the query text and reference text can be obtained first. Then, text extraction can be performed on the reference texts to obtain the first and second parts of the text corresponding to each reference text. Next, the query text, reference text, and corresponding first and second parts of the text from the training sample set are provided to the current semantic feature vector generation model to obtain semantic feature vectors corresponding to the query text, reference text, first part, and second part of the text, respectively. Then, based on the similarity between the semantic feature vector corresponding to the query text and the semantic feature vectors corresponding to the reference text, first part, and second part of the text, global and local matching information between the query text and the reference text can be determined. The loss value of the loss function is then determined based on the determined global and local matching information. Finally, if the training termination condition is not met, the parameters of the current semantic feature vector generation model are adjusted according to the determined loss value.

[0087] use Figures 1-5 The method disclosed in the paper for training a semantic feature vector generation model can combine the overall matching result of the sample query text and the sample reference text with the local matching result of the partial text extracted from the sample reference text to calculate the loss value. Therefore, the semantic feature vector generation model can use the mutual information between global matching and local matching to optimize the representation effect of the generated semantic feature vector, which is especially helpful to improve the recall rate of fuzzy text matching.

[0088] Figure 6 A flowchart illustrating an example of a semantic similarity determination method 600 according to an embodiment of this specification is shown.

[0089] like Figure 6As shown in step 610, the query text and the reference text are respectively provided to the semantic feature vector generation model to obtain the semantic feature vectors corresponding to the query text and the reference text respectively.

[0090] In this embodiment, the semantic feature vector of the reference text can incorporate information from partial text extracted based on the reference text. In one example, the text to be queried can be text entered by the user through an input terminal, and the reference text can be text locally on the server used to match the text to be queried. In another example, both the text to be queried and the reference text can be entered or selected by the user through an input terminal, in which case the text to be queried and the reference text can be any two texts whose semantic similarity needs to be determined.

[0091] In some optional implementations of this embodiment, the semantic feature vector generation model described above can be achieved through methods such as... Figure 1-5 The method described in the Chinese embodiment is used for training.

[0092] In step 620, the semantic similarity between the query text and the reference text is determined based on their respective semantic feature vectors. In one example, the vector similarity between the semantic feature vectors corresponding to the query text and the reference text can be used to determine the semantic similarity between the query text and the reference text. The representation of the vector similarity can include, but is not limited to, at least one of the following: Euclidean distance, cosine similarity, and cosine distance.

[0093] use Figure 6 The semantic similarity determination method disclosed herein can apply the semantic feature vector of a reference text incorporating local textual information to determine the semantic similarity of texts. Since the semantic feature vector of the aforementioned reference text incorporating local textual information has a more reasonable semantic representation capability, it considers the matching situation with some texts when determining the semantic similarity of texts. This allows it to more accurately reflect the semantic information of the reference text and helps improve the accuracy of the semantic similarity determination method.

[0094] Figure 7 A flowchart is shown as yet another example of a semantic similarity determination method 700 according to an embodiment of this specification.

[0095] like Figure 7 As shown, at 710, text extraction is performed on the reference text to obtain the first part of the text and the second part of the text corresponding to the reference text.

[0096] In this embodiment, text extraction can be performed on the reference text in various ways to obtain a first part of the text and a second part of the text corresponding to the reference text. In one example, the text extraction method can be referred to the foregoing. Figure 2The corresponding description of step 220 in the embodiments will not be repeated here.

[0097] In some optional implementations of this embodiment, the semantic similarity determination method described above can be applied to service search. The first part of the text may include the name of the service. The second part of the text may include keywords related to the functions involved in the service. In one example, the above content concerning service search can be referred to the foregoing. Figure 2 The corresponding descriptions of the optional implementations of step 220 in the embodiments are not repeated here.

[0098] In step 720, the query text, the reference text, and the first and second parts of the text corresponding to the reference text are respectively provided to the semantic feature vector generation model to obtain the semantic feature vectors corresponding to the query text, the reference text, and the first and second parts of the text corresponding to the reference text.

[0099] In this embodiment, the semantic feature vector generation model described above can be any model used for text vectorization. It should be noted that the semantic feature vector generation model can be a conventional model, meaning that the semantic feature vectors it generates do not need to incorporate information from local text. Optionally, the semantic feature vector generation model can also be as described above... Figure 2-5 The model trained using the method described in the embodiments. In one example, the query text can be text entered by the user through an input terminal, and the reference text can be text locally on the server used to match the query text. In another example, both the query text and the reference text can be entered or selected by the user through an input terminal, in which case the query text and the reference text can be any two texts whose semantic similarity needs to be determined.

[0100] In step 730, the global matching information and local matching information between the query text and the reference text are determined based on the similarity between the semantic feature vector corresponding to the query text and the reference text, as well as the semantic feature vectors corresponding to the first and second parts of the reference text.

[0101] In this embodiment, the similarity between the semantic feature vector corresponding to the query text and the semantic feature vector corresponding to the reference text can be determined as the global matching information between the query text and the reference text. The similarity between the semantic feature vector corresponding to the query text and the semantic feature vector corresponding to the first part of the reference text can be determined as the first local matching information between the query text and the reference text. The similarity between the semantic feature vector corresponding to the query text and the semantic feature vector corresponding to the second part of the reference text can be determined as the second local matching information between the query text and the reference text. Then, the first and second local matching information can be fused into the aforementioned local matching information using various methods (e.g., weighted summation, maximum value, minimum value, etc.).

[0102] In step 740, based on the determined global and local matching information, the semantic similarity between the query text and the reference text is determined.

[0103] In this embodiment, based on the determined global and local matching information, the semantic similarity between the query text and the reference text can be determined in various ways. For example, weighted summation, maximum value, minimum value, etc., can be used to determine the semantic similarity between the query text and the reference text. Optionally, the weights of the aforementioned weighted summation can be obtained, for example, using a trained network model based on an attention mechanism.

[0104] Figure 8 A schematic diagram illustrating an example of a semantic similarity determination method 800 according to embodiments of this specification is shown. Figure 8 In the example, text extraction can be performed on the reference text first, yielding the first and second parts of the text corresponding to each reference text. Next, the query text, reference text, and their corresponding first and second parts are provided to the semantic feature vector generation model to obtain semantic feature vectors corresponding to the query text, reference text, first part, and second part, respectively. Then, the global and local matching information between the query text and the reference text can be determined based on the similarity between the semantic feature vector corresponding to the query text and the semantic feature vectors corresponding to the reference text, first part, and second part. Finally, the semantic similarity between the query text and the reference text is determined based on the determined global and local matching information.

[0105] use Figures 7-8The semantic similarity determination method disclosed in the paper can comprehensively determine the semantic similarity between the query text and the reference text by combining the global matching information of the complete text of the query text and the local matching information of the query text and the partial text extracted based on the reference text. Furthermore, by using different fusion methods of global matching information and local matching information, it can help improve the applicability of the semantic similarity determination method in different application scenarios.

[0106] Figure 9 A flowchart illustrating an example of a semantic search method 900 according to an embodiment of this specification is shown.

[0107] like Figure 9 As shown, at 910, the query text provided by the user is received. In one example, the query text provided by the user can be received in various ways. For example, the query text can be text directly entered by the user, or it can be converted into text by optical character recognition (OCR) or automatic speech recognition (ASR) from images, videos, voice, etc. entered by the user using the user's terminal. There is no limitation here.

[0108] In step 920, the semantic similarity between the query text and each reference text in the reference text set is determined according to the semantic similarity determination method described above. In one example, this can be done according to... Figures 6-8 The semantic similarity determination method described herein is used to determine the semantic similarity between the query text and each reference text in the reference text set. The reference text set may include multiple reference texts. The reference texts included in the reference text set can be set according to actual needs. For example, it may include all candidate texts or a subset of candidate texts recalled based on various coarse screening methods.

[0109] In step 930, semantic search results are determined from the reference text set based on the semantic similarity between the query text and each reference text. In one example, semantic search results can be determined from the reference text set in various ways based on the semantic similarity between the query text and each reference text. For example, the reference texts with the highest semantic similarity can be determined as semantic search results. Another example is that reference texts with semantic similarity greater than a preset threshold can be used as candidate results, and then several reference texts can be selected as semantic search results from these candidate results through methods such as random selection or selection based on user preferences.

[0110] At step 940, the semantic search results are presented to the user. In one example, the semantic search results can be presented to the user in various forms. For example, they can be arranged in a list format, ordered from highest to lowest semantic similarity. Optionally, the corresponding semantic similarity scores can also be displayed near each semantic search result.

[0111] It should be noted that the users provided with the semantic search results can be the same users as those described in step 910 above, or users using the same client as those described in step 910 above; there is no limitation here.

[0112] In some optional implementations of this embodiment, the semantic search method described above can be applied to service search. The semantic search results may include service pages that match the query text. For a more detailed description of service search, please refer to the foregoing. Figure 2 The corresponding descriptions of the optional implementation methods in the embodiments are not repeated here.

[0113] Based on the above, a semantic search method applicable to fields such as service search is provided. Compared to existing service search methods that rely on exact matching of search text with service titles or keywords, text vectorization can improve the recall rate of fuzzy matching. Furthermore, compared to existing general text representation methods, a semantic feature vector generation model optimized based on mutual information between global and local matching is employed. This improves the text vectorization model's understanding of titles or keywords, thereby contributing to higher accuracy in semantic search.

[0114] use Figure 9 The semantic search method disclosed in the paper can apply a trained semantic feature vector generation model to vectorize text to optimize the representation effect of the generated semantic feature vector, or use a combination of global matching information and local matching information to determine the semantic similarity of text, thereby providing a matching basis for semantic search methods, which is especially helpful to improve the recall rate of fuzzy text matching.

[0115] Figure 10 A block diagram of an example of an apparatus 1000 for training a semantic feature vector generation model according to an embodiment of this specification is shown. This apparatus embodiment can be used with... Figures 2-5 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0116] like Figure 10 As shown, the apparatus 1000 for training a semantic feature vector generation model includes a sample acquisition unit 1010, a text extraction unit 1020, and a training unit 1030.

[0117] The sample acquisition unit 1010 is configured to acquire a training sample set. This training sample set may include the text to be queried and reference text. The operation of the sample acquisition unit 1010 can be referenced above. Figure 2 The operation described in section 210.

[0118] The text extraction unit 1020 is configured to extract text from the sample reference text, obtaining a first part of the text and a second part of the text corresponding to the sample reference text. The operation of the text extraction unit 1020 can be referenced above. Figure 2 The operation described in section 220.

[0119] In one example, the first part of the text may include the title of the sample reference text. The second part of the text may include the keywords of the sample reference text.

[0120] In one example, the semantic feature vector generation model trained by the method described above can be applied to service search. The first part of the text mentioned above may include the name of the service. The second part of the text mentioned above may include keywords related to the functions involved in the service.

[0121] For a detailed description of the above content, please refer to the reference above. Figure 2 The corresponding description in the optional implementation of 220.

[0122] Training unit 1030 is configured to provide training samples from the training sample set, along with the corresponding first and second part texts, to the current semantic feature vector generation model to obtain semantic feature vectors corresponding to the sample query text, the sample reference text, the first part text, and the second part text, respectively. Based on the similarity between the semantic feature vectors corresponding to the sample query text and the sample reference text, the global and local matching information between the sample query text and the sample reference text are determined. The loss value of the loss function is determined based on the global and local matching information. If the training termination condition is not met, the parameters of the current semantic feature vector generation model are adjusted according to the loss value, and the training steps continue. The operation of training unit 1030 can be referenced above. Figure 2 The operations described are 210 to 270.

[0123] In one example, the loss function may include a mutual information loss term to characterize the consistency between global matching information and the local matching information. The training unit 1030 described above can be further configured to: determine the global matching information between the sample query text and the sample reference text based on the similarity between the semantic feature vector corresponding to the sample reference text and the semantic feature vector corresponding to the sample query text; and determine the local matching information between the sample query text and the sample reference text based on the similarity between at least one of the semantic feature vectors corresponding to the first part of the text and the second part of the text and the semantic feature vector corresponding to the sample query text. The operation of the training unit 1030 can be referenced above. Figure 3 The process described in section 300 involves determining the global and local matching information between the sample query text and the sample reference text.

[0124] In one example, the training unit 1030 described above can be further configured to: determine the semantic similarity between the semantic feature vector corresponding to the sample query text and the semantic feature vectors corresponding to the first part of the text and the second part of the text, respectively; and determine the local matching information based on the larger semantic similarity between the determined semantic feature vector corresponding to the sample query text and the semantic feature vectors corresponding to the first part of the text and the second part of the text, respectively. The operation of the training unit 1030 can be referenced above. Figure 4 The process 400 describes the determination of local matching information between the sample query text and the sample reference text.

[0125] In one example, the mutual information loss term used to characterize the consistency between global matching information and local matching information may include a mutual information loss term used to characterize the difference between the distribution of global matching information and the distribution of local matching information.

[0126] In one example, the training sample set described above may also include matching information labels. The loss function may also include a supervised loss term to characterize the difference between the global matching information and the corresponding matching information labels.

[0127] In one example, the first part of the text above may include the title of the sample reference text. The second part of the text above may include the keywords of the sample reference text.

[0128] For a detailed description of the above content, please refer to the reference above. Figure 2 The corresponding descriptions in the optional implementations of 220 and 250 are described.

[0129] In one example, the training unit 1030 described above can be further configured to: determine the current semantic feature vector generation model as the completed semantic feature vector generation model when the training termination condition is met. The operation of the training unit 1030 can be referenced above. Figure 2 The corresponding description in description 280.

[0130] Figure 11 A block diagram of an example of a semantic similarity determination apparatus 1100 according to an embodiment of this specification is shown. This apparatus embodiment can be used with... Figure 6 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0131] like Figure 11 As shown, the semantic similarity determination device 1100 includes a vector generation unit 1110 and a similarity determination unit 1120.

[0132] Vector generation unit 1110 is configured to provide the query text and reference text to the semantic feature vector generation model, respectively, to obtain semantic feature vectors corresponding to the query text and the reference text. The semantic feature vector of the reference text incorporates information extracted from the reference text. The operation of vector generation unit 1110 can be referenced above. Figure 6 The operation of 610 is described.

[0133] The similarity determination unit 1120 is configured to determine the semantic similarity between the query text and the reference text based on the semantic feature vectors corresponding to the query text and the reference text, respectively. The operation of the similarity determination unit 1120 can be referenced above. Figure 6 The operation described in section 620.

[0134] In one example, the semantic feature vector generation model described above can be trained using the method described above for training semantic feature vector generation models. For a detailed description of the above example, please refer to the reference above. Figure 2-5 The corresponding description in the described method.

[0135] Figure 12 A block diagram of an example of a semantic similarity determination apparatus 1200 according to an embodiment of this specification is shown. This apparatus embodiment can be used with... Figures 7-8 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0136] like Figure 12 As shown, the semantic similarity determination device 1200 includes an extraction unit 1210, a vectorization unit 1220, a matching information determination unit 1230, and a semantic similarity determination unit 1240.

[0137] Extraction unit 1210 is configured to extract text from reference text to obtain a first part of text and a second part of text corresponding to the reference text. The operation of extraction unit 1210 can be referenced above. Figure 7 The operation of 710 is described.

[0138] Vectorization unit 1220 is configured to provide the query text, reference text, and the first and second parts of the text corresponding to the reference text to the semantic feature vector generation model, respectively, to obtain semantic feature vectors corresponding to the query text, reference text, and the first and second parts of the text corresponding to the reference text. The operation of vectorization unit 1220 can be referenced above. Figure 7 The operation described in section 720.

[0139] The matching information determination unit 1230 is configured to determine global matching information and local matching information between the query text and the reference text based on the similarity between the semantic feature vector corresponding to the query text and the reference text, and the semantic feature vectors corresponding to the first part of the reference text and the second part of the reference text. The operation of the matching information determination unit 1230 can be referred to the above. Figure 7 The operation of 730 is described.

[0140] The semantic similarity determination unit 1240 is configured to determine the semantic similarity between the query text and the reference text based on the determined global matching information and local matching information. The operation of the semantic similarity determination unit 1240 can be referenced above. Figure 7 The operation described in section 740.

[0141] In one example, the semantic similarity determination device described above can be applied to service search. The first part of the text may include the name of the service. The second part of the text may include keywords related to the functions involved in the service. A detailed description of the above content can be found in the reference above. Figure 7 The corresponding description in the optional implementation of 710.

[0142] Figure 13 A block diagram illustrating an example of a semantic search apparatus 1300 according to an embodiment of this specification is shown. This apparatus embodiment can be used with... Figure 9 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0143] like Figure 13 As shown, the semantic search device 1300 includes a receiving unit 1310, similarity determination units 1320, a semantic search unit 1330, and a result providing unit 1340.

[0144] The receiving unit 1310 is configured to receive query text provided by the user. The operation of the receiving unit 1310 can be referenced above. Figure 9 The operation of 910 is described.

[0145] Each similarity determination unit 1320 is configured to determine the semantic similarity between the query text and each reference text in the reference text set according to the semantic similarity determination method described above. The operation of each similarity determination unit 1320 can be referred to the above description. Figure 9 The operation described in section 920.

[0146] Semantic search unit 1330 is configured to determine semantic search results from the set of reference texts based on the semantic similarity between the query text and each reference text. The operation of semantic search unit 1330 can be referenced above. Figure 9 The operation described in section 930.

[0147] The result providing unit 1340 is configured to provide the semantic search results to the user. The operation of the result providing unit 1340 can be referenced above. Figure 9 The operation described in section 940.

[0148] In one example, the semantic search method described above can be applied to service search. The semantic search results may include service pages that match the query text. A more detailed description of service search can be found in the foregoing. Figure 2 The corresponding descriptions of the optional implementation methods in the embodiments are not repeated here.

[0149] Reference above Figures 1 to 13 Embodiments of the method, semantic similarity determination method, semantic search method, and apparatus for training a semantic feature vector generation model according to the embodiments of this specification have been described.

[0150] The apparatus, semantic similarity determination apparatus, and semantic search apparatus for training semantic feature vector generation models, as described in the embodiments of this specification, can be implemented in hardware, software, or a combination of hardware and software. Taking software implementation as an example, as a logically defined apparatus, it is formed by the processor of its host device reading the corresponding computer program instructions from the memory into memory and executing them. In the embodiments of this specification, the apparatus, semantic similarity determination apparatus, and semantic search apparatus for training semantic feature vector generation models can, for example, be implemented using electronic devices.

[0151] Figure 14 A schematic diagram of an apparatus 1400 for training a semantic feature vector generation model according to an embodiment of this specification is shown.

[0152] like Figure 14As shown, the apparatus 1400 for training a semantic feature vector generation model may include at least one processor 1410, a memory (e.g., non-volatile memory) 1420, a memory 1430, and a communication interface 1440, and the at least one processor 1410, memory 1420, memory 1430, and communication interface 1440 are connected together via a bus 1450. At least one processor 1410 executes at least one computer-readable instruction (i.e., the elements implemented in software above) stored or encoded in the memory.

[0153] In one embodiment, computer-executable instructions are stored in memory, which, when executed, cause at least one processor 1410 to: acquire a training sample set, wherein the training sample set includes sample query text and sample reference text; perform text extraction on the sample reference text to obtain a first part of text and a second part of text corresponding to the sample reference text; and perform the following training steps: provide the training samples in the training sample set and the corresponding first part of text and second part of text to a current semantic feature vector generation model to obtain semantic feature vectors corresponding to the sample query text, the sample reference text, the first part of text, and the second part of text, respectively; determine global matching information and local matching information between the sample query text and the sample reference text based on the similarity between the semantic feature vectors corresponding to the sample query text and the sample reference text, the first part of text, and the second part of text, respectively; determine the loss value of the loss function based on the global matching information and the local matching information; and adjust the parameters of the current semantic feature vector generation model based on the loss value and continue to execute the training steps when the training termination condition is not met.

[0154] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 1410 to perform the above-described combinations in the various embodiments of this specification. Figure 2-5 The description includes various operations and functions.

[0155] Figure 15 A schematic diagram of a semantic similarity determination apparatus 1500 according to an embodiment of this specification is shown.

[0156] like Figure 15 As shown, the semantic similarity determination apparatus 1500 may include at least one processor 1510, a memory (e.g., non-volatile memory) 1520, a RAM 1530, and a communication interface 1540, and the at least one processor 1510, memory 1520, RAM 1530, and communication interface 1540 are connected together via a bus 1550. At least one processor 1510 executes at least one computer-readable instruction (i.e., the elements implemented in software described above) stored or encoded in the memory.

[0157] In one embodiment, computer-executable instructions are stored in memory, which, when executed, cause at least one processor 1510 to: provide a query text and a reference text to a semantic feature vector generation model to obtain semantic feature vectors corresponding to the query text and the reference text, respectively, wherein the semantic feature vector of the reference text incorporates information from partial text extracted based on the reference text; and determine the semantic similarity between the query text and the reference text based on the semantic feature vectors corresponding to the query text and the reference text, respectively.

[0158] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 1510 to perform the above-described combinations in the various embodiments of this specification. Figure 6 The description includes various operations and functions.

[0159] Figure 16 A schematic diagram of a semantic similarity determination apparatus 1600 according to an embodiment of this specification is shown.

[0160] like Figure 16 As shown, the semantic similarity determination apparatus 1600 may include at least one processor 1610, a memory (e.g., non-volatile memory) 1620, a RAM 1630, and a communication interface 1640, and the at least one processor 1610, memory 1620, RAM 1630, and communication interface 1640 are connected together via a bus 1650. The at least one processor 1610 executes at least one computer-readable instruction (i.e., the elements implemented in software described above) stored or encoded in the memory.

[0161] In one embodiment, computer-executable instructions are stored in a memory, which, when executed, cause at least one processor 1610 to: extract text from a reference text to obtain a first part of text and a second part of text corresponding to the reference text; provide a query text, the reference text, the first part of text and the second part of text corresponding to the reference text to a semantic feature vector generation model to obtain semantic feature vectors corresponding to the query text, the reference text, the first part of text and the second part of text corresponding to the reference text, respectively; determine global matching information and local matching information between the query text and the reference text based on the similarity between the semantic feature vector corresponding to the query text and the semantic feature vectors corresponding to the first part of text and the second part of text corresponding to the reference text, respectively; and determine the semantic similarity between the query text and the reference text based on the determined global matching information and local matching information.

[0162] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 1610 to perform the above-described combinations in the various embodiments of this specification. Figures 7-8 The description includes various operations and functions.

[0163] Figure 17 A schematic diagram of a semantic search device 1700 according to an embodiment of this specification is shown.

[0164] like Figure 17 As shown, the semantic search device 1700 may include at least one processor 1710, a memory (e.g., non-volatile memory) 1720, a main memory 1730, and a communication interface 1740, and the at least one processor 1710, memory 1720, main memory 1730, and communication interface 1740 are connected together via a bus 1750. At least one processor 1710 executes at least one computer-readable instruction (i.e., the elements implemented in software described above) stored or encoded in the memory.

[0165] In one embodiment, computer-executable instructions are stored in memory that, when executed, cause at least one processor 1710 to: receive query text provided by a user; determine the semantic similarity between the query text and various reference texts in a reference text set according to the semantic similarity determination method described above; determine a semantic search result from the reference text set based on the semantic similarity between the query text and the various reference texts; and provide the semantic search result to the user.

[0166] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 1710 to perform the above-described combinations in the various embodiments of this specification. Figure 9 The description includes various operations and functions.

[0167] According to one embodiment, a program product, such as a computer-readable medium, is provided. The computer-readable medium may have instructions (i.e., the elements implemented in software as described above), which, when executed by a computer, cause the computer to perform the above-described combinations of the various embodiments of this specification. Figure 1-9 The description includes various operations and functions.

[0168] Specifically, a system or apparatus equipped with a readable storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer or processor of the system or apparatus can read and execute the instructions stored in the readable storage medium.

[0169] In this case, the program code itself, which can be read from a readable medium, can perform the functions of any of the above embodiments. Therefore, the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0170] The computer program code required for the operation of each part of this manual can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB, .NET, and Python; conventional procedural programming languages ​​such as C, Visual Basic 2003, Perl, COBOL 2002, PHP, and ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service, such as Software as a Service (SaaS).

[0171] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0172] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0173] Not all steps and units in the above process and system structure diagrams are mandatory; some steps or units can be omitted as needed. The execution order of each step is not fixed and can be determined as required. The device structure described in the above embodiments can be a physical structure or a logical structure. That is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0174] The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply that it is "preferred" or "advantageous" over other embodiments. Detailed descriptions are included for the purpose of providing an understanding of the described techniques. However, these techniques may be practiced without these detailed descriptions. In some instances, well-known structures and apparatuses are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0175] The optional embodiments of the present specification have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present specification are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present specification, various simple modifications can be made to the technical solutions of the embodiments of the present specification, and these simple modifications all fall within the protection scope of the embodiments of the present specification.

[0176] The foregoing description of this specification is provided to enable any person skilled in the art to implement or use the content of this specification. Various modifications to the content of this specification will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the scope of protection of this specification. Therefore, this specification is not limited to the examples and designs described herein, but is consistent with the widest scope of the principles and novel features disclosed herein.

Claims

1. A method for training a semantic feature vector generation model, comprising: Obtain a training sample set, wherein the training sample set includes sample query text and sample reference text; Text extraction is performed on the sample reference text to obtain a first part of the text and a second part of the text corresponding to the sample reference text; and Perform the following training steps: The training samples in the training sample set, along with the corresponding first and second parts of text, are provided to the current semantic feature vector generation model to obtain semantic feature vectors corresponding to the sample query text, the sample reference text, the first part of text, and the second part of text, respectively. Based on the similarity between the semantic feature vectors corresponding to the sample query text and the sample reference text, the first part of the text and the second part of the text, respectively, the global matching information and the local matching information between the sample query text and the sample reference text are determined. The local matching information is used to characterize the degree of matching between the whole of the sample query text and the part of the sample reference text. The loss value of the loss function is determined based on the global matching information and the local matching information; and If the training termination condition is not met, adjust the parameters of the current semantic feature vector generation model based on the loss value and continue the training steps.

2. The method as described in claim 1, wherein, The loss function includes a mutual information loss term to characterize the consistency between the global matching information and the local matching information. The process of determining the global and local matching information between the sample query text and the sample reference text based on the similarity between the semantic feature vectors corresponding to the sample reference text, the first part of the text, and the second part of the text includes: The global matching information between the sample query text and the sample reference text is determined based on the similarity between the semantic feature vector corresponding to the sample reference text and the semantic feature vector corresponding to the sample query text; and The local matching information between the sample query text and the sample reference text is determined based on the similarity between at least one of the semantic feature vectors corresponding to the first part of the text and the second part of the text and the semantic feature vector corresponding to the sample query text.

3. The method as described in claim 2, wherein, The step of determining the local matching information between the sample query text and the sample reference text based on the similarity between at least one of the semantic feature vectors corresponding to the first part of the text and the second part of the text and the semantic feature vector corresponding to the sample query text includes: Determine the semantic similarity between the semantic feature vector corresponding to the sample query text and the semantic feature vectors corresponding to the first part of the text and the second part of the text, respectively; and The local matching information is determined based on the greater semantic similarity between the semantic feature vector corresponding to the sample query text and the semantic feature vectors corresponding to the first part of the text and the second part of the text.

4. The method of claim 2, wherein, The mutual information loss term used to characterize the consistency between the global matching information and the local matching information includes a mutual information loss term used to characterize the difference between the distribution of the global matching information and the distribution of the local matching information.

5. The method of claim 4, wherein, The training sample set also includes matching information labels, and the loss function also includes a supervised loss term used to characterize the difference between the global matching information and the corresponding matching information labels.

6. The method as described in any one of claims 1 to 5, wherein, The first part of the text includes the title of the sample reference text, and the second part of the text includes the keywords of the sample reference text.

7. The method of claim 6, wherein, The semantic feature vector generation model is applied to service search. The first part of the text includes the name of the service, and the second part of the text includes keywords related to the functions involved in the service.

8. A method for determining semantic similarity, comprising: The query text and the reference text are respectively provided to the semantic feature vector generation model to obtain semantic feature vectors corresponding to the query text and the reference text, wherein the semantic feature vector of the reference text incorporates information from partial text extracted based on the reference text, and the semantic feature vector generation model is trained by the method described in any one of claims 1 to 7; and The semantic similarity between the query text and the reference text is determined based on the semantic feature vectors corresponding to the query text and the reference text, respectively.

9. A method for determining semantic similarity, comprising: Text extraction is performed on the reference text to obtain a first part of the text and a second part of the text corresponding to the reference text; The text to be queried, the reference text, the first part of the text and the second part of the text corresponding to the reference text are respectively provided to the semantic feature vector generation model to obtain the semantic feature vectors corresponding to the text to be queried, the reference text, the first part of the text and the second part of the text corresponding to the reference text respectively; Based on the similarity between the semantic feature vector corresponding to the query text and the reference text, and the semantic feature vectors corresponding to the first and second parts of the reference text, global matching information and local matching information between the query text and the reference text are determined. The local matching information characterizes the degree of matching between the overall query text and a part of the reference text. Based on the determined global and local matching information, the semantic similarity between the query text and the reference text is determined.

10. The semantic similarity determination method as described in claim 9, wherein, The semantic similarity determination method is applied to service search, where the first part of the text includes the name of the service, and the second part of the text includes keywords related to the functions involved in the service.

11. A semantic search method, comprising: Receive query text provided by the user; The semantic similarity between the query text and each reference text in the reference text set is determined according to the semantic similarity determination method as described in any one of claims 8 to 10. Based on the semantic similarity between the query text and each reference text, semantic search results are determined from the reference text set; and The semantic search results are provided to the user.

12. The semantic search method as described in claim 11, wherein, The semantic search method is applied to service search, and the semantic search results include service pages that match the query text.

13. An apparatus for training a semantic feature vector generation model, comprising: The sample acquisition unit is configured to acquire a training sample set, wherein the training sample set includes sample query text and sample reference text; The text extraction unit is configured to extract text from the sample reference text to obtain a first part of text and a second part of text corresponding to the sample reference text; and The training unit is configured to provide training samples from the training sample set, along with corresponding first and second part texts, to the current semantic feature vector generation model to obtain semantic feature vectors corresponding to the sample query text, the sample reference text, the first part text, and the second part text, respectively; determine global and local matching information between the sample query text and the sample reference text based on the similarity between the semantic feature vectors corresponding to the sample query text and the sample reference text, respectively, wherein the local matching information is used to characterize the degree of matching between the whole of the sample query text and the part of the sample reference text; determine the loss value of the loss function based on the global and local matching information; and adjust the parameters of the current semantic feature vector generation model and continue training steps based on the loss value when the training termination condition is not met.

14. The apparatus of claim 13, wherein, The loss function includes a mutual information loss term to characterize the consistency between the global matching information and the local matching information. The training unit is further configured as follows: The global matching information between the sample query text and the sample reference text is determined based on the similarity between the semantic feature vector corresponding to the sample reference text and the semantic feature vector corresponding to the sample query text. as well as The local matching information between the sample query text and the sample reference text is determined based on the similarity between at least one of the semantic feature vectors corresponding to the first part of the text and the second part of the text and the semantic feature vector corresponding to the sample query text.

15. The apparatus of claim 14, wherein, The training unit is further configured as follows: Determine the semantic similarity between the semantic feature vector corresponding to the sample query text and the semantic feature vectors corresponding to the first part of the text and the second part of the text, respectively; and The local matching information is determined based on the greater semantic similarity between the semantic feature vector corresponding to the sample query text and the semantic feature vectors corresponding to the first part of the text and the second part of the text.

16. The apparatus of claim 14, wherein, The mutual information loss term used to characterize the consistency between the global matching information and the local matching information includes a mutual information loss term used to characterize the difference between the distribution of the global matching information and the distribution of the local matching information.

17. The apparatus as claimed in any one of claims 13 to 16, wherein, The semantic feature vector generation model is applied to service search. The first part of the text includes the name of the service, and the second part of the text includes keywords related to the functions involved in the service.

18. A semantic similarity determination device, comprising: A vector generation unit is configured to provide a query text and a reference text to a semantic feature vector generation model, respectively, to obtain semantic feature vectors corresponding to the query text and the reference text, wherein the semantic feature vector of the reference text incorporates information from a portion of the text extracted from the reference text, and the semantic feature vector generation model is trained using the method described in any one of claims 1 to 7; and The similarity determination unit is configured to determine the semantic similarity between the query text and the reference text based on the semantic feature vectors corresponding to the query text and the reference text, respectively.

19. A semantic similarity determination device, comprising: The extraction unit is configured to extract text from the reference text to obtain a first part of text and a second part of text corresponding to the reference text. The vectorization unit is configured to provide the query text, the reference text, the first part of the text and the second part of the text corresponding to the reference text to the semantic feature vector generation model, respectively, to obtain semantic feature vectors corresponding to the query text, the reference text, the first part of the text and the second part of the text corresponding to the reference text, respectively. The matching information determination unit is configured to determine global matching information and local matching information between the query text and the reference text based on the similarity between the semantic feature vector corresponding to the query text and the reference text, and the semantic feature vector corresponding to the first part of the text and the second part of the text corresponding to the reference text, respectively. The local matching information is used to characterize the degree of matching between the whole of the query text and the part of the reference text. as well as The semantic similarity determination unit is configured to determine the semantic similarity between the query text and the reference text based on the determined global matching information and local matching information.

20. The semantic similarity determination apparatus as described in claim 19, wherein, The semantic similarity determination device is applied to service search, where the first part of the text includes the name of the service, and the second part of the text includes keywords related to the functions involved in the service.

21. A semantic search device, comprising: The receiving unit is configured to receive query text provided by the user. Each similarity determination unit is configured to determine the semantic similarity between the query text and each reference text in the reference text set according to the semantic similarity determination method as described in any one of claims 8 to 10. The semantic search unit is configured to determine semantic search results from the set of reference texts based on the semantic similarity between the query text and each reference text; as well as The result providing unit is configured to provide the semantic search results to the user.

22. The semantic search apparatus of claim 21, wherein, The semantic search device is used for service search, and the semantic search results include service pages that match the query text.

23. An apparatus for training a semantic feature vector generation model, comprising: At least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, wherein the at least one processor executes the computer program to implement the method as claimed in any one of claims 1 to 7.

24. A semantic similarity determination device, comprising: At least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, wherein the at least one processor executes the computer program to implement the semantic similarity determination method as described in claim 8.

25. A semantic similarity determination device, comprising: At least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, wherein the at least one processor executes the computer program to implement the semantic similarity determination method as described in claim 9 or 10.

26. A semantic search device, comprising: At least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, wherein the at least one processor executes the computer program to implement the semantic search method as described in claim 11 or 12.

27. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7, performs the semantic similarity determination method as described in claim 8, performs the semantic similarity determination method as described in claim 9 or 10, or performs the semantic search method as described in claim 11 or 12.

28. A computer program product comprising a computer program executed by a processor to implement the method of any one of claims 1 to 7, to implement the semantic similarity determination method of claim 8, to implement the semantic similarity determination method of claim 9 or 10, or to implement the semantic search method of claim 11 or 12.

Images