Semantic similarity determination method and related equipment

Abstract

The invention provides a semantic similarity determination device, which comprises a statement pair generation module used for respectively combining a target statement with at least one candidate statement into at least one statement pair; the at least one first semantic feature extraction network is respectively used for extracting semantic features of the input statement pair; the semantic feature combination module is used for combining the semantic features output by the at least one first semantic feature extraction network to obtain a semantic feature sequence; the second semantic feature extraction network is used for respectively determining semantic features with global information of the at least one statement pair based on the semantic feature sequence; and the full connection layer is used for respectively determining the similarity of the at least one statement pair based on the semantic features with the global information of the at least one statement pair. The invention further provides a semantic similarity determination method, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a semantic similarity determination method and related devices. Background Art

[0002] Semantic similarity calculation technology can calculate the semantic similarity score between two texts, so it can be applied to text retrieval services. As large-model retrieval enhancement generation technology has received more and more attention in recent years, semantic similarity calculation, as one of the processes, has gradually become an important research topic. The quality of the semantic similarity calculation algorithm can directly affect the quality of the text retrieved in the retrieval enhancement generation technology, and subsequently affect the effectiveness of the large model in answering user questions based on the relevant text. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure provide a semantic similarity determination device, a semantic similarity determination method and related equipment, which can predict the similarity between a sentence and multiple other sentences from a global perspective, so as to accurately determine the relative ranking position of each similarity in all sentence pairs.

[0004] The semantic similarity determination device described in the embodiment of the present disclosure may include:

[0005] A sentence pair generation module, used for combining the target sentence with at least one candidate sentence into at least one sentence pair;

[0006] At least one first semantic feature extraction network, used to extract semantic features of the input sentence pair;

[0007] A semantic feature combination module, used for combining the semantic features output by the at least one first semantic feature extraction network to obtain a semantic feature sequence;

[0008] a second semantic feature extraction network, configured to respectively determine semantic features with global information of the at least one sentence pair based on the semantic feature sequence; and

[0009] The fully connected layer is used to determine the similarity of the at least one sentence pair based on the semantic features of the at least one sentence pair with global information.

[0010] In an embodiment of the present disclosure, the first semantic feature extraction network may be composed of a multi-layer transformer encoder.

[0011] In an embodiment of the present disclosure, the second semantic feature extraction network may be composed of at least one layer of transformer encoder.

[0012] In an embodiment of the present disclosure, the fully connected layer may be composed of a multi-layer perceptron.

[0013] In an embodiment of the present disclosure, parameters of the at least one first semantic feature extraction network, the second extraction network, and the fully connected layer can be determined through a model training process using a circle loss function as a loss function.

[0014] Based on the above-mentioned semantic similarity determination device, an embodiment of the present disclosure also provides a semantic similarity determination method, which may include: combining the target sentence with at least one candidate sentence into at least one sentence pair; extracting the semantic features of the at least one sentence pair respectively; combining the semantic features of the at least one sentence pair to obtain a semantic feature sequence; determining the semantic features with global information of the at least one sentence pair based on the semantic feature sequence; and determining the similarity of the at least one sentence pair based on the semantic features with global information of the at least one sentence pair.

[0015] In an embodiment of the present disclosure, combining a target sentence with at least one candidate sentence to form at least one sentence pair may include: for each candidate sentence in the at least one candidate sentence, combining the target sentence and the candidate sentence to obtain a sentence combination; adding a classification marker before the sentence combination; and adding a segmentation marker between the target sentence and the candidate sentence to obtain a sentence pair consisting of the target sentence and the candidate sentence.

[0016] In an embodiment of the present disclosure, respectively extracting the semantic features of the at least one sentence pair may include: respectively inputting the at least one sentence pair into at least one first semantic feature extraction network composed of a multi-layer converter encoder; and obtaining the semantic features of the at least one sentence pair output by the at least one first semantic feature extraction network.

[0017] In an embodiment of the present disclosure, combining the semantic features of the at least one sentence pair may include: concatenating the semantic features of the at least one sentence pair in an input sample dimension to obtain the semantic feature sequence.

[0018] In an embodiment of the present disclosure, determining the semantic features with global information of the at least one sentence pair based on the semantic feature sequence may include: inputting the semantic feature sequence into a second semantic feature extraction network composed of at least one layer of converter encoder; and obtaining the semantic features with global information of the at least one sentence pair output by the second semantic feature extraction network.

[0019] In an embodiment of the present disclosure, determining the similarity of at least one sentence pair based on the semantic features with global information of the at least one sentence pair may include: inputting the semantic features with global information of the at least one sentence pair into a fully connected layer respectively; and using the output of the fully connected layer as the similarity of the at least one sentence pair respectively.

[0020] In an embodiment of the present disclosure, the above-mentioned semantic similarity determination method may further include: training the parameters of the at least one first semantic feature extraction network, the second semantic feature extraction network and the fully connected layer using a circle loss function as a loss function.

[0021] In addition, an embodiment of the present disclosure further provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned semantic similarity determination method when executing the program.

[0022] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the above-mentioned semantic similarity determination method.

[0023] An embodiment of the present disclosure further provides a computer program product, including computer program instructions, which, when executed on a computer, enable the computer to execute the above-mentioned semantic similarity determination method.

[0024] In the above-mentioned semantic similarity determination method and related equipment, in order to complete the prediction of the similarity between the target sentence and at least one candidate sentence, after the target sentence and each candidate sentence are respectively formed into sentence pairs and the semantic features of each sentence pair are respectively extracted, the semantic features of multiple sentence pairs are combined, and the semantic features with global information of the above-mentioned multiple sentence pairs are respectively determined based on the semantic feature sequences obtained by the combination. By combining the semantic features of multiple sentence pairs and mutually encoding the semantic features of multiple sentence pairs, each sentence pair can learn the contrast information between other sentence pairs, so that the output features with features between sentence pairs can be obtained, that is, the semantic features with global information of multiple sentence pairs are obtained. Finally, the similarity of each sentence pair is determined based on the semantic features with global information of the above-mentioned multiple sentence pairs. The similarity of each sentence pair determined by the above-mentioned method is predicted from a global perspective, and the relative ranking position of the similarity of each sentence pair in all sentence pairs can be accurately determined, which greatly improves the ranking effect of sentence similarity. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic diagram of the internal structure of a semantic similarity determination device described in some embodiments of the present disclosure is shown.

[0027] Figure 2 The implementation process of the semantic similarity determination method described in some embodiments of the present disclosure is shown.

[0028] Figure 3 A more specific schematic diagram of the hardware structure of an electronic device described in some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0030] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0031] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0032] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0033] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0034] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0035] As mentioned earlier, the quality of the semantic similarity calculation algorithm can directly affect the quality of the text retrieved in the retrieval enhancement generation technology, and subsequently affect the effectiveness of the large model in answering user questions based on relevant text. Therefore, it has become an important research topic.

[0036] Existing semantic similarity calculation algorithms usually use point-wise input, that is, only the two sentences "sentence 1-sentence 2" whose similarity needs to be calculated are input. In this case, the model can only use the existing two sentence information to encode and calculate the similarity of the two sentences. If there are more pairs of "sentence 1-sentence 3" and "sentence 1-sentence 4" that need to be compared, they are input into the above model in a similar way, and the similarity scores of each pair are obtained and then sorted according to the scores. However, in the actual use of semantic similarity calculation algorithms, most of the cases are based on the perspective of comparison, that is, given a sentence 1, the model needs to calculate the similarity with the other ten existing sentences and give the answer from a global perspective. In this case, the semantic similarity calculation needs to pay more attention to the relative position ranking of the similarity of the current sentence pair in the pair, that is, to look at the problem from a "global" perspective. However, since the existing model only obtains the information of the two sentences that need to be calculated for similarity, it is unable to look at the problem from a "global" perspective, which leads to poor similarity sorting effect of the model output.

[0037] In view of this, an embodiment of the present disclosure provides a semantic similarity determination device, which can predict the similarity between a sentence and multiple other sentences from a global perspective, so as to more accurately determine the relative ranking position of the similarity of each sentence pair among all sentence pairs, thereby achieving the goal of improving the ranking effect of sentence similarity.

[0038] Figure 1 The internal structure of the semantic similarity determination device described in the embodiment of the present disclosure is shown. Figure 1 As shown, the semantic similarity determination device 100 described in the embodiment of the present disclosure may include:

[0039] A sentence pair generating module 110, configured to combine a target sentence with at least one candidate sentence into at least one sentence pair;

[0040] At least one first semantic feature extraction network 120, wherein each first semantic feature extraction network 120 is used to extract semantic features of an input sentence pair;

[0041] A semantic feature combination module 130, configured to combine the semantic features output by the at least one first semantic feature extraction network to obtain a semantic feature sequence;

[0042] A second semantic feature extraction network 140 is used to determine the semantic features with global information of the at least one sentence pair based on the semantic feature sequence; and

[0043] The fully connected layer 150 is used to determine the similarity of the at least one sentence pair based on the semantic features of the at least one sentence pair with global information.

[0044] In the embodiment of the present disclosure, the semantic similarity determination device is mainly used to determine the similarity between the target sentence and each candidate sentence. Generally, there may be multiple candidate sentences.

[0045] The specific implementation of each module in the above-mentioned semantic similarity determination device will be described in detail below with reference to specific examples.

[0046] In an embodiment of the present disclosure, the sentence pair generation module 110 is specifically used to first combine the target sentence and the candidate sentence to obtain a sentence combination for each candidate sentence, then add a classification marker before the sentence combination, and add a segmentation marker between the target sentence and the candidate sentence, so as to obtain a sentence pair consisting of the target sentence and the candidate sentence. Specifically, the classification marker can be [CLS]; the segmentation marker can be [SEP]. In this way, the sentence pair can be in the form of "[CLS] target sentence [SEP] candidate sentence".

[0047] In the embodiments of the present disclosure, the number of the above-mentioned first semantic feature extraction networks 120 generally corresponds to the number of the above-mentioned sentence pairs. In some embodiments of the present disclosure, the above-mentioned first semantic feature extraction network 120 may be composed of a multi-layer transformer encoder (Transformer Encoder) for extracting the semantic features of the input sentence pairs. Specifically, the vector output at the [CLS] position in the output vector of the above-mentioned first semantic feature extraction network 120 may be used as the semantic feature of the above-mentioned sentence pair, which may also be referred to as the feature representation of the sentence pair. It should be noted that the embodiments of the present disclosure do not limit the specific number of layers of the multi-layer Transformer Encoder contained in the above-mentioned first semantic feature extraction network 120, and it can be flexibly determined according to actual needs.

[0048] In an embodiment of the present disclosure, the semantic feature combination module 130 may concatenate the semantic features of at least one sentence pair output by the at least one first semantic feature extraction network 120 in the input sample dimension, thereby obtaining the semantic feature sequence. For example, if the at least one first semantic feature extraction network 120 outputs the semantic features of N sentence pairs in total, the length of the semantic feature sequence may be considered to be N.

[0049] In some embodiments of the present disclosure, the second semantic feature extraction network 140 may be composed of at least one layer of Transformer Encoder, which is used to determine the semantic features with global information of the at least one sentence pair based on the semantic feature sequence. It should be noted that the embodiments of the present disclosure do not limit the specific number of layers of the at least one layer of Transformer Encoder included in the second semantic feature extraction network 140, and can be flexibly determined according to actual needs.

[0050] In an embodiment of the present disclosure, the fully connected layer may be composed of a multilayer perceptron (MLP), which is used to determine the similarity of the at least one sentence pair based on the semantic features with global information of the at least one sentence pair. In this way, for each sentence pair, the fully connected layer will output a similarity of the sentence pair.

[0051] In the above-mentioned semantic similarity determination device, in order to complete the prediction of the similarity between the target sentence and at least one candidate sentence, after the target sentence and each candidate sentence are respectively formed into sentence pairs and the semantic features of each sentence pair are respectively extracted, the semantic features of multiple sentence pairs are further combined, and the semantic features with global information of the above-mentioned multiple sentence pairs are respectively determined based on the semantic feature sequences obtained by the combination. Finally, the similarity of each sentence pair is respectively determined based on the semantic features with global information of the above-mentioned multiple sentence pairs. In the above-mentioned process, by combining the semantic features of multiple sentence pairs and mutually encoding the semantic features of multiple sentence pairs, each sentence pair can learn the contrast information between other sentence pairs, so that the output features with features between sentence pairs can be obtained, that is, the semantic features with global information of multiple sentence pairs are obtained. In this way, the similarity of each sentence pair determined by the above-mentioned method is based on the semantic features with global information of the above-mentioned multiple sentence pairs, that is, it is predicted from a global perspective, so that the relative ranking position of the similarity of each sentence pair in all sentence pairs can be accurately determined, which greatly improves the ranking effect of sentence similarity.

[0052] Furthermore, in an embodiment of the present disclosure, the parameters of the at least one first semantic feature extraction network, the second extraction network, and the fully connected layer can be determined by a model training process using a circle loss function as a loss function. The circle loss function can be as follows:

[0053]

[0054] in, is the predicted value of the negative sample, L is the number of negative samples, is the predicted value of the positive sample, K is the number of positive samples, Δ n =m,Δ p =1-m, O n =-m,O p =1+m, m and γ are hyperparameters. In practical applications, the hyperparameters m and γ can be pre-set according to actual needs.

[0055] Using the Circle Loss loss function to calculate the loss of similarity prediction can solve the data inefficiency problem caused by the need for a fixed number of candidate sentence samples in the cross entropy loss function. At the same time, the Circle Loss loss function's adaptive weighting function for samples can significantly improve the accuracy of semantic similarity prediction.

[0056] Based on the above-mentioned semantic similarity determination device, an embodiment of the present disclosure further provides a semantic similarity determination method.

[0057] Figure 2 The implementation process of the method for determining semantic similarity described in the embodiment of the present disclosure is shown. Figure 2 As shown, the method for determining semantic similarity described in the embodiment of the present disclosure may include the following steps.

[0058] In step 210, the target sentence is combined with at least one candidate sentence into at least one sentence pair.

[0059] In step 220, semantic features of the at least one sentence pair are extracted respectively.

[0060] In step 230, the semantic features of the at least one sentence pair are combined to obtain a semantic feature sequence.

[0061] In step 240, semantic features with global information of the at least one sentence pair are determined based on the semantic feature sequence.

[0062] In step 250, the similarity of the at least one sentence pair is determined based on the semantic features of the at least one sentence pair with global information.

[0063] The specific implementation of each step of the above semantic similarity determination method will be described in detail below with reference to specific examples.

[0064] In an embodiment of the present disclosure, in the above step 210, a specific method of combining the target sentence with at least one candidate sentence to form at least one sentence pair may include: performing the following operations for each candidate sentence in the at least one candidate sentence:

[0065] First, the target sentence and the candidate sentence are combined to obtain a sentence combination.

[0066] Then, add the classification marker before the statement combination.

[0067] Finally, a segmentation marker is added between the target sentence and the candidate sentence.

[0068] In this way, a sentence pair consisting of the target sentence and the candidate sentence can be obtained.

[0069] Specifically, the classification marker may be [CLS]. The segmentation marker may be [SEP]. Thus, the sentence pair may be in the form of "[CLS] target sentence [SEP] candidate sentence". It can be understood that the classification marker [CLS] is usually used to mark the beginning of input; the segmentation marker [SEP] is usually used to mark the end of a sentence and the beginning of the next sentence.

[0070] In an embodiment of the present disclosure, in the above step 220, the specific method of extracting the semantic features of the above at least one sentence pair may include: inputting the above at least one sentence pair into at least one first semantic feature extraction network composed of multiple layers of Transformer Encoder; and obtaining the semantic features of at least one sentence pair output by the at least one first semantic feature extraction network. In an embodiment of the present disclosure, encoding the input sentence pair through the first semantic feature extraction network composed of multiple layers of Transformer Encoder can obtain a vector representation of the sentence pair, which is also called the semantic feature of the sentence pair.

[0071] In an embodiment of the present disclosure, in the above step 230, the specific method of combining the semantic features of the above at least one sentence pair may include: splicing the semantic features of the above at least one sentence pair in the input sample dimension to obtain the above semantic feature sequence. It should be noted that the length of the above semantic feature sequence can be considered to be N, where N is the number of the above semantic features, that is, the number of sentence pairs in the above at least one sentence pair, and also the number of the above candidate sentences.

[0072] In an embodiment of the present disclosure, in the above step 240, the specific method of determining the semantic features with global information of the above at least one sentence pair based on the above semantic feature sequence may include: inputting the above semantic feature sequence into a second semantic feature extraction network composed of at least one layer of Transformer Encoder; and obtaining the semantic features with global information of the above at least one sentence pair output by the above second semantic feature extraction network. In an embodiment of the present disclosure, the semantic features of at least one sentence pair contained in the input semantic feature sequence are mutually encoded by the second semantic feature extraction network composed of at least one layer of Transformer Encoder, thereby obtaining a vector representation with features between sentence pairs, that is, obtaining the semantic features with global information of the above at least one sentence pair. The reason why the above vector representation is called a semantic feature with global information is that the vector representation obtained by encoding the above second semantic feature extraction network carries features between sentence pairs, and therefore carries the global information of the above at least one sentence pair.

[0073] In an embodiment of the present disclosure, in the above step 250, the specific method of determining the similarity of the at least one sentence pair based on the semantic features with global information of the at least one sentence pair may include: inputting the semantic features with global information of the at least one sentence pair into a fully connected layer respectively; and using the output of the fully connected layer as the similarity of the at least one sentence pair. In some embodiments of the present disclosure, the fully connected layer may be implemented by MLP.

[0074] As mentioned above, in an embodiment of the present disclosure, the above-mentioned semantic similarity determination method may further include: training the parameters of at least one first semantic feature extraction network, a second semantic feature extraction network and the above-mentioned fully connected layer in the above-mentioned semantic similarity determination device. In the specific training process, the Circle Loss loss function can be used as the loss function. The specific expression of the Circle Loss loss function can be referred to the above-mentioned embodiment, and will not be repeated here.

[0075] In the above-mentioned method for determining semantic similarity, after the target sentence and each candidate sentence are respectively formed into sentence pairs and the semantic features of each sentence pair are respectively extracted, the semantic features of multiple sentence pairs are first combined. Then, the semantic features with global information of the above-mentioned multiple sentence pairs are respectively determined based on the semantic feature sequences obtained by the combination. Finally, the similarity of each sentence pair is respectively determined based on the semantic features with global information of the above-mentioned multiple sentence pairs. In the above-mentioned process, by combining the semantic features of multiple sentence pairs and mutually encoding the semantic features of multiple sentence pairs, each sentence pair can learn the contrast information between other sentence pairs, so that the output features with features between sentence pairs can be obtained, that is, the semantic features with global information of multiple sentence pairs are obtained. In this way, the similarity of each sentence pair determined by the above-mentioned method is based on the semantic features with global information of the above-mentioned multiple sentence pairs, that is, it is predicted from a global perspective, so that the relative ranking position of the similarity of each sentence pair in all sentence pairs can be accurately determined, which greatly improves the ranking effect of sentence similarity.

[0076] Furthermore, when training the model, using the Circle Loss loss function to calculate the loss of similarity prediction can solve the data inefficiency problem caused by the need for a fixed number of candidate sentence samples in the cross entropy loss function. At the same time, the Circle Loss loss function's adaptive weighting function for samples can significantly improve the accuracy of semantic similarity prediction.

[0077] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for determining the semantic similarity described in any of the above embodiments is implemented.

[0078] Figure 3 A more specific hardware structure diagram of an electronic device provided in this embodiment is shown, and the device may include: a processor 2010, a memory 2020, an input / output interface 2030, a communication interface 2040, and a bus 2050. The processor 2010, the memory 2020, the input / output interface 2030, and the communication interface 2040 are connected to each other in communication within the device through the bus 2050.

[0079] The processor 2010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0080] The memory 2020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 2020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 2020 and are called and executed by the processor 2010.

[0081] The input / output interface 2030 is used to connect input / output devices to realize information input and output. The input / output devices can be configured in the device as components, or can be externally connected to the device to provide corresponding functions. The input devices can include microphones, various sensors, etc., and the output devices can include displays, speakers, vibrators, indicator lights, etc.

[0082] The communication interface 2040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0083] The bus 2050 includes a path that transmits information between the various components of the device (eg, the processor 2010, the memory 2020, the input / output interface 2030, and the communication interface 2040).

[0084] It should be noted that, although the above device only shows the processor 2010, the memory 2020, the input / output interface 2030, the communication interface 2040, and the bus 2050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0085] The electronic device of the above embodiment is used to implement the corresponding semantic similarity determination method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0086] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the semantic similarity determination method as described in any of the above embodiments.

[0087] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0088] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the task processing method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0089] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0090] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0091] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0092] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A semantic similarity determination device, comprising: A sentence pair generation module, used for combining the target sentence with at least one candidate sentence into at least one sentence pair; At least one first semantic feature extraction network, used to extract semantic features of the input sentence pair; A semantic feature combination module, used for combining the semantic features output by the at least one first semantic feature extraction network to obtain a semantic feature sequence; A second semantic feature extraction network, used for respectively determining the semantic features with global information of the at least one sentence pair based on the semantic feature sequence; as well as The fully connected layer is used to determine the similarity of the at least one sentence pair based on the semantic features of the at least one sentence pair with global information.

2. The semantic similarity determination device according to claim 1, wherein: The first semantic feature extraction network is composed of a multi-layer transformer encoder.

3. The semantic similarity determination device according to claim 1, wherein: The second semantic feature extraction network is composed of at least one layer of transformer encoder.

4. The semantic similarity determination device according to claim 1, wherein: The fully connected layer is composed of a multi-layer perceptron.

5. The semantic similarity determination device according to claim 1, wherein: The parameters of the at least one first semantic feature extraction network, the second extraction network and the fully connected layer are determined through a model training process using a circle loss function as a loss function.

6. A method for determining semantic similarity, comprising: Combining the target sentence with at least one candidate sentence into at least one sentence pair; respectively extracting semantic features of the at least one sentence pair; Combining the semantic features of the at least one sentence pair to obtain a semantic feature sequence; Determine the semantic features with global information of the at least one sentence pair based on the semantic feature sequence; as well as The similarity of the at least one sentence pair is determined based on the semantic features of the at least one sentence pair with global information.

7. The method for determining semantic similarity according to claim 6, wherein: Combining the target sentence with at least one candidate sentence into at least one sentence pair comprises: For each candidate sentence in the at least one candidate sentence, Combining the target sentence and the candidate sentence to obtain a sentence combination; adding a classification marker before the sentence combination; and A segmentation marker is added between the target sentence and the candidate sentence to obtain a sentence pair consisting of the target sentence and the candidate sentence.

8. The method for determining semantic similarity according to claim 6, wherein: Extracting the semantic features of the at least one sentence pair respectively includes: Inputting the at least one sentence pair into at least one first semantic feature extraction network composed of a multi-layer transformer encoder respectively; and Acquire semantic features of the at least one sentence pair output by at least one first semantic feature extraction network.

9. The method for determining semantic similarity according to claim 6, wherein: Combining the semantic features of the at least one sentence pair includes: splicing the semantic features of the at least one sentence pair in the input sample dimension to obtain the semantic feature sequence.

10. The method for determining semantic similarity according to claim 8, wherein: Determining the semantic features with global information of the at least one sentence pair based on the semantic feature sequence includes: Inputting the semantic feature sequence into a second semantic feature extraction network consisting of at least one layer of transformer encoder; and Acquire a semantic feature with global information of the at least one sentence pair output by the second semantic feature extraction network.

11. The method for determining semantic similarity according to claim 10, wherein: Determining the similarity of the at least one sentence pair based on the semantic features with global information of the at least one sentence pair comprises: Inputting the semantic features with global information of the at least one sentence pair into the fully connected layer respectively; and The outputs of the fully connected layer are respectively used as the similarity of the at least one sentence pair.

12. The method for determining semantic similarity according to claim 11, further comprising: The parameters of the at least one first semantic feature extraction network, the second semantic feature extraction network, and the fully connected layer are trained using a circle loss function as a loss function.

13. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for determining semantic similarity according to any one of claims 6 to 12 is implemented.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the semantic similarity determination method according to any one of claims 6 to 12.

15. A computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the semantic similarity determination method according to any one of claims 6 to 12.

Images