A semantic matching method, device, equipment and readable storage medium

Abstract

The application provides a semantic matching method and device, equipment and readable storage medium, relates to the field of artificial intelligence, especially to the field of natural language processing. The method comprises: obtaining a first text and a second text to be matched; inputting the first text and the second text into a semantic similarity prediction network to determine the similarity of the first text and the second text; extracting a first feature vector of the first text and the second text; inputting the first feature vector into a similarity dynamic threshold prediction network to determine the similarity dynamic threshold corresponding to the first text and the second text; and determining the matching result of the first text and the second text according to the similarity and the similarity dynamic threshold. The application increases the dynamic threshold prediction network branch, uses the similarity dynamic threshold output by the dynamic threshold prediction network to correct the similarity output by the similarity prediction network, and improves the accuracy of semantic matching.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a semantic matching method, apparatus, device, and readable storage medium. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. Research in the field of AI includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and fundamental AI theories.

[0003] Semantic matching is one of the most fundamental and widely used technologies in the field of Natural Language Processing (NLP). The essence of semantic matching is semantic understanding, which plays a crucial role in information retrieval, text clustering, machine translation, and intelligent question answering. However, existing semantic matching schemes generally suffer from low matching accuracy.

[0004] Application content

[0005] Embodiments of this application provide a semantic matching method, apparatus, device, and readable storage medium, which solve the problem of poor matching accuracy in existing semantic matching methods.

[0006] In a first aspect, this application provides a semantic matching method, comprising: acquiring a first text and a second text to be matched; inputting the first text and the second text into a semantic similarity prediction network to determine the similarity between the first text and the second text; extracting a first feature vector from the first text and the second text; inputting the first feature vector into a similarity dynamic threshold prediction network to determine a similarity dynamic threshold corresponding to the first text and the second text; and determining a matching result of the first text and the second text based on the similarity and the similarity dynamic threshold.

[0007] This application embodiment improves the accuracy of semantic matching by adding a dynamic threshold prediction network branch and using the similarity output of the dynamic threshold prediction network to correct the similarity output of the similarity prediction network.

[0008] In one possible implementation, the aforementioned first feature vector at least represents the length features of the first and second texts. This solves the problem in existing semantic matching methods that do not consider the sentence length features of the problem to be matched, resulting in poor matching accuracy for short texts.

[0009] In one possible implementation, the length feature includes at least: the absolute value L1 of the length difference between the first text and the second text, L2 of the length of the first text and the sum of the lengths of the second text, and L of the sum of the lengths of the common subsequences of the first text and the second text. common and the L common One or more of the ratios of L2 and L2, wherein the length of the common subsequence is determined based on the number of characters in the same string, the same string being a string in which N consecutive characters are the same in the string of the first text and the string of the second text, where N is a positive integer greater than 1.

[0010] This application embodiment extracts the absolute value L1 of the length difference of the problems to be identified, L2 of the sum of the lengths of the problems to be identified, and L of the sum of the common subsequences of the problems to be identified. common and L common The ratio of L2 to the length-related features of the matched text is used as input to the similarity dynamic threshold prediction network to determine the dynamic similarity threshold. The dynamic similarity threshold is then used to correct the similarity output of the existing semantic similarity prediction network to obtain more accurate matching results, thereby improving the accuracy of semantic matching results for shorter texts.

[0011] In another possible implementation, the first feature vector, in addition to representing the length features of the first and second texts, also represents the word semantic features of the first and second texts.

[0012] This application embodiment increases the accuracy of semantic matching by adding the semantic features of the text to be matched as a factor in determining the dynamic similarity threshold.

[0013] In another possible implementation, determining the matching result of the first text and the second text based on the similarity and the dynamic similarity threshold includes: if the similarity is greater than or equal to the dynamic similarity threshold, then the first text and the second text are determined to match; otherwise, the first text and the second text are determined not to match.

[0014] In another possible implementation, the semantic similarity prediction network includes a pre-trained feature extractor and a similarity classifier;

[0015] Inputting the first text and the second text into a semantic similarity prediction network to determine the similarity between the first text and the second text includes: the pre-trained feature extractor extracting second feature vectors for the first text and the second text respectively, wherein the second feature vectors represent the semantic features of the first text and the second text; and inputting the second feature vectors into the similarity classifier to determine the similarity between the first text and the second text.

[0016] In one example, the pre-trained feature extractor is a BERT model.

[0017] In another possible implementation, the semantic similarity prediction network is trained based on training text pairs and the labels corresponding to the training text pairs; the similarity dynamic threshold prediction network is trained based on a first feature vector of the training text pairs and the label corresponding to the first feature vector; wherein, the label corresponding to the first feature vector is determined based on the output of the semantic similarity prediction network for the training text pairs and the label corresponding to the training text pairs.

[0018] In another possible implementation, the similarity dynamic threshold prediction network is a feedforward neural network.

[0019] Secondly, this application also provides a semantic matching apparatus, comprising: an acquisition module for acquiring a first text and a second text to be matched; a first determination module for inputting the first text and the second text into a semantic similarity prediction network to determine the similarity between the first text and the second text; an extraction module for extracting a first feature vector from the first text and the second text; a second determination module for inputting the first feature vector into a similarity dynamic threshold prediction network to determine a similarity dynamic threshold corresponding to the first text and the second text; and a matching module for determining a matching result between the first text and the second text based on the similarity and the similarity dynamic threshold.

[0020] In one possible implementation, the first feature vector represents at least the length features of the first and second texts.

[0021] In another possible implementation, the length feature includes at least: the absolute value of the length difference between the first text and the second text L1, the sum of the lengths of the first text and the second text L2, and the sum of the lengths of the common subsequences of the first text and the second text L... common , and the L common One or more of the ratios of L2 and L2, wherein the length of the common subsequence is determined based on the number of characters in the same string, the same string being a string in which N consecutive characters are the same in the string of the first text and the string of the second text, where N is a positive integer greater than 1.

[0022] In another possible implementation, the first feature vector at least represents the length features of the first and second texts, as well as the word semantic features of the first and second texts.

[0023] In another possible implementation, the matching module is further configured to: determine that the first text and the second text match if the similarity is greater than or equal to the dynamic similarity threshold, and otherwise determine that the first text and the second text do not match.

[0024] In another possible implementation, the semantic similarity prediction network includes a pre-trained feature extractor and a similarity classifier;

[0025] The first determining module is further configured to:

[0026] The pre-trained feature extractor extracts second feature vectors from the first text and the second text respectively, wherein the second feature vectors represent the semantic features of the first text and the second text;

[0027] The second feature vector is input into the similarity classifier to determine the similarity between the first text and the second text.

[0028] In another possible implementation, the pre-trained feature extractor is a BERT model.

[0029] In another possible implementation, the semantic similarity prediction network is trained based on training text pairs and the labels corresponding to the training text pairs;

[0030] The similarity dynamic threshold prediction network is trained based on the first feature vector of the training text pair and the label corresponding to the first feature vector;

[0031] The label corresponding to the first feature vector is determined based on the output of the semantic similarity prediction network for the training text pair and the label corresponding to the training text pair.

[0032] In another possible implementation, the similarity dynamic threshold prediction network is a feedforward neural network.

[0033] Thirdly, this application also provides an electronic device, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method described in the first aspect or any possible implementation thereof.

[0034] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in the first aspect or any possible implementation thereof.

[0035] Fifthly, this application also provides a computer program or computer program product, the computer program or computer program product including instructions that, when executed, implement the method described in the first aspect or any possible implementation of the first aspect.

[0036] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0037] Figure 1 A schematic diagram of an artificial intelligence main framework provided in an embodiment of this application;

[0038] Figure 2 A system architecture diagram of the semantic matching system provided in the embodiments of this application;

[0039] Figure 3 This is a schematic diagram illustrating semantic matching based on a CNN model, provided as an embodiment of this application.

[0040] Figure 4 This is a schematic diagram illustrating another semantic matching based on a CNN model, provided as an embodiment of this application.

[0041] Figure 5 A schematic diagram of the hardware structure of a chip provided in an embodiment of this application;

[0042] Figure 6 A flowchart illustrating a semantic matching method provided in an embodiment of this application;

[0043] Figure 7 This is a schematic diagram of the structure of a semantic matching model provided in an embodiment of this application;

[0044] Figure 8 This is a diagram illustrating the training of the BERT model.

[0045] Figure 9 This is a schematic diagram of a semantic similarity prediction network.

[0046] Figure 10 This application provides a schematic diagram of the structure of a semantic matching device;

[0047] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0048] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0049] Figure 1 A schematic diagram of an artificial intelligence framework is shown, which describes the overall workflow of an artificial intelligence system and is applicable to general artificial intelligence domain needs.

[0050] The above-mentioned artificial intelligence framework will be elaborated from two dimensions: "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0051] The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it could be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom."

[0052] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence, information (provided and processed by technology) to the industrial ecosystem of systems.

[0053] (1) Infrastructure:

[0054] The infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by intelligent chips, including but not limited to hardware acceleration chips such as central processing units (CPUs), neural-network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs). The basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0055] (2) Data

[0056] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0057] (3) Data processing

[0058] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0059] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0060] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0061] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0062] (4) General ability

[0063] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0064] (5) Smart products and industry applications

[0065] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They encapsulate overall artificial intelligence solutions, productize intelligent information decision-making, and realize practical applications. Their application areas mainly include: intelligent manufacturing, intelligent transportation, smart home, intelligent healthcare, intelligent security, autonomous driving, safe city, and intelligent terminals.

[0066] The semantic matching method, apparatus, device, and readable storage medium provided in this application are mainly applied in the field of natural language processing, such as semantic matching. Semantic matching can be applied to scenarios such as information retrieval, text clustering, machine translation, and intelligent question answering.

[0067] Taking the intelligent question answering scenario as an example, the intelligent question answering system receives the question input by the user (the user can input the question in the form of voice, text or image, which is not limited in this application), performs a semantic matching task, matches the standard question with the highest similarity to the user's input question in a pre-configured standard question answering library, or matches the standard question with the required similarity to the user's input question, and then returns the answer corresponding to the standard question as feedback to the user's question.

[0068] Semantic matching schemes can be broadly categorized into two types. One approach involves manually extracting semantic information from sentences by methods such as keyword extraction or sentence structure analysis, and then comparing the corresponding features of sentence pairs to determine similarity. The problem with this approach is the difficulty in manually constructing features, and the challenge in achieving comprehensive coverage, often resulting in high precision but low recall. The other approach utilizes deep learning-based sentence similarity models. These models input the sentence pairs to be matched into a trained model, calculate the similarity scores, and then compare these scores to a preset threshold. If the similarity score is greater than or equal to the threshold, the sentence pairs are considered a match. However, existing deep learning-based similarity models determine match results based solely on the similarity score of the sentence pairs and a fixed preset threshold. In other words, regardless of the sentence type (i.e., any length, sentence structure, language, etc.), the match determination is based entirely on the similarity score of the sentence pairs and a fixed preset threshold (e.g., a threshold of 0.5). The similarity requirements for matching different types of sentence pairs may be different. For example, for shorter sentences, since they have fewer words and each word is closely related to the context, the similarity requirement will be higher. If a similarity of 0.5 or higher is used to determine whether a match is made, the matching results will inevitably be inaccurate.

[0069] This application provides a semantic matching method that adds a dynamic threshold prediction network branch to a similarity prediction network. Feature vectors of sentence pairs to be matched are extracted, and these feature vectors are input into the dynamic threshold prediction network branch to obtain a dynamically adjusted similarity threshold. The matching result is then determined based on the similarity of the sentence pairs obtained from the similarity prediction network and the dynamic similarity threshold. This method dynamically adjusts the similarity threshold according to the features of the sentence pairs, improving the accuracy of semantic matching.

[0070] To facilitate understanding of this solution, the embodiments of this application first combine... Figure 2 The semantic matching system provided in the embodiments of this application will be described below, see appendix. Figure 2 The data acquisition device 260 is used to collect training data, which includes labeled training text pairs.

[0071] After collecting the training data, the data acquisition device 260 stores the training data in the database 230. The training device 220 generates the target model / rule 201 based on the training data maintained in the database 230. The following will describe in more detail how the training device 220 obtains the target model / rule 201 based on the training data. The target model / rule 201 can realize the semantic matching method provided in the embodiments of this application, that is, identify whether the text pairs to be matched match.

[0072] In practical applications, the training data maintained in database 230 may not all come from the data acquisition device 260; it may also be received from other devices. Furthermore, it should be noted that training device 220 may not necessarily train the target model / rule 201 entirely based on the training data maintained in database 230; it may also obtain training data from the cloud or other sources for model training. The above description should not be construed as limiting the embodiments of this application.

[0073] The target model / rule 201 trained using training device 220 can be applied to different systems or devices, such as... Figure 2 In the execution device 210, the execution device 210 can be a terminal, such as a mobile phone, tablet computer, laptop computer, augmented reality (AR), virtual reality (VR), wearable device, smart robot, vehicle terminal, etc., or it can be a server or cloud, etc.

[0074] exist Figure 2 In this embodiment, the execution device 210 configures the input / output (I / O) interface 212 for data exchange with external devices. Users can input data into the I / O interface 212 through the client device 240. This data may include text entered by the user in this embodiment. It should be explained that this text can be input in any way; for example, the user can input voice data, and the corresponding text can be obtained through voice recognition; or the user can input image data, and the corresponding text can be obtained through image recognition; or the user can directly type the text data.

[0075] Preprocessing module 213 and preprocessing module 214 are used to preprocess the input data (such as text input by the user) received by I / O interface 212. For example, in this embodiment, preprocessing module 213 merges and connects the first text and the second text, and preprocessing module 214 extracts the first feature vector of the first text and the second text to be matched.

[0076] The calculation module 211 is used to perform calculations and other related processing on the data input from the preprocessing module 213 and the preprocessing module 214 according to the target model / rule 201 mentioned above.

[0077] During the preprocessing of input data by the execution device 210, or during the calculation and related processing of the calculation module 211 of the execution device 210, the execution device 210 can call the data, code, etc. of the database storage system 250 for corresponding processing, or store the data, instructions, etc. obtained from the corresponding processing into the data storage system 250.

[0078] Finally, I / O interface 212 returns the processing results (such as retrieval results, translation results, question answers, inference results, etc.) to client device 240 for the user. It should be understood that the target model / rule 201 differs for different natural language processing tasks, and its processing results also differ accordingly.

[0079] It is worth noting that the training device 220 can generate a target model / rule 201 corresponding to different downstream systems. This target model / rule 201 can achieve the above-mentioned goals or complete the above-mentioned tasks, thereby providing the user with the required results. It should also be noted that the training device 220 can generate corresponding preprocessing models for the target model / rule 201 corresponding to different downstream systems, such as the preprocessing models corresponding to preprocessing modules 213 and / or 214.

[0080] exist Figure 2 In the scenario shown, the user can manually specify data to be input into execution device 210 (e.g., input a piece of text), for example, by operating on the interface provided by I / O interface 212. Alternatively, client device 240 can automatically input data into I / O interface 212 (e.g., input a piece of text) and obtain results. If automatic data input by client device 240 requires user authorization, the user can set appropriate permissions in client device 240. The user can view the results output by execution device 210 on client device 240 (e.g., the output results could be translated text, answers to questions, search results, etc.), and the specific presentation format can be display, sound, animation, etc. Client device 240 can also act as a data acquisition terminal, collecting data such as... Figure 2 The input data (the first and second texts to be matched) from the input I / O interface 212 and the matching results output by the target model / rule 201 are used as new training sample data and stored in the database 230.

[0081] It is worth noting that Figure 2This is merely a schematic diagram of a system architecture provided in an embodiment of this application, and the positional relationships between the devices, components, modules, etc., shown in the diagram do not constitute any limitation. For example, in Figure 2 In this context, the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 may also be placed within the execution device 210.

[0082] like Figure 2 As shown, the target model / rule 201 trained by the training device 220 can be the semantic similarity prediction network and / or the dynamic similarity threshold prediction network in this application embodiment. Specifically, the semantic similarity prediction network and / or the dynamic similarity threshold prediction network provided in this application embodiment are neural networks, such as convolutional neural networks (CNN), deep convolutional neural networks (DCNN), etc.

[0083] Since CNN is a very common type of neural network, the following will combine... Figure 3 This section focuses on a detailed explanation of the structure of CNNs. A CNN is a deep neural network with convolutional structures, and it's a deep learning architecture. Deep learning architectures refer to learning at multiple levels of abstraction using machine learning algorithms. As a deep learning architecture, a CNN is a feed-forward artificial neural network where each neuron responds to overlapping regions in the input image.

[0084] like Figure 3 As shown, the Convolutional Neural Network (CNN) 100 may include an input layer 110, convolutional / pooling layers 120 (pooling layers are optional), and neural network layers 130. The details of these layers are described below.

[0085] Convolutional / pooling layers 120:

[0086] Convolutional layers:

[0087] like Figure 3The convolutional / pooling layer 120 shown may include layers 121-126 as in Examples 121-126. In one implementation, layer 121 is a convolutional layer, layer 122 is a pooling layer, layer 123 is a convolutional layer, layer 124 is a pooling layer, layer 125 is a convolutional layer, and layer 126 is a pooling layer. In another implementation, layers 121 and 122 are convolutional layers, layer 123 is a pooling layer, layers 124 and 125 are convolutional layers, and layer 126 is a pooling layer. That is, the output of the convolutional layer can be used as the input of a subsequent pooling layer, or as the input of another convolutional layer to continue the convolution operation.

[0088] The following section uses convolutional layer 121 as an example to introduce the internal working principle of a convolutional layer.

[0089] Convolutional layer 121 can include many convolution operators, also known as kernels. In natural language processing, a convolution operator acts as a filter that extracts specific information from the input speech or semantic information (such as text). A convolution operator can essentially be a weight matrix, which is usually predefined.

[0090] The weight values ​​in these weight matrices need to be obtained through extensive training in practical applications. The weight matrices formed by the weight values ​​obtained through training can extract information from the input data, thereby helping the convolutional neural network 100 to make correct predictions.

[0091] When a convolutional neural network 100 has multiple convolutional layers, the initial convolutional layers (e.g., 121) tend to extract more general features, which can also be called low-level features. As the depth of the convolutional neural network 100 increases, the features extracted by later convolutional layers (e.g., 126) become more and more complex, such as high-level semantic features. Features with higher semantic levels are more suitable for the problem to be solved.

[0092] Pooling layer:

[0093] Because it is often necessary to reduce the number of training parameters, pooling layers are often introduced periodically after convolutional layers, i.e., ... Figure 3 In example 120, layers 121-126 can be a convolutional layer followed by a pooling layer, or multiple convolutional layers followed by one or more pooling layers. In natural language data processing, the sole purpose of pooling layers is to reduce the size of the data.

[0094] Neural network layer 130:

[0095] After processing by the convolutional / pooling layers 120, the convolutional neural network 100 is still insufficient to output the required information. As mentioned earlier, the convolutional / pooling layers 120 only extract features and reduce the parameters introduced by the input data. However, to generate the final output information (the required class information or other relevant information), the convolutional neural network 100 needs to utilize neural network layers 130 to generate one or more outputs representing the required number of classes. Therefore, neural network layers 130 may include multiple hidden layers (such as...). Figure 3 As shown in 131, 132 to 13n) and output layer 140, the parameters contained in these multi-layer hidden layers can be pre-trained based on relevant training data for specific task types, such as speech or semantic recognition, classification or generation, etc.

[0096] After the multiple hidden layers in neural network layer 130, the final layer of the entire convolutional neural network 100 is the output layer 140. This output layer 140 has a loss function similar to classification cross-entropy, specifically used to calculate the prediction error. Once the entire convolutional neural network 100 has undergone forward propagation (such as...), the loss function is applied. Figure 3 The propagation from 110 to 140 is completed (forward propagation), and the reverse propagation (such as...) Figure 3 The propagation from 140 to 110 (backpropagation) will begin to update the weight values ​​and biases of the layers mentioned above, in order to reduce the loss of the convolutional neural network 100 and the error between the output of the convolutional neural network 100 through the output layer and the ideal result.

[0097] It should be noted that, as Figure 3 The convolutional neural network 100 shown is merely an example of a convolutional neural network. In specific applications, convolutional neural networks can also exist in the form of other network models, such as... Figure 4 The multiple convolutional / pooling layers shown are run in parallel, and the extracted features are all input into the full neural network layer 130 for processing.

[0098] Figure 5 This is a schematic diagram of the hardware structure of a chip provided in an embodiment of this application. The chip includes a neural network processor (NPU) 50. This chip can be configured as follows: Figure 2 In the execution device 210 shown, the chip is used to perform the calculations of the calculation module 211. This chip can also be located in, for example... Figure 2 The training device 220 shown is used to complete the training work of the training device 220 and output the target model / rule 201. For example... Figure 3 and Figure 4 The algorithms for each layer in the convolutional neural network shown can all be implemented in, for example... Figure 5 This is achieved in the chip shown.

[0099] The semantic matching method of this application embodiment can be executed in the arithmetic circuit 503 and / or vector calculation unit 507 in the neural network processor 50, thereby obtaining the matching result of the text to be matched.

[0100] The following is a brief introduction to the various modules and units in the neural network processor 50.

[0101] The neural network processor 50 is mounted as a coprocessor on the host CPU, and tasks are assigned by the host CPU. The core of the neural network processor 50 is the arithmetic circuit 503. When the neural network processor 50 is working, the controller 504 in the neural network processor 50 can control the arithmetic circuit 503 to retrieve matrix data from the memory and perform multiplication operations.

[0102] In some implementations, the arithmetic circuit 503 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 503 is a two-dimensional pulsating array. The arithmetic circuit 503 can also be a one-dimensional pulsating array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.

[0103] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 502 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 501 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 508.

[0104] The vector computation unit 507 can further process the output of the computation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. For example, the vector computation unit 507 can be used for network computation in non-convolutional / non-fully connected layers (FC) of neural networks, such as pooling, batch normalization, local response normalization, etc.

[0105] In some implementations, vector computation unit 507 can store the processed output vector into unified buffer 506. For example, vector computation unit 507 can apply a nonlinear function to the output of arithmetic circuit 503, such as a vector of accumulated values, to generate activation values. In some implementations, vector computation unit 507 generates normalized values, merged values, or both. In some implementations, the processed output vector can be used as activation input to arithmetic circuit 503, for example, for use in subsequent layers of a neural network.

[0106] The unified memory 506 is used to store input data and output data.

[0107] The weight data is directly transferred from the external memory to the input memory 501 and / or the unified memory 506 through the Direct Memory Access Controller (DMAC) 505, the weight data in the external memory is stored in the weight memory 502, and the data in the unified memory 506 is stored in the external memory.

[0108] BIU stands for Bus Interface Unit 510, which is used to enable interaction between the main CPU, DMAC and instruction fetch memory 509 via a bus.

[0109] The instruction fetch buffer 509, connected to the controller 504, is used to store the instructions used by the controller 504.

[0110] The controller 504 is used to call the instructions cached in the instruction memory 509 to control the operation of the computing accelerator.

[0111] Generally, the unified memory 506, input memory 501, weighted memory 502, and instruction fetch memory 509 can all be on-chip memories. The external memory of the NPU can be memory located outside the NPU itself, such as double data rate synchronous dynamic random access memory (DDR SDRAM), high bandwidth memory (HBM), or other readable and writable memory. It should be understood that... Figure 5 The chip hardware structure shown is merely illustrative and is not intended to limit the scope of this application.

[0112] The following is combined with Figure 6 The semantic matching method of the embodiments of this application will be described in detail. This method can be... Figure 2 The execution device 210 performs the training. The target model / rules are configured within the execution device 210. Figure 5 The chip structure. As mentioned above, the execution device can be a mobile phone, tablet computer, laptop computer, augmented reality (AR), virtual reality (VR), vehicle terminal, etc., or it can be a server or cloud. The semantic matching method of this application embodiment is described below using a mobile phone as an example of the execution device.

[0113] like Figure 6 As shown, the semantic matching method provided in this application embodiment includes steps S601-S605.

[0114] In step S601, the text 1 and text 2 to be matched are obtained.

[0115] In this embodiment, text 1 can be obtained by the mobile phone acquiring text input by the user. For example, the mobile phone receives voice data input by the user and identifies the corresponding text through the phone's voice recognition module; or the user inputs image data and identifies the corresponding text through the phone's image recognition module; or the mobile phone receives text data directly typed by the user. Text 1 can also be obtained by the mobile phone acquiring text selected by the user. For example, the mobile phone determines text 1 based on the text selected by the user on the phone's display interface or based on the voice command selected by the user. In this embodiment, the mobile phone can interact with the user through various methods to obtain text 1, and this application does not limit this.

[0116] Text 2 is text stored on the server or mobile phone. For example, in an intelligent question-answering scenario, Text 2 is a question from a pre-configured standard question-answering library (e.g., a manually maintained standard question-answering library). When the mobile phone receives Text 1, it retrieves the question from the standard question-answering library configured on the mobile phone / server and obtains Text 2. Of course, in other scenarios, Text 2 will be adapted to different application scenarios.

[0117] It is understandable that both Text 1 and Text 2 can be long or short text, or one of them can be short and the other long. Short text refers to text with a character count not exceeding a preset threshold. For example, if the preset threshold is 5, then text with more than 5 characters is considered long text. For instance, "What is the preceding line to 'Do not forget to tell your grandfather at the family sacrifice'?" has more than 5 characters and is therefore considered long text. Text with a character count less than or equal to 5 is considered short text. For example, "What is today's date?" has no more than 5 characters and is therefore considered short text.

[0118] The strings of Text 1 and Text 2 include, but are not limited to, words, punctuation marks, special characters, etc. Optionally, Text 1 and Text 2 can be Chinese texts, such as "我爱中国", or foreign language texts, such as the English "I love China", or a mixture of Chinese and foreign languages, such as the mixture of Chinese and English "I love 中国". That is, this application does not limit the length of the characters, the content of the characters, the language of the text content, etc. of Text 1 and Text 2.

[0119] In steps S602 to S604, Text 1 and Text 2 are input into the semantic similarity prediction network branch to determine the similarity between Text 1 and Text 2, extract the feature vectors of Text 1 and Text 2, and input the extracted feature vectors into the similarity dynamic threshold prediction network branch to determine the similarity dynamic threshold between Text 1 and Text 2.

[0120] See Figure 7 , the target model / rule 201 disclosed in the embodiment of this application, that is, the semantic matching model includes a semantic similarity prediction network branch and a similarity dynamic threshold prediction network branch. After obtaining Text 1 and Text 2 to be matched through step S601, Text 1 and Text 2 are input into the semantic matching model. Text 1 and Text 2 obtain the similarity P between Text 1 and Text 2 through the trained semantic similarity prediction network branch. The first feature vector is extracted from Text 1 and Text 2 through the manual feature module, and the similarity dynamic threshold T between Text 1 and Text 2 is obtained by inputting the first feature vector into the similarity dynamic threshold prediction network branch. opt .

[0121] It is understandable that semantic similarity prediction networks can be DSSM (Deep Structured Semantic Models), CNN-DSSM (Convolutional Neural Network-Deep Structured Semantic Models), or LSTM-DSSM (Long Short Term Memory-Deep Structured Semantic Models), etc. Alternatively, it can involve a pre-trained + fine-tuned approach, where a pre-trained semantic feature extractor model is trained using a large-scale unlabeled corpus, and then a sentence similarity classifier is connected downstream of the pre-trained semantic feature extractor for task fine-tuning. In other words, the semantic similarity prediction network includes a pre-trained model and a downstream task model (similarity classifier), where the pre-trained semantic feature extractor model can be a Transformer model, or a BERT (Bidirectional Encoder Representations from Transformers) model, etc. Fine-tuning refers to performing small-scale training on a pre-trained model using specific task objectives (downstream tasks) and task data (downstream data) to make minor adjustments to the parameters of the pre-trained model, ultimately resulting in a model adapted to specific data and tasks.

[0122] In one example, such as Figure 7 As shown, the semantic similarity prediction network includes a BERT model and a similarity classifier network. After text 1 and text 2 are input into the BERT model, the BERT model extracts sentence vectors (i.e., second feature vectors) representing text 1 and text 2. The output of the BERT model (i.e., sentence vectors of text 1 and text 2) is used as the input of the similarity classifier network to obtain the similarity P between text 1 and text 2.

[0123] It is easy to understand that this application does not limit the type of similarity classifier network. The similarity classifier network can be any neural network, such as Convolutional Neural Network (CNN), LSTM (Long Short Term Memory) network, attention mechanism network, etc.

[0124] In some examples, the semantic similarity prediction network is trained using supervised learning, i.e., on labeled corpora (i.e., training text pairs). The labeled corpora can include corpora labeled with "match" and corpora labeled with "mismatch". For example, matching corpora can be labeled with "1" and mismatch corpora can be labeled with "0".

[0125] Labeled corpora are input into a semantic similarity prediction network to obtain predicted values. These predicted values ​​are compared with the labeled values ​​(i.e., labels). Based on the differences, the parameters of the semantic similarity prediction network are updated, that is, the weight vector of each layer of the neural network is updated (of course, there is usually an initialization process before the first update, that is, the parameters of each layer in the neural network are pre-configured). For example, if the predicted value output by the similarity prediction network is too high, the weight vector is adjusted to make it predict lower. This adjustment is continued until the similarity prediction network can predict the true target value. Through multiple rounds of training iterations, the similarity prediction network is trained until the loss function converges, or the difference between the predicted value and the labeled value is less than a certain threshold, or the number of training iterations reaches a preset number.

[0126] See also Figure 7 Before text 1 and text 2 enter the similarity dynamic threshold prediction network, they must first pass through a manual feature extraction module to obtain the first feature vector. The first feature vector can represent features not extracted by the BERT model, such as length-related features.

[0127] Because the BERT model is a network model based on a self-attention mechanism, such as Figure 8 As shown. The essence of the self-attention mechanism is to determine the specific meaning of the current word by combining the context within the sentence. Its limitation is that if the input sentence is short and there isn't much context to refer to, the confidence level of the sentence vector output by the BERT model for the short sentence should be lower than that of the sentence vector for a longer sentence under the same conditions. In other words, if the length of the input sentences is unequal, the confidence level of the sentence vector output by BERT will be different, and not all of them will be equal to 1. Figure 9 As shown. However, the similarity probability derived by the BERT model from sentence vectors is based on the confidence level of the sentence vector itself being 1. The feature vectors extracted by the BERT model for short or long texts are fixed-length sentence vectors, such as 768-dimensional, which ignores the influence of sentence length features on the predicted similarity. This will result in poor accuracy of the similarity prediction between text 1 and text 2 by the semantic similarity prediction network, especially when text 1 and text 2 are short sentences.

[0128] With this in mind, the first feature vector can be a length-related feature representing text 1 and text 2, to correct the poor prediction accuracy of semantic similarity prediction networks for short sentences. The manual feature extraction module can extract the required feature vectors programmatically, without participating in the training of the dynamic similarity threshold prediction network.

[0129] In one example, the length features of text 1 and text 2 are characterized by length-related features of text 1 and text 2. These could include, for example, the absolute value L1 of the length difference between the first and second texts, L2 of the sum of the lengths of text 1 and text 2, and L of the sum of the lengths of common subsequences of text 1 and text 2. common and L common One or more of the ratios of L1 and L2, wherein the length of the common subsequence is determined based on the number of characters in the same string, the same string being a string in which N consecutive characters are the same in the strings of text1 and text2, where N is a positive integer greater than 1.

[0130] This application embodiment extracts the absolute value L1 of the length difference between text 1 and text 2, L2 of the sum of the lengths of text 1 and text 2, and L of the sum of the common subsequences of text 1 and text 2. common and L common The ratio of L2 to the length-related features of the matched text is used as input to the similarity dynamic threshold prediction network to determine the dynamic similarity threshold. The dynamic similarity threshold is then used to correct the similarity output of the existing semantic similarity prediction network to obtain more accurate matching results and improve the accuracy of semantic matching results.

[0131] In another example, the first feature vector can also represent features not extracted by other BERT models, such as the semantic features of individual characters in text 1 and text 2. The hand-crafted feature module extracts the first feature vectors representing the semantic features of individual characters in text 1 and text 2 respectively. Then, the first feature vectors of text 1 and text 2 are concatenated and used as input to the similarity dynamic threshold prediction network to obtain the similarity dynamic threshold T. opt .

[0132] In another example, the first feature vector can also represent multiple features not extracted by the BERT model. For example, the first feature vector represents the semantic features of each character in text 1 and text 2, as well as the length features of text 1 and text 2. The hand-crafted feature module extracts the first feature vectors representing the semantic features of each character in text 1, the first feature vectors representing the semantic features of each character in text 2, and the feature vectors representing the length features of text 1 and text 2, respectively. The first feature vectors of text 1, text 2, and the feature vectors representing the length features of text 1 and text 2 are concatenated and used as input to the similarity dynamic threshold prediction network to obtain the similarity dynamic threshold T. opt .

[0133] This application embodiment further extracts other features of text 1 and text 2 that were not extracted by the BERT model, and uses them as input to the similarity dynamic threshold prediction network to obtain the similarity dynamic threshold. This corrects the similarity output of the existing semantic similarity prediction network in more dimensions, thereby further improving the accuracy of semantic matching.

[0134] In one example, the similarity dynamic threshold prediction network is a feedforward neural network, such as a multilayer perceptron network.

[0135] In another example, the similarity dynamic threshold prediction network could also be a more complex neural network, such as a convolutional neural network (CNN).

[0136] The similarity dynamic threshold prediction network is trained using supervised learning, i.e., on labeled training data. The training data includes the first feature vector corresponding to the training corpus and the label corresponding to the first feature vector. The label is determined based on the semantic similarity prediction network's output for training text pairs and the corresponding label for those training text pairs (i.e., whether the known training text pairs match or do not match; a match is labeled 1, and a mismatch is labeled 0).

[0137] In other words, during the training phase of the similarity dynamic threshold prediction network, the similarity prediction network, trained on the input text pairs, produces the predicted value output by the similarity prediction network. Based on the difference between the predicted value output by the similarity network and the label corresponding to the training text (i.e., whether the known training text pairs match or do not match, labeled 1 if they match and 0 if they do not match), the label corresponding to the first feature vector is determined.

[0138] The training text is manually extracted by the input module to obtain the first feature vector. This first feature vector is then used as input to the dynamic threshold prediction network for similarity, resulting in a predicted value from the dynamic threshold prediction network. The predicted value from the dynamic threshold prediction network is compared with the predicted value output by the similarity prediction network and the label corresponding to the first feature vector. Based on the difference between the two, the parameters of the semantic similarity prediction network are updated, i.e., the weight vector of each layer of the neural network is updated (of course, there is usually an initialization process before the first update, i.e., pre-configuring the parameters of each layer in the neural network). For example, if the predicted value output by the dynamic threshold prediction network is too high, the weight vector is adjusted to make it predict lower. This adjustment continues until the dynamic threshold prediction network can predict the true target value. Through multiple rounds of training iterations, the training of the dynamic threshold prediction network is completed when the loss function of the dynamic threshold prediction network converges, or the difference between the predicted value and the label is less than a certain threshold, or the number of training iterations reaches a preset number.

[0139] Back Figure 6 In step S605, the similarity P of the similarity prediction network branch output and the dynamic threshold T of the similarity prediction network output are used to predict the similarity. opt Determine the matching result between text 1 and text 2.

[0140] If the similarity P is greater than or equal to the dynamic similarity threshold T opt If text 1 and text 2 match, then the similarity P is greater than or equal to the dynamic similarity threshold T. opt If the match is true, then text 1 and text 2 are determined to be a match; otherwise, text 1 and text 2 are determined to be a mismatch.

[0141] When using intelligent question answering, if text 1 and text 2 are determined to match, the corresponding answer for text 2 in the standard question answering database is retrieved and provided to the user. If text 1 and text 2 do not match, text 1 is matched against other questions in the standard question answering database until a match is found, and the corresponding answer is provided to the user. If, after iterating through all questions in the standard question answering database, text 1 still does not find a similar question, matching stops, and a report is sent to the cloud server indicating that no match was found. Please maintain the information in the standard question answering database promptly.

[0142] Compared to the currently popular pretrained + fine-tuned semantic matching scheme in the NLP field, the semantic matching method in this application adds a dynamic threshold prediction network branch. The similarity of the output of the dynamic threshold prediction network is dynamically adjusted by the similarity of the output of the similarity prediction network, thereby improving the accuracy of semantic matching.

[0143] Figure 10This application provides a schematic diagram of the structure of a semantic matching device. For example... Figure 10 As shown, the semantic matching device 300 includes at least:

[0144] The acquisition module 301 is used to acquire the first and second texts to be matched;

[0145] The first determining module 302 is used to input the first text and the second text into a semantic similarity prediction network to determine the similarity between the first text and the second text.

[0146] Extraction module 303 is used to extract the first feature vector of the first text and the second text;

[0147] The second determining module 304 is used to input the first feature vector into the similarity dynamic threshold prediction network to determine the similarity dynamic threshold corresponding to the first text and the second text.

[0148] The matching module 305 is used to determine the matching result of the first text and the second text based on the similarity and the dynamic similarity threshold.

[0149] In one possible implementation, the first feature vector represents at least the length features of the first and second texts.

[0150] In another possible implementation, the length feature includes at least: the absolute value of the length difference between the first text and the second text L1, the sum of the lengths of the first text and the second text L2, and the sum of the lengths of the common subsequences of the first text and the second text L... common , and the L common One or more of the ratios of L2 and L2, wherein the length of the common subsequence is determined based on the number of characters in the same string, the same string being a string in which N consecutive characters are the same in the string of the first text and the string of the second text, where N is a positive integer greater than 1.

[0151] In another possible implementation, the first feature vector at least represents the length features of the first and second texts, as well as the word semantic features of the first and second texts.

[0152] In another possible implementation, the matching module 305 is further configured to: determine that the first text and the second text match if the similarity is greater than or equal to the similarity dynamic threshold, and otherwise determine that the first text and the second text do not match.

[0153] In another possible implementation, the semantic similarity prediction network includes a pre-trained feature extractor and a similarity classifier;

[0154] The first determining module 302 is further configured to:

[0155] The pre-trained feature extractor extracts second feature vectors from the first text and the second text respectively, wherein the second feature vectors represent the semantic features of the first text and the second text;

[0156] The second feature vector is input into the similarity classifier to determine the similarity between the first text and the second text.

[0157] In another possible implementation, the pre-trained feature extractor is a BERT model.

[0158] In another possible implementation, the semantic similarity prediction network is trained based on training text pairs and the labels corresponding to the training text pairs;

[0159] The similarity dynamic threshold prediction network is trained based on the first feature vector of the training text pair and the label corresponding to the first feature vector;

[0160] The label corresponding to the first feature vector is determined based on the output of the semantic similarity prediction network for the training text pair and the label corresponding to the training text pair.

[0161] In another possible implementation, the similarity dynamic threshold prediction network is a feedforward neural network.

[0162] The semantic matching device 300 according to the embodiments of this application can correspond to the execution of the methods described in the embodiments of this application, and the above and other operations and / or functions of each module in the semantic matching device 300 are respectively for implementing Figure 6-9 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0163] It should also be noted that the embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines.

[0164] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform any of the methods described above.

[0165] This application also provides a computer program or computer program product that includes instructions that, when executed, cause a computer to perform any of the methods described above.

[0166] This application also provides an electronic device, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement any of the methods described above.

[0167] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0169] As shown in Figure 11, the electronic device 1100 includes a processor 1101, a memory 1102, a bus 1103, a microphone 1104, a speaker 1105, a display 1106, and a communication interface 1107. The processor 1101, memory 1102, microphone 1104, speaker 1105, display 1106, and communication interface 1107 communicate via the bus 1103, or via wireless transmission or other means. The microphone 1104 can receive voice data input by the user, such as a question in a smart question-and-answer scenario; the speaker 1105 can play audio data, such as, in a smart question-and-answer scenario, the speaker 1105 playing voice data containing the answer to the user's question; the display 1106 can display multimedia content, such as, in a smart question-and-answer scenario, the display 1106 displaying the answer to the user's question, which can be text content and / or image content and / or video content, etc. For example, if a user asks "What will the weather be like tomorrow?", the display can show the user the text "Tomorrow will be sunny, with a temperature of 10-20 degrees Celsius and a northerly wind of level 6". If the user asks "What does a car look like?", the display will show the user an image of a car. If the user asks "How do a standard push-up?", the display will show the user a video of a standard push-up. The communication interface 1107 is used for communication connections with other communication devices; the memory 1102 stores executable program code, and the processor 1101 can call the program code stored in the memory 1102 to execute the semantic matching method in the aforementioned method embodiments.

[0170] It should be understood that, in the embodiments of this application, the processor 1101 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0171] The memory 1102 may include read-only memory and random access memory, and provides instructions and data to the processor 1101. The memory 1102 may also include non-volatile random access memory. For example, the memory 1102 may also store a training dataset.

[0172] The memory 1102 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DRRAM).

[0173] In addition to the data bus, bus 1103 may also include a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus 1103 in the figure.

[0174] It should be understood that the electronic device 1100 according to the embodiments of this application may correspond to the semantic matching device in the embodiments of this application, and may correspond to the execution of the device according to the embodiments of this application. Figure 6-9 The corresponding entities in the method shown, and the above and other operations and / or functions of each device in the electronic device 1100, are respectively implemented to achieve Figure 6-9 The corresponding processes of each method will not be elaborated here for the sake of brevity.

[0175] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0176] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, processor-executed software modules, or a combination of both. The software modules can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0177] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A semantic matching method, characterized in that, include: Get the first and second text to be matched; The first text and the second text are input into a semantic similarity prediction network to determine the similarity between the first text and the second text. Extract the first feature vector from the first text and the second text; The first feature vector is input into the similarity dynamic threshold prediction network to determine the similarity dynamic thresholds corresponding to the first text and the second text. If the similarity is greater than or equal to the dynamic similarity threshold, then the first text and the second text are determined to match; otherwise, the first text and the second text are determined not to match. The semantic similarity prediction network includes a pre-trained feature extractor and a similarity classifier; The step of inputting the first text and the second text into a semantic similarity prediction network to determine the similarity between the first text and the second text includes: The pre-trained feature extractor extracts second feature vectors from the first text and the second text respectively, wherein the second feature vectors represent the semantic features of the first text and the second text; The second feature vector is input into the similarity classifier to determine the similarity between the first text and the second text.

2. The method according to claim 1, characterized in that, The first feature vector at least represents the length features of the first text and the second text.

3. The method according to claim 2, characterized in that, The length features include at least: the absolute value of the length difference between the first text and the second text L1, the length of the first text and the sum of the lengths of the second text L2, and the sum of the lengths of the common subsequences of the first text and the second text L... common and the L common One or more of the ratios of L2 and L2, wherein the length of the common subsequence is determined based on the number of characters in the same string, the same string being a string in which N consecutive characters are the same in the string of the first text and the string of the second text, where N is a positive integer greater than 1.

4. The method according to claim 1, characterized in that, The first feature vector at least represents the length features of the first text and the second text, as well as the word semantic features of the first text and the second text.

5. The method according to claim 1, characterized in that, The pre-trained feature extractor is the BERT model.

6. The method according to claim 1, characterized in that, The semantic similarity prediction network is trained based on training text pairs and the labels corresponding to the training text pairs; The similarity dynamic threshold prediction network is trained based on the first feature vector of the training text pair and the label corresponding to the first feature vector; The label corresponding to the first feature vector is determined based on the output of the semantic similarity prediction network for the training text pair and the label corresponding to the training text pair.

7. The method according to any one of claims 1-6, characterized in that, The similarity dynamic threshold prediction network is a feedforward neural network.

8. A semantic matching device, characterized in that, include: The acquisition module is used to acquire the first and second texts to be matched. The first determining module is used to input the first text and the second text into a semantic similarity prediction network to determine the similarity between the first text and the second text. The extraction module is used to extract the first feature vector of the first text and the second text; The second determining module is used to input the first feature vector into the similarity dynamic threshold prediction network to determine the similarity dynamic thresholds corresponding to the first text and the second text. The matching module is used to determine that the first text and the second text match when the similarity is greater than or equal to the dynamic similarity threshold, and otherwise determine that the first text and the second text do not match. The semantic similarity prediction network includes a pre-trained feature extractor and a similarity classifier; The first determining module is further configured to: The pre-trained feature extractor extracts second feature vectors from the first text and the second text respectively, wherein the second feature vectors represent the semantic features of the first text and the second text; The second feature vector is input into the similarity classifier to determine the similarity between the first text and the second text.

9. The apparatus according to claim 8, characterized in that, The first feature vector at least represents the length features of the first text and the second text.

10. The apparatus according to claim 9, characterized in that, The length features include at least: the absolute value of the length difference between the first text and the second text L1, the length of the first text and the sum of the lengths of the second text L2, and the sum of the lengths of the common subsequences of the first text and the second text L... common and the L common One or more of the ratios of L2 and L2, wherein the length of the common subsequence is determined based on the number of characters in the same string, the same string being a string in which N consecutive characters are the same in the string of the first text and the string of the second text, where N is a positive integer greater than 1.

11. The apparatus according to claim 8, characterized in that, The first feature vector at least represents the length features of the first text and the second text, as well as the word semantic features of the first text and the second text.

12. The apparatus according to claim 8, characterized in that, The pre-trained feature extractor is the BERT model.

13. The apparatus according to claim 8, characterized in that, The semantic similarity prediction network is trained based on training text pairs and the labels corresponding to the training text pairs; The similarity dynamic threshold prediction network is trained based on the first feature vector of the training text pair and the label corresponding to the first feature vector; The label corresponding to the first feature vector is determined based on the output of the semantic similarity prediction network for the training text pair and the label corresponding to the training text pair.

14. The apparatus according to any one of claims 8-13, characterized in that, The similarity dynamic threshold prediction network is a feedforward neural network.

15. An electronic device comprising a memory and a processor, characterized in that, The memory stores executable code, and the processor executes the executable code to implement the method according to any one of claims 1-7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed in a computer, it causes the computer to perform the method described in any one of claims 1-7.

Images