Text processing method, device, electronic device and storage medium

Abstract

The embodiment of the present application discloses a text processing method, device, electronic device and storage medium. It belongs to the field of natural language processing technology. The method includes: obtaining a first text content and a first word vector; obtaining a second text content and a second word vector; obtaining a target probability based on an attention mechanism, a first word vector and a second word vector; if the target probability value is greater than a probability threshold, determining that the first text content and the second text content are semantically matched. Thus, in the process of semantically matching the first text content and the second text content, by pre-storing the word vector of the second text content used for semantic matching with the first text content, in the actual semantic matching process, it is no longer necessary to input the second text content into the model to obtain the word vector, but the previously pre-stored word vector can be directly used, thereby reducing the time consumption in the text processing process and improving the efficiency of semantic matching.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and more specifically, to a text processing method, device, electronic device and storage medium. Background Art

[0002] With the development of natural language processing technology, computers can process the acquired text content to obtain text content with similar or identical semantics. In this process, it is necessary to obtain the semantic similarity between texts. However, in the relevant methods of obtaining the semantic similarity between texts, the efficiency of obtaining the semantic similarity needs to be improved. Summary of the invention

[0003] In view of the above problems, the present application proposes a text processing method, device, electronic device and storage medium to improve the above problems.

[0004] In a first aspect, the present application provides a text processing method, the method comprising: obtaining first text content and a first word vector, the first word vector being the word vector corresponding to the first text content; obtaining second text content and a second word vector, the second text content being the text content currently used for semantic matching with the first text content, the second word vector being a pre-stored word vector corresponding to the second text content; obtaining a target probability based on an attention mechanism, the first word vector and the second word vector, the target probability representing the degree of semantic matching between the first text content and the second text content; if the target probability value is greater than a probability threshold, determining that the first text content and the second text content are semantically matched.

[0005] In a second aspect, the present application provides a text processing method, the method comprising: obtaining text input into an information query interface as a first text content, obtaining a first word vector, the first word vector being a word vector corresponding to the first text content; obtaining a second text content and a second word vector, the second text content being text content used to match the first text content, the second word vector being a pre-stored word vector corresponding to the second text content; obtaining a target probability based on an attention mechanism, the first word vector, and the second word vector, the target probability representing a degree of semantic matching between the first text content and the second text content; if the target probability value is greater than a probability threshold, determining that the first text content is semantically matched with the second text content; and outputting information associated with the second text content to the information query interface.

[0006] In a third aspect, the present application provides a text processing device, the device comprising: a first data acquisition unit, a second data acquisition unit, a probability acquisition unit and a text matching unit. The first data acquisition unit is used to acquire a first text content and a first word vector, the first word vector being the word vector corresponding to the first text content; the second data acquisition unit is used to acquire a second text content and a second word vector, the second text content being the text content currently used for semantic matching with the first text content, the second word vector being the pre-stored word vector corresponding to the second text content; the probability acquisition unit is used to acquire a target probability based on an attention mechanism, the first word vector and the second word vector, the target probability representing the degree of semantic matching between the first text content and the second text content; the text matching unit is used to determine that the first text content and the second text content are semantically matched if the target probability value is greater than a probability threshold.

[0007] In a fourth aspect, the present application provides a text processing device, which includes: a query data acquisition unit, a second data acquisition unit, a probability acquisition unit, a text matching unit, and an information output unit.

[0008] Among them, a query data acquisition unit is used to acquire the text input in the information query interface as the first text content, and acquire a first word vector, wherein the first word vector is the word vector corresponding to the first text content; a second data acquisition unit is used to acquire the second text content and the second word vector, wherein the second text content is the text content used to match the first text content, and the second word vector is a pre-stored word vector corresponding to the second text content; a probability acquisition unit is used to acquire a target probability based on the attention mechanism, the first word vector and the second word vector, wherein the target probability represents the degree of semantic matching between the first text content and the second text content; a text matching unit is used to determine that the first text content is semantically matched with the second text content if the target probability value is greater than a probability threshold; and an information output unit is used to output information associated with the second text content to the information query interface.

[0009] In a fifth aspect, the present application provides an electronic device comprising a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above method.

[0010] In a sixth aspect, the present application provides a computer-readable storage medium, in which a program code is stored, wherein the above method is executed when the program code is executed by a processor.

[0011] The present application provides a text processing method, device, electronic device and storage medium, which first obtains the first text content and the word vector corresponding to the first text content as the first word vector, and obtains the second text content currently used for semantic matching with the first text content and the second word vector corresponding to the second text content, and then obtains the target probability based on the attention mechanism, the first word vector and the second word vector, wherein the target probability represents the degree of semantic matching between the first text content and the second text content; if the target probability value is greater than the probability threshold, it is determined that the first text content and the second text content are semantically matched. Thus, in the process of semantic matching between the first text content and the second text content, by pre-storing the word vector of the second text content used for semantic matching with the first text content, in the actual semantic matching process, it is no longer necessary to input the second text content into the model to obtain the word vector, but the previously pre-stored word vector can be directly used, thereby reducing the time consumption in the text processing process and improving the efficiency of semantic matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A schematic diagram showing an application environment involved in an embodiment of the present application is shown;

[0014] Figure 2 A flowchart of a text processing method proposed in one embodiment of the present application is shown;

[0015] Figure 3 A flowchart of a text processing method proposed in another embodiment of the present application is shown;

[0016] Figure 4 Shows Figure 3 A flowchart of the steps included in S250;

[0017] Figure 5 A flowchart of a text processing method proposed in another embodiment of the present application is shown;

[0018] Figure 6 A flowchart of a text processing method proposed in another embodiment of the present application is shown;

[0019] Figure 7 A schematic diagram showing three stages of a text processing method proposed in this application;

[0020] Figure 8 A flowchart of a text processing method proposed in another embodiment of the present application is shown;

[0021] Fig. 9 A schematic diagram of an information query interface in an embodiment of the present application is shown;

[0022] Fig.10 A schematic diagram showing information associated with a second text outputted in an information query interface in an embodiment of the present application;

[0023] Fig.11 A structural block diagram of a text processing device proposed in an embodiment of the present application is shown;

[0024] Fig.12 A structural block diagram of a text processing device proposed in another embodiment of the present application is shown;

[0025] Fig.13 A structural block diagram of an electronic device for executing a text processing method according to an embodiment of the present application is shown;

[0026] Fig.14 A storage unit for storing or carrying program codes for implementing a text processing method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0029] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0030] Among them, with the development of text processing technology in artificial intelligence technology, many scenarios involving text recognition based on text processing technology and natural language processing technology have emerged. For example, the intelligent question-and-answer scenario. In the intelligent question-and-answer scenario, users can input the questions they want to know in the form of text, and the intelligent question-and-answer system can query the corresponding answers based on the questions input by the user and feedback to the user. For another example, in the news information search scenario, users can input the topics they want to know in the form of text, and the search system can search for corresponding information based on the topic and feedback to the user.

[0031] However, after studying the related text recognition methods in the text recognition scenario, the inventors found that the related text recognition methods still have the problem of recognition efficiency to be improved. In the related text recognition methods, the specific meaning of the text input by the user will be identified by semantic matching. In the process of semantic matching, the contents of the two texts that need to be semantically matched (the text input by the user and the text used to match the text input by the user) are input into the model, and then the output vector of the model is used to calculate whether the two texts for semantic matching are similar. In the related recognition method, each time, the two texts to be matched need to be input into the model to obtain the output vector of the model for semantic matching, and the model is time-consuming during operation, which will cause delays in text recognition and affect recognition efficiency. In addition, when there are many texts that need to be compared, it will cause greater delays. For example, if it is necessary to find a text that matches the text input by the user from 100 texts, it is necessary to semantically match the text input by the user with the 100 texts.

[0032] Therefore, in order to improve the above-mentioned problems, the inventors proposed the text processing method, device, electronic device and storage medium provided in the present application. In the method, the first text content and the word vector corresponding to the first text content are first obtained as the first vector, and the second text content currently used for semantic matching with the first text content and the second vector corresponding to the second text content are obtained. Then, based on the attention mechanism, the first vector and the second vector, the target probability is obtained, and the target probability represents the degree of semantic matching between the first text content and the second text content; if the target probability value is greater than the probability threshold, it is determined that the first text content and the second text content are semantically matched.

[0033] Therefore, when it is necessary to perform semantic matching between the first text content and the second text content, by pre-storing the word vectors of the second text content used for semantic matching with the first text content, in the actual semantic matching process, there is no need to input the second text content into the model to obtain the word vectors, but the previously pre-stored word vectors can be directly used, thereby reducing the time consumption in the text processing process and improving the efficiency of semantic matching.

[0034] Before further describing the embodiments of the present application in detail, an application environment involved in the embodiments of the present application is introduced.

[0035] like Figure 1 As shown, Figure 1 The schematic diagram of the application environment involved in the embodiment of the present application is shown. Among them, a client 110 and a server 120 are included. Among them, the client 110 is used to collect the text input by the user, and then send the collected text to the server 120. After receiving the text input by the user, the server 120 will further obtain the first vector, the second text content and the second vector and then execute the text processing method provided by the embodiment of the present application. When the server 120 recognizes that the first text content and the second text content are semantically matched through the text processing method, the information associated with the second text content is returned to the client 110. After receiving the returned information, the client 110 will display it.

[0036] It should be noted that Figure 1 This is an exemplary application environment, and the method provided in the embodiment of the present application can also be run in other application environments. For example, the text processing method provided in the embodiment of the present application can be executed by the client 110.

[0037] It should be noted that the server 120 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The electronic device where the client 110 is located may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.

[0038] The professional terms involved in the embodiments of the present application are first explained below.

[0039] Word embedding: A word embedding is a vector corresponding to a word or phrase in a text content. The word embedding represents the meaning of a single word or phrase itself.

[0040] Sentence Embedding: The sentence embedding is the vector corresponding to the text content as a whole. The sentence embedding represents the meaning expressed by the text content as a whole.

[0041] For example, the word vector corresponding to the text content "Let's go exercise" may include the word vector corresponding to "I", the word vector corresponding to "we", the word vector corresponding to "go", the word vector corresponding to "sports" and the word vector corresponding to "movement". The sentence vector corresponding to "Let's go exercise" is a vector after linear transformation based on the word vector corresponding to each of the above characters, wherein the vector after linear transformation can represent the overall meaning of the text "Let's go exercise". The overall meaning can be understood as the meaning expressed by the text obtained by combining each character in the text content as a whole.

[0042] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0043] See also Figure 2 , Figure 2 FIG. 1 is a flowchart of a text processing method proposed in an embodiment of the present application, the method comprising:

[0044] S110: Obtain first text content and a first word vector, where the first word vector is a word vector corresponding to the first text content.

[0045] It should be noted that, in the embodiment of the present application, the first text content is the text content that needs to be semantically matched. In the embodiment of the present application, there can be multiple ways to obtain the first text content.

[0046] As a way, the text processing method provided in this embodiment can be triggered by a semantic matching request triggered by a client. If the text content to be semantically matched is carried in the semantic matching request, the semantic matching request can be parsed to obtain the text content carried in the semantic matching request as the first text content.

[0047] As another way, it is not necessary to carry the text content to be semantically matched in the foregoing semantic matching request. In this case, after receiving the semantic matching request, a response message can be returned to the client that sent the semantic matching request. After receiving the response message, the client then sends the text content to be semantically matched, so that the received text content to be semantically matched can be used as the first text content.

[0048] After obtaining the first text content, the corresponding word vector of the first text content can be further obtained as the first word vector. In this embodiment, the word vector corresponding to the first text content includes the word vectors corresponding to each character in the first text content. Exemplarily, if the first text content is "application turnover", then the first word vector corresponding to "application turnover" will include the word vector corresponding to "申", the word vector corresponding to "请", the word vector corresponding to "周", and the word vector corresponding to "转".

[0049] S120: Obtain a second text content and a second word vector. The second text content is the text content currently used for semantic matching with the first text content, and the second word vector is the word vector corresponding to the second text content stored in advance.

[0050] Among them, in this embodiment, the second text content is the text content used for semantic matching with the first text content. In this embodiment, in the process of semantically matching the second text content with the first text content, the matching is based on the word vectors of the first text content and the second text content. Then, after obtaining the second text content, the corresponding word vector of the second text content will be obtained. Among them, the word vector corresponding to the second text content is pre-calculated and stored in a specified storage area. In this way, when the word vector corresponding to the second text content needs to be obtained, it can be directly read from the specified storage area to obtain it, rather than obtaining the word vector corresponding to the second text content by calculation in real time.

[0051] S130: Obtain a target probability based on the attention mechanism, the first word vector, and the second word vector. The target probability represents the degree of semantic matching between the first text content and the second text content.

[0052] It should be noted that the attention mechanism can enable the machine to pay more attention to the key content in the text that affects the semantics of the text, and then update the word vector corresponding to the text content, so that the updated word vector can accurately express the meaning of the corresponding text content. In this embodiment, obtaining the target probability based on the attention mechanism, the first word vector and the second word vector can be understood as inputting the first word vector and the second word vector into a model based on the attention mechanism (attention model), and then updating the first word vector and the second word vector based on the vector output by the model of the attention mechanism, so that the updated first word vector can more accurately express the meaning of the first text content, and correspondingly, the updated second word vector can more accurately express the meaning of the second text content.

[0053] S140: If the target probability value is greater than the probability threshold, determine that the first text content and the second text content are semantically matched.

[0054] As a method, a probability threshold representing the semantic match between two texts can be set in advance, and then after obtaining a target probability representing the degree of semantic match between the first text content and the second text content, the target probability is compared with the probability threshold. If the target probability value is greater than the probability threshold, it is determined that the first text content and the second text content are semantically matched.

[0055] The present embodiment provides a text processing method, which first obtains the first text content and the word vector corresponding to the first text content as the first word vector, and obtains the second text content currently used for semantic matching with the first text content and the second word vector corresponding to the second text content, and then obtains the target probability based on the attention mechanism, the first word vector and the second word vector, the target probability represents the degree of semantic matching between the first text content and the second text content; if the target probability value is greater than the probability threshold, it is determined that the first text content and the second text content are semantically matched. Thus, in the process of semantic matching between the first text content and the second text content, by pre-storing the word vector of the second text content used for semantic matching with the first text content, in the actual semantic matching process, it is no longer necessary to input the second text content into the model to obtain the word vector, but the previously pre-stored word vector can be directly used, thereby reducing the time consumption in the text processing process and improving the efficiency of semantic matching.

[0056] See also Figure 3 , Figure 3 FIG. 1 is a flowchart of a text processing method proposed in an embodiment of the present application, the method comprising:

[0057] S210: Obtain first text content and a first word vector, where the first word vector is a word vector corresponding to the first text content.

[0058] Optionally, in this embodiment, the word vector of the first text content can be obtained through a specified model. For example, the specified model can be a Bert model, an Albert model or a Robert model. As a way, in the process of obtaining the first word vector through a specified model, each word in the first text content can be converted into a one-dimensional vector to obtain an initial one-dimensional vector corresponding to each word in the first text content. It should be noted that the initial one-dimensional vector here can be understood as the initial word vector of each word in the first text content. Then, the initial one-dimensional vector corresponding to each word is input into the specified model, so as to obtain the word vector corresponding to each word in the first text content output by the specified model, and then the word vector corresponding to each word output by the specified model is combined to obtain the first word vector corresponding to the first text content. Optionally, the first word vector corresponding to the first text content obtained by the combination can be in sequence form.

[0059] It should be noted that the word vector corresponding to each character in the first text content output by the specified model can integrate the meaning of the context of the first text content more than the initial one-dimensional vector corresponding to each character in the first text content, so that the word vector corresponding to each character in the first text content output by the specified model can more accurately express the corresponding meaning. Exemplarily, the first text content is "I like to use apples", then the apple can be understood as a kind of fruit, or as a mobile phone brand, and combined with the context, it can be known that for apple users, the action of "using" is not "eating", so the actual intention to be expressed in combination with the meaning of the voice can determine that apple is a mobile phone brand. Then, in the aforementioned initial one-dimensional vector, the one-dimensional vector corresponding to apple may represent a kind of fruit, and after being processed and output by the aforementioned specified model, the word vector corresponding to apple can represent a brand.

[0060] S220: Obtain second text content and a second word vector, where the second text content is text content currently used for semantic matching with the first text content, and the second word vector is a pre-stored word vector corresponding to the second text content.

[0061] The second text content is text content used for semantic matching with the first text content. It should be noted that in this embodiment, the second text content can be obtained based on the first text content, and then the semantics of the second text content and the first text content are matched.

[0062] In this embodiment, there may be multiple ways of acquiring the second text content based on the first text content.

[0063] As a method, in the process of obtaining the second text content based on the first text content, a text with the same or similar content as the first text content can be used as the second text content. It should be noted that the semantics of two texts with the same or similar content may not be the same, so subsequent steps are used to determine whether the first text content and the second text content are actually semantically matched. For example, the two texts "apply for turnover" and "failed to apply for turnover" have the same content of "apply" and "turnover", but the semantics of the two texts "apply for turnover" and "failed to apply for turnover" themselves are different, and the subsequent steps of this embodiment can identify whether the two texts are actually semantically the same.

[0064] S230: Obtain each element in the attention matrix based on the first word vector and the second word vector.

[0065] It should be noted that the attention matrix can be included in the model based on the attention mechanism. In this embodiment, the attention matrix can be constructed based on the first word vector and the second word vector. Exemplarily, the first word vector and the second word vector are both in sequence form. For example, the first word vector is The second word vector is Among them, a la Represents the word vector corresponding to the word in the first text content. For example, a1 represents the word vector corresponding to the first word in the first text content, a2 represents the word vector corresponding to the second word in the first text content, and Represents the word vector corresponding to the lath word in the first text content. Similarly, b1 represents the word vector corresponding to the first word in the first text content, b2 represents the word vector corresponding to the second word in the first text content, and Represent the word vector corresponding to the lbth word in the first text content.

[0066] Optionally, each element in the attention matrix can be obtained based on the following formula:

[0067] e ij :=F'(a i ,b j ):=(a i ) T b j ,

[0068] Among them, e ij Represents the element in the i-th row and j-th column of the attention matrix. Correspondingly, the element in the i-th row and j-th column of the attention matrix is ​​represented by the first word vector in the sequence form The i-th word vector is transposed and the second word vector in sequence form The product of the j-th word vector in is obtained.

[0069] S240: Obtain a reference vector based on each element in the attention matrix and the second word vector.

[0070] Among them, the reference vector can be understood as the vector output by the model where the attention matrix is ​​located. The reference vector represents the attention relationship between each word in the first text content and each word in the second text content. Then, the reference vector can be used to update the first word vector and the second word vector, so that the updated first word vector can more accurately express the actual meaning of the first text content, and the updated second word vector can more accurately express the actual meaning of the second text content.

[0071] Optionally, the reference vector generated based on the attention mechanism can be calculated based on the following formula:

[0072]

[0073] Among them, lhs_aligned i represents the obtained reference vector, and the exp function represents the exponential function with the natural constant e as the base. For example, exp(e ij ) represents the base e and the exponent (e ij ) function. In this formula, exp(e ij )and Then the ratio is added to the second word vector The j-th word vector in is calculated and then the product is based on The reference vector can be obtained by accumulating them.

[0074] S250: Calculate the target probability based on the reference vector, the first word vector, and the second word vector.

[0075] As a way, Figure 4 As shown, the target probability is calculated based on the reference vector, the first word vector and the second word vector, including:

[0076] S251: Concatenate the reference vector and the first word vector to obtain a first vector to be processed.

[0077] Among them, concatenating the reference vector with the first word vector can be understood as the aforementioned updating of the first word vector based on the reference vector, and the corresponding first vector to be processed can be understood as the vector after the first word vector is updated. Because after being processed by the aforementioned attention mechanism, the first vector to be processed can more accurately express the semantic content actually to be expressed by the first text content than the first word vector.

[0078] It should be noted that, in this embodiment, when performing vector concatenation, vector elements of corresponding positions in two vectors may be concatenated. The vector includes multiple vector elements. For example, the first word vector Among them, a1, a2 until are all vector elements in the first word vector, and the second word vector In the middle, b1, b2 and so on are all vector elements in the second word vector, then the vector elements with corresponding positions can be understood as two vector elements with the same position in their respective vectors. For example, in the first word vector, a1 is the vector element at the first position, and if the vector element at the first position in the reference vector is c1, then the vector element a1 and the vector element c1 are vector elements with corresponding positions, and then the vector element c1 and the vector element a1 will be spliced ​​in S251. Then, by splicing the reference vector with the vector elements with corresponding positions in the first word vector, the first vector to be processed can be obtained.

[0079] It should be noted that concatenating vector elements can be understood as taking the contents of each of the two vector elements as the contents of the concatenated vector element. For example, a vector element is [a 11 , a 12 , a 13 , a 14 ], and the other vector element is [b 11 , b 12 , b 13 , b 14 ], then the concatenated vector element is [a 11 , a 12 , a 13 , a 14 , b 11 , b 12 , b 13 , b 14 ].

[0080] S252: Concatenate the vector obtained by transposing the reference vector with the second word vector to obtain a second vector to be processed.

[0081] Correspondingly, concatenating the reference vector with the second word vector can be understood as the aforementioned update of the second word vector based on the reference vector, and the corresponding second processed vector can be understood as the vector after the second word vector is updated. Because after being processed by the aforementioned attention mechanism, the second processed vector can more accurately express the semantic content actually to be expressed by the second text content than the second word vector. Among them, in the process of generating the second processed vector, the aforementioned generated reference vector will be transposed first, and then the transposed vector will be concatenated with the second word vector to obtain the second processed vector.

[0082] S253: Performing pooling processing on the first to-be-processed vector and the second to-be-processed vector, and performing nonlinear processing on the result of the pooling processing to obtain a target probability.

[0083] It should be noted that the aforementioned first vector to be processed and the second vector to be processed still include the word vectors corresponding to each character in the text content. For example, the first vector to be processed still includes the word vectors corresponding to each character in the first text content, but the word vectors corresponding to each character included in the first vector to be processed can more accurately express the actual meaning of each character in the context of the first text content than the word vectors corresponding to each character included in the first word vector. Correspondingly, the second vector to be processed still includes the word vectors corresponding to each character in the second text content, but the word vectors corresponding to each character included in the second vector to be processed can more accurately express the actual meaning of each character in the context of the second text content than the word vectors corresponding to each character included in the second word vector.

[0084] In this embodiment, the first vector to be processed is pooled to integrate the features of each word in the first text content into the features of the first text content as a whole, thereby obtaining a vector that can represent the overall meaning of the first text content. Correspondingly, the second vector to be processed is pooled to integrate the features of each word in the second text content into the features of the second text content as a whole, thereby obtaining a vector that can represent the overall meaning of the second text content.

[0085] Optionally, the first vector to be processed and the second vector to be processed are pooled, and the result of the pooling is nonlinearly processed to obtain a target probability, including: performing a first pooling process on the first vector to be processed to obtain a first vector to be calculated; performing a second pooling process on the second vector to be processed to obtain a second vector to be calculated; performing a nonlinear process on the first vector to be calculated to obtain a first probability value, and performing a nonlinear process on the second vector to be calculated to obtain a second probability value; and calculating the average of the first probability value and the second probability value as the target probability.

[0086] The first pooling process and the second pooling process may be the same. Optionally, performing the first pooling process on the first vector to be processed to obtain the first vector to be calculated includes: performing global maximum pooling process on the first vector to be processed to obtain the first maximum pooling vector; performing global average pooling process on the first vector to be processed to obtain the first average pooling vector; and concatenating the first maximum pooling vector with the first average pooling vector to obtain the first vector to be calculated.

[0087] Among them, optionally, performing a second pooling process on the second vector to be processed to obtain a second vector to be calculated, including: performing a global maximum pooling process on the second vector to be processed to obtain a second maximum pooling vector; performing a global average pooling process on the second vector to be processed to obtain a second average pooling vector; and splicing the second maximum pooling vector with the second average pooling vector to obtain a second vector to be calculated.

[0088] S260: Compare the target probability with the probability threshold.

[0089] S261: If the target probability value is greater than the probability threshold, determine that the first text content and the second text content are semantically matched.

[0090] S262: If the target probability value is not greater than the probability threshold, obtain the next second text content, and perform semantic matching between the next second text content and the first text content.

[0091] It should be noted that each time the second text content is semantically matched with the first text content, it can be performed based on the aforementioned S230 to S260.

[0092] A text processing method provided by this embodiment, in the process of semantically matching the first text content and the second text content, by pre-storing the word vector of the second text content used for semantic matching with the first text content, in the actual semantic matching process, it is no longer necessary to input the second text content into the model to obtain the word vector, but the previously pre-stored word vector can be directly used, thereby reducing the time consumption in the text processing process and improving the efficiency of semantic matching. In addition, in this embodiment, each element in the attention matrix is ​​first constructed based on the first word vector and the second word vector, and then the target probability representing the similarity between the first text content and the second text content is obtained based on the constructed attention matrix, so that the attention mechanism itself can focus more on the characteristics of the key content, so that in the process of calculating the target probability, the calculated target probability can better represent the probability of similarity between the key content in the first text content and the key content in the second text content, thereby improving the accuracy of semantic matching between the first text content and the second text content.

[0093] See also Figure 5 , Figure 5 FIG. 1 is a flowchart of a text processing method proposed in an embodiment of the present application, the method comprising:

[0094] S310: Obtain first text content and a first word vector, where the first word vector is a word vector corresponding to the first text content.

[0095] S320: Based on the sentence vector of the first text content, obtain the text content to be matched and the pre-stored word vector corresponding to the text content to be matched from a designated storage area.

[0096] It should be noted that in the embodiment of the present application, the second text content can be obtained from the text content to be matched. Then the text content to be matched can be obtained first in order to achieve the acquisition of the second text content. Optionally, the designated storage area is a storage area corresponding to a pre-established knowledge base, and the text content to be matched can be obtained from the knowledge base by recall. Optionally, the sentence vector of the first text content can be used as a reference for recalling the text content to be matched. In this way, based on the sentence vector of the first text content, the text content to be matched and the pre-stored word vector corresponding to the text content to be matched are obtained from the designated storage area, including: similarly matching the sentence vector of the first text content with the sentence vector of the text content in the designated storage area; using the successfully matched text content as the text content to be matched, and obtaining the word vector corresponding to the text content to be matched from the designated storage area.

[0097] Among them, in the process of similarity matching the sentence vector of the first text content with the sentence vector of the text content in the specified storage area, the similarity between the texts can be calculated based on the cosine similarity or the Manhattan distance, and then the texts with higher similarity are recalled as the texts to be matched. Exemplarily, if the configuration is to use the top 100 texts ranked by similarity as the texts with higher similarity, then when obtaining the texts to be matched, 100 texts will be obtained, and then the 100 texts will be used as the second text content respectively to obtain the target probabilities between the 100 texts and the first text content respectively.

[0098] In this embodiment, the word vectors corresponding to the text content to be matched are also pre-stored in the designated storage area. For example, the text content to be matched and the corresponding word vectors are both stored in the knowledge base, so that when the text content to be matched is recalled, the word vector corresponding to the text to be matched can be obtained, without having to obtain the text content to be matched and then obtain the word vector corresponding to the text to be matched through the model in real time, or go to other storage areas outside the knowledge base to obtain the word vector of the text content to be matched. The word vectors recalled together with the text content to be matched can be directly buffered in the local memory together with the text content to be matched, so that when the second text content is determined and the corresponding second word vector is obtained, the second word vector can be directly read from the local memory, thereby improving the rate of obtaining the second word vector and further improving the speed of complete semantic matching.

[0099] It should be noted that although the sentence vector of the first text content will be used in the recall process of the text to be matched, the sentence vector used in the recall process is a relatively rough expression of the overall meaning of the first text content, and the vector obtained by the pooling processing of S253 in this embodiment (for example, the first vector to be calculated) is a more accurate expression of the overall meaning of the first text content. Therefore, the first vector to be calculated can express the overall meaning of the first text content more accurately than the sentence vector used in the recall process.

[0100] As a method, before obtaining the text content to be matched from the designated storage area based on the sentence vector of the first text content, it also includes: obtaining the identification of each word in the first text content based on the dictionary; based on the Bert model and the identification of each word, obtaining the word vector and sentence vector of the first text content.

[0101] S330: The text content currently used to match the first text content in the text content to be matched is used as the second text content, and a word vector of the second text content is obtained as the second word vector.

[0102] S340: Obtain a target probability based on the attention mechanism, the first word vector, and the second word vector, where the target probability represents a degree of semantic matching between the first text content and the second text content.

[0103] S350: If the target probability value is greater than the probability threshold, determine that the first text content and the second text content are semantically matched.

[0104] A text processing method provided by the present embodiment, in the process of semantically matching the first text content and the second text content, by pre-storing the word vector of the second text content used for semantic matching with the first text content, in the actual semantic matching process, it is no longer necessary to input the second text content into the model to obtain the word vector, but the previously pre-stored word vector can be directly used, thereby reducing the time consumption in the text processing process and improving the efficiency of semantic matching. In addition, in the present embodiment, the second text content can be selected from the text content to be matched recalled from the designated storage area based on the first text content, and the word vector and sentence vector of each text content have been pre-stored in the designated storage area, so that when recalling the text content to be matched, the word vector corresponding to the text content to be matched can also be recalled at the same time, so that when obtaining the pre-stored word vector corresponding to the second text content, it is not necessary to input it into the model to obtain the word vector, but the word vector can also be directly read from the local, thereby further improving the efficiency of semantic matching.

[0105] See also Figure 6 , Figure 6 A text processing method proposed in an embodiment of the present application is shown, and the method includes:

[0106] S410: Obtain predefined text content.

[0107] It should be noted that the predefined text content is the text content used to match the first text content input by the user. In the process of generating the predefined text content, information associated with each predefined text content can also be generated, so that when it is recognized that the predefined text content has a semantic match with the first text content, the information associated with the predefined text content is fed back to the user.

[0108] For example, in an intelligent question-and-answer scenario, predefined questions can be obtained as predefined text content, and answers corresponding to the predefined questions can be generated. Among them, the answer can be understood as information associated with the predefined question. In the intelligent question-and-answer scenario, the question entered by the user is used as the first text content, and then the answer associated with the predefined question that semantically matches the question entered by the user can be fed back to the user. For another example, in a news information search scenario, predefined news topics can be obtained as predefined content, and consulting content corresponding to each predefined news topic can be generated. Among them, the consulting content can be understood as information associated with the predefined news topic. Then, after the news topic entered by the user is used as the first text content, the consulting content corresponding to the predefined news topic that semantically matches the news topic entered by the user can be fed back to the user.

[0109] S420: Obtain the word vector and sentence vector corresponding to the predefined text content.

[0110] Optionally, the word vector and sentence vector corresponding to the predefined text content can be obtained through the Bert model mentioned in the above embodiment.

[0111] S430: Storing the predefined text content, the corresponding word vectors and sentence vectors in a knowledge base.

[0112] S440: Obtain first text content.

[0113] S450: Obtain a first word vector of the first text content and a sentence vector of the first text content.

[0114] S460: Recalling the text content to be matched and the word vectors corresponding to the text content to be matched from the knowledge base based on the sentence vector of the first text content.

[0115] Among them, recalling text content from the knowledge base can be understood as acquiring text content from the storage area corresponding to the knowledge base. In this embodiment, the text content in the knowledge base is the aforementioned predefined text content. Then recalling the text content to be matched from the knowledge base can be understood as recalling the text content to be matched from the aforementioned predefined text content.

[0116] S470: Obtain second text content from the text content to be matched, and obtain a second word vector.

[0117] S480: Obtain a target probability based on the attention mechanism, the first word vector, and the second word vector, where the target probability represents a semantic match between the first text content and the second text content;

[0118] S490: If the target probability value is greater than the probability threshold, determine that the first text content and the second text content are semantically matched.

[0119] Below through Figure 7 The processing flow involved in this embodiment will be described again.

[0120] for Figure 7 The processing flow shown in the figure may include three stages, namely stage 80, stage 81 and stage 82. Among them, the first stage to be executed is stage 80, which corresponds to the aforementioned S410 to S430. Then, in stage 80, the predefined text content is input into the model (e.g., Bert model), and the word vector 12 and sentence vector of the predefined text content output by the model are obtained, and the word vector 12 and sentence vector are stored in the knowledge base.

[0121] Then, stage 81 will be executed, which corresponds to the aforementioned S450 to S470. Correspondingly, in stage 81, the first text content can be obtained, and the first text content can be input into a model (for example, a Bert model) to obtain a first word vector 10 and a sentence vector corresponding to the first text content. Optionally, the second text content can be recalled from the knowledge base based on the sentence vector of the first text content, and the aforementioned word vector 12 can be obtained.

[0122] During the execution phase 82, the first word vector 10 is input into the attention matrix based on the direction indicated by the arrow 13, and the second word vector 12 is input into the attention matrix based on the direction indicated by the arrow 14, thereby obtaining a reference vector 15 output by the model where the attention matrix is ​​located.

[0123] Afterwards, the first word vector 10 is input into the concatenation layer based on the direction indicated by arrow 20, and the reference vector 15 is input into the concatenation layer based on the direction indicated by arrow 21 for concatenation to obtain the first vector to be processed in the aforementioned embodiment. Also, the second word vector 12 is input into the concatenation layer based on the direction indicated by arrow 22, and the vector 16 after the transposition of the reference vector 15 is input into the concatenation layer based on the direction indicated by arrow 23 for concatenation to obtain the second vector to be processed in the aforementioned embodiment.

[0124] Optionally, if the size of the first word vector 10 is S a ×M, the size of the second word vector 12 is S b ×M, then the reference vector 15 can be S a ×S b , and the size of the vector 16 after the transposition of the reference vector 15 can be S b ×S a After concatenating the first word vector 10 and the reference vector 15, the size of the first vector to be processed can be S a ×(M+S b ), correspondingly, the size of the second vector to be processed can be S b ×(M+S a ).

[0125] For the first vector to be processed, the global maximum pooling layer 40 is input based on the direction of arrow 30 for pooling processing to obtain a first maximum pooling vector, and the global average pooling layer 41 is input based on the direction shown by arrow 31 for pooling processing to obtain a first average pooling vector. For the second vector to be processed, the global maximum pooling layer 40 is input based on the direction shown by arrow 32 for pooling processing to obtain a second maximum pooling vector, and the global average pooling layer 41 is input based on the direction shown by arrow 33 for pooling processing to obtain a second average pooling vector.

[0126] After the first maximum pooling vector is input to the splicing layer based on the direction shown by arrow 42 and the first average pooling vector is input to the splicing layer based on arrow 48 for splicing, the first vector to be calculated obtained by splicing is input to the aggregation layer (Aggregate) 51 along the directions shown by arrow 43 and arrow 49 respectively. After the second maximum pooling vector is input to the splicing layer based on the direction shown by arrow 46 and the second average pooling vector is input to the splicing layer based on arrow 44 for splicing, the second vector to be calculated obtained by splicing is input to the aggregation layer 52 along the directions shown by arrow 45 and arrow 47 respectively. In the aggregation layer 50, the aggregation is performed in a manner that the first vector to be calculated is in front and the second vector to be calculated is in the back. In the aggregation layer (Aggregate) 51, the aggregation is performed in a manner that the second vector to be calculated is in front and the first vector to be calculated is in the back.

[0127] The vector obtained by the aggregation layer 50 will then pass through the fully connected layer (dense) 60 and the nonlinear layer (sigmoid) 70 in sequence to obtain the aforementioned first probability value, and the vector obtained by the aggregation layer 51 will then pass through the fully connected layer 60 and the nonlinear layer 70 in sequence to obtain the aforementioned second probability value, and then the target probability in the aforementioned embodiment is obtained by averaging the first probability value and the second probability value. If the target probability is greater than the probability threshold, it is determined that the first text content and the second text content are semantically matched. Among them, the fully connected layer 60 is used to fully connect and reduce the dimension of the received vector, and output the reduced-dimensional vector to the nonlinear layer 70.

[0128] See also Figure 8 , Figure 8 A text processing method proposed in an embodiment of the present application is shown, and the method includes:

[0129] S510: Obtain text input into the information query interface as first text content, and obtain a first word vector, where the first word vector is a word vector corresponding to the first text content.

[0130] The information query interface may correspond to different information in different scenarios. For example, in the intelligent question-and-answer scenario, the information query interface may be an interface for inputting questions, and the corresponding information queried may be the answer to the question. For another example, in the news information search scenario, the information query interface may be an interface for inputting news topics, and the corresponding information queried may be the consulting content corresponding to the news topics.

[0131] For example, Fig. 9 As shown, in Fig. 9 In the information query interface shown, the user inputs "Why hasn't the product I redeemed arrived in my account yet?", and then "Why hasn't the product I redeemed arrived in my account yet?" is used as the first text content.

[0132] S520: Obtain second text content and a second word vector, where the second text content is text content used to match the first text content, and the second word vector is a pre-stored word vector corresponding to the second text content.

[0133] S530: Obtain a target probability based on the attention mechanism, the first word vector, and the second word vector, where the target probability represents the degree of semantic matching between the first text content and the second text content.

[0134] S540: If the target probability value is greater than the probability threshold, determine that the first text content and the second text content are semantically matched.

[0135] S550: Outputting information associated with the second text content to the information query interface.

[0136] For example, Fig.10 As shown, if the current second text content is "withdrawal arrival time", and it is recognized that the "withdrawal arrival time" matches the semantics of the first text content, the information associated with the "withdrawal arrival time" will be output to the aforementioned Fig. 9 The information query interface shown will then be converted to Fig.10 The display status shown. Fig.10 In the interface shown, the answer corresponding to "withdrawal arrival time" will be displayed.

[0137] A text processing method provided in this embodiment, after obtaining a first text content input from an information query interface, in the process of semantically matching the first text content with the second text content, by pre-storing the word vector of the second text content used for semantic matching with the first text content, so that in the actual semantic matching process, there is no need to input the second text content into the model to obtain the word vector, but the previously pre-stored word vector can be directly used, thereby reducing the time consumption in the text processing process and improving the efficiency of semantic matching, so that after the user enters the content to be queried in the information query interface, the corresponding feedback information can be displayed more quickly.

[0138] See also Fig.11 , Fig.11 The figure shows a text processing device 600 proposed in an embodiment of the present application, and the device 600 includes:

[0139] The first data acquisition unit 610 is used to acquire the first text content and the first word vector, where the first word vector is the word vector corresponding to the first text content.

[0140] The second data acquisition unit 620 is used to acquire second text content and a second word vector, where the second text content is the text content currently used for semantic matching with the first text content, and the second word vector is a pre-stored word vector corresponding to the second text content.

[0141] As a method, the second data acquisition unit 620 is specifically used to obtain the text content to be matched and the pre-stored word vector corresponding to the text content to be matched from a designated storage area based on the sentence vector of the first text content; and use the text content in the text content to be matched that is currently used to match the first text content as the second text content.

[0142] Optionally, the second data acquisition unit 620 is specifically used to perform similarity matching between the sentence vector of the first text content and the sentence vector of the text content in the specified storage area; use the successfully matched text content as the text content to be matched, and obtain the word vector corresponding to the text content to be matched from the specified storage area.

[0143] In addition, the first data acquisition unit 610 is also used to obtain the identifier of each word in the first text content based on the dictionary; based on the Bert model and the identifier of each word, obtain the word vector and sentence vector of the first text content.

[0144] The probability acquisition unit 630 is used to acquire a target probability based on the attention mechanism, the first word vector and the second word vector, where the target probability represents the degree of semantic matching between the first text content and the second text content.

[0145] The text matching unit 640 is configured to determine that the first text content and the second text content are semantically matched if the target probability value is greater than a probability threshold.

[0146] As a method, the probability acquisition unit 630 is specifically used to obtain each element in the attention matrix based on the first word vector and the second word vector; obtain a reference vector based on each element in the attention matrix and the second word vector; and calculate the target probability based on the reference vector, the first word vector and the second word vector.

[0147] Optionally, the probability acquisition unit 630 is specifically used to concatenate the reference vector with the first word vector to obtain a first vector to be processed; concatenate the vector after the transposition of the reference vector with the second word vector to obtain a second vector to be processed; perform pooling on the first vector to be processed and the second vector to be processed, and perform nonlinear processing on the result of the pooling process to obtain a target probability.

[0148] Among them, optionally, the probability acquisition unit 630 is specifically used to perform a first pooling process on the first vector to be processed to obtain a first vector to be calculated; perform a second pooling process on the second vector to be processed to obtain a second vector to be calculated; perform a nonlinear process on the first vector to be calculated to obtain a first probability value, and perform a nonlinear process on the second vector to be calculated to obtain a second probability value; calculate the average of the first probability value and the second probability value as the target probability.

[0149] As a method, the probability acquisition unit 630 is specifically used to perform global maximum pooling processing on the first vector to be processed to obtain a first maximum pooling vector; perform global average pooling processing on the first vector to be processed to obtain a first average pooling vector; and concatenate the first maximum pooling vector and the first average pooling vector to obtain a first vector to be calculated.

[0150] As a method, the probability acquisition unit 630 is specifically used to perform global maximum pooling processing on the second vector to be processed to obtain a second maximum pooling vector; perform global average pooling processing on the second vector to be processed to obtain a second average pooling vector; and concatenate the second maximum pooling vector with the second average pooling vector to obtain a second vector to be calculated.

[0151] See also Fig.12 , Fig.12 The figure shows a text processing device 700 proposed in an embodiment of the present application, and the device 700 includes:

[0152] The query data acquisition unit 710 is used to acquire the text input in the information query interface as the first text content, and acquire the first word vector, which is the word vector corresponding to the first text content.

[0153] The second data acquisition unit 620 is used to acquire second text content and a second word vector, where the second text content is text content used to match the first text content, and the second word vector is a pre-stored word vector corresponding to the second text content.

[0154] The probability acquisition unit 630 is used to acquire a target probability based on the attention mechanism, the first word vector and the second word vector, where the target probability represents the degree of semantic matching between the first text content and the second text content.

[0155] The text matching unit 640 is configured to determine that the first text content and the second text content are semantically matched if the target probability value is greater than a probability threshold.

[0156] The information output unit 720 is used to output the information associated with the second text content to the information query interface.

[0157] The present application provides a text processing device, which, in the process of semantically matching a first text content with a second text content, pre-stores the word vector of the second text content used for semantic matching with the first text content. In this way, in the actual semantic matching process, it is no longer necessary to input the second text content into the model to obtain the word vector, but the previously pre-stored word vector can be directly used, thereby reducing the time consumption in the text processing process and improving the efficiency of semantic matching.

[0158] It should be noted that the device embodiment in the present application corresponds to the aforementioned method embodiment. The specific principles in the device embodiment can be found in the contents of the aforementioned method embodiment and will not be repeated here.

[0159] The following will be combined Fig.13 An electronic device provided by the present application is described.

[0160] See also Fig.13 Based on the above text processing method, the embodiment of the present application also provides another electronic device 200 including a processor 104 that can execute the above text processing method. The electronic device 200 can be a smart phone, a tablet computer, a computer or a portable computer. The electronic device 200 also includes a memory 104 and a network module 106. The memory 104 stores a program that can execute the content of the above embodiment, and the processor 102 can execute the program stored in the memory 104.

[0161] Among them, the processor 102 may include one or more cores for processing data and a message matrix unit. The processor 102 uses various interfaces and lines to connect various parts of the entire electronic device 200, and executes various functions and processes data of the electronic device 200 by running or executing instructions, programs, code sets or instruction sets stored in the memory 104, and calling data stored in the memory 104. Optionally, the processor 102 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 102 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 102, but may be implemented separately through a communication chip.

[0162] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data) created by the terminal 100 during use.

[0163] The network module 106 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. For example, the network module 106 can send broadcast data, and can also parse the broadcast data sent by other devices. The network module 106 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM) cards, memories, etc. The network module 106 can communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. For example, the network module 106 can exchange information with a base station.

[0164] Please refer to Fig.14 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 1100 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0165] The computer readable storage medium 1100 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 1100 includes a non-transitory computer-readable storage medium. The computer readable storage medium 1100 has storage space for program code 810 that performs any method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 1110 can be compressed, for example, in an appropriate form.

[0166] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device performs the above-mentioned text processing method.

[0167] We tested two tasks, question pair matching and semantic similarity calculation (for Chinese and English respectively), and the results are shown in the following table:

[0168] LCQMC(Chinese)Test(ACC) Google Bert-Base native model 86.90％ google bert-base Siamese Network 84.08％ Text processing method in the embodiment of the present application 86.30％ STS-B(English)Test(Pearson / Spearman) Google Bert-Medium native model 81.9％ / 80.6％ google bert-medium Siamese Network 78.4％ / 77.6％ Text processing method in the embodiment of the present application 81.3％ / 80.1％

[0169] Among them, LCQMC is the Chinese data set used in the above test, and STS-B is the English data set used in the above test. Among them, in the test based on the LCQMC data set, the accuracy of the text processing method in the embodiment of the present application can reach 86.30%. In the test based on the STS-B data set, when the Pearson correlation coefficient is used as the judgment basis, the accuracy can reach 81.3%, and when the Spearman correlation coefficient is used as the judgment basis, the accuracy can reach 80.1%.

[0170] In the time-consuming test, the semantic similarity calculation operation was performed on a user's question and 100 candidate questions (which can be understood as predefined questions) in the intelligent customer service environment. The time consumption is shown in the following table:

[0171] Model or algorithm Time consumed (ms) ALBERT-zh-tiny native model 1048 Text processing method in the embodiment of the present application 234 ALBERT-zh-tiny Siamese Network 26

[0172] In summary, the present application provides a text processing method, device, electronic device and storage medium, which first obtains the first text content and the word vector corresponding to the first text content as the first word vector, and obtains the second text content currently used for semantic matching with the first text content and the second word vector corresponding to the second text content, and then obtains the target probability based on the attention mechanism, the first word vector and the second word vector, and the target probability represents the degree of semantic matching between the first text content and the second text content; if the target probability value is greater than the probability threshold, it is determined that the first text content and the second text content are semantically matched. Therefore, in the process of semantically matching the first text content and the second text content, by pre-storing the word vector of the second text content used for semantic matching with the first text content, in the actual semantic matching process, it is no longer necessary to input the second text content into the model to obtain the word vector, but the previously pre-stored word vector can be directly used, thereby reducing the time consumption in the text processing process and improving the efficiency of semantic matching.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

[0174] In this application, when relevant data collection and processing are applied in the embodiments, they should be strictly in accordance with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

Claims

1. A text processing method, characterized in that: The method comprises: Obtaining a first text content and a first word vector, where the first word vector is a word vector corresponding to the first text content; the first word vector includes a word vector corresponding to each word in the first text content; the first text content is text input in an information query interface of an intelligent question and answer scenario; Acquire a second text content and a second word vector, wherein the second text content is the text content currently used for semantic matching with the first text content, and the second word vector is a pre-stored word vector corresponding to the second text content; the second word vector includes a word vector corresponding to each word in the second text content; Acquire a target probability based on an attention mechanism, the first word vector, and the second word vector, wherein the target probability represents a degree of semantic matching between the first text content and the second text content; If the target probability value is greater than the probability threshold, determining that the first text content and the second text content are semantically matched; and outputting information associated with the second text content to the information query interface, the information associated with the second text content being used as an answer to the first text content; The obtaining of the second text content and the second word vector includes: Performing similarity matching on the sentence vector of the first text content and the sentence vector of the text content in the storage area corresponding to the knowledge base, wherein the similarity matching is performed based on cosine similarity or Manhattan distance; The successfully matched text content is used as the text content to be matched, and the word vector corresponding to the text content to be matched is obtained from the storage area corresponding to the knowledge base, so as to buffer the text content to be matched and the word vector corresponding to the text content to be matched in the local memory; the successfully matched text content is a preset number of text contents whose similarity to the first text content is ranked at the top of the similarity ranking; Reading the text content to be matched currently used to match the first text content from the local memory as the second text content, and reading the second word vector of the second text content from the local memory; The text content is input into the model, and the model outputs the word vector of the text content and the sentence vector of the text content; and the first text content is input into the model, and the model outputs the word vector of the first text content and the sentence vector of the text content; The acquiring the target probability based on the attention mechanism, the first word vector and the second word vector includes: Obtain each element in the attention matrix based on the first word vector and the second word vector; Obtain a reference vector based on each element in the attention matrix and the second word vector; Concatenate the reference vector and the first word vector to obtain a first vector to be processed; Concatenate the vector obtained by transposing the reference vector with the second word vector to obtain a second vector to be processed; Performing a first pooling process on the first vector to be processed to obtain a first vector to be calculated; Performing a second pooling process on the second vector to be processed to obtain a second vector to be calculated; The first vector to be calculated is subjected to nonlinear processing to obtain a first probability value, and the second vector to be calculated is subjected to nonlinear processing to obtain a second probability value; An average of the first probability value and the second probability value is calculated as the target probability.

2. The method according to claim 1, characterized in that The performing a first pooling process on the first vector to be processed to obtain a first vector to be calculated includes: Performing global maximum pooling processing on the first to-be-processed vector to obtain a first maximum pooling vector; Performing global average pooling processing on the first to-be-processed vector to obtain a first average pooling vector; The first maximum pooling vector and the first average pooling vector are concatenated to obtain a first vector to be calculated.

3. The method according to claim 1, characterized in that The performing a second pooling process on the second vector to be processed to obtain a second vector to be calculated includes: Performing global maximum pooling processing on the second to-be-processed vector to obtain a second maximum pooling vector; Performing global average pooling processing on the second to-be-processed vector to obtain a second average pooling vector; The second maximum pooling vector is concatenated with the second average pooling vector to obtain a second vector to be calculated.

4. The method according to claim 1, characterized in that: Before similarity matching is performed between the sentence vector of the first text content and the sentence vector of the text content in the storage area corresponding to the knowledge base, the method further includes: Obtaining an identification of each word in the first text content based on a dictionary; Based on the Bert model and the identification of each word, a word vector and a sentence vector of the first text content are obtained.

5. A text processing device, characterized in that: The device comprises: A first data acquisition unit is used to acquire a first text content and a first word vector, where the first word vector is a word vector corresponding to the first text content; the first word vector includes a word vector corresponding to each sub-word in the first text content; the first text content is a text input in an information query interface of an intelligent question and answer scenario; A second data acquisition unit is used to acquire a second text content and a second word vector, wherein the second text content is the text content currently used for semantic matching with the first text content, and the second word vector is a pre-stored word vector corresponding to the second text content; the second word vector includes a word vector corresponding to each character in the second text content; and is specifically used to perform similarity matching between the sentence vector of the first text content and the sentence vector of the text content in the storage area corresponding to the knowledge base, wherein the similarity matching is performed based on cosine similarity or Manhattan distance; the text content with successful matching is used as the text content to be matched, and the word vector corresponding to the text content to be matched is obtained from the storage area corresponding to the knowledge base, so as to buffer the text content to be matched and the word vector corresponding to the text content to be matched in the local memory; the text content with successful matching is a preset number of text contents whose similarity to the first text content is ranked at the top of the similarity ranking; the text content to be matched currently used for matching with the first text content will be read from the local memory as the second text content, and the second word vector of the second text content will be read from the local memory; A probability acquisition unit is used to acquire a target probability based on an attention mechanism, the first word vector and the second word vector, wherein the target probability represents the degree of semantic matching between the first text content and the second text content; the probability acquisition unit is specifically used to obtain each element in the attention matrix based on the first word vector and the second word vector; obtain a reference vector based on each element in the attention matrix and the second word vector; concatenate the reference vector with the first word vector to obtain a first vector to be processed; concatenate the vector after the transposition of the reference vector with the second word vector to obtain a second vector to be processed; perform a first pooling process on the first vector to be processed to obtain a first vector to be calculated; perform a second pooling process on the second vector to be processed to obtain a second vector to be calculated; perform a nonlinear process on the first vector to be calculated to obtain a first probability value, and perform a nonlinear process on the second vector to be calculated to obtain a second probability value; calculate the average of the first probability value and the second probability value as the target probability; A text matching unit, configured to determine that the first text content and the second text content are semantically matched if the target probability value is greater than a probability threshold, and output information associated with the second text content to the information query interface, the information associated with the second text content being used as an answer to the first text content; Among them, the text content is input into the model, and the model outputs the word vector of the text content and the sentence vector of the text content; and the first text content is input into the model, and the model outputs the word vector of the first text content and the sentence vector of the text content.

6. The device according to claim 5, characterized in that The probability acquisition unit is specifically used to perform global maximum pooling processing on the first vector to be processed to obtain a first maximum pooling vector; perform global average pooling processing on the first vector to be processed to obtain a first average pooling vector; and concatenate the first maximum pooling vector with the first average pooling vector to obtain a first vector to be calculated.

7. The device according to claim 6, characterized in that The probability acquisition unit is specifically used to perform global maximum pooling processing on the second vector to be processed to obtain a second maximum pooling vector; perform global average pooling processing on the second vector to be processed to obtain a second average pooling vector; and concatenate the second maximum pooling vector with the second average pooling vector to obtain a second vector to be calculated.

8. The device according to claim 5, characterized in that The first data acquisition unit is specifically used to obtain the identification of each word in the first text content based on a dictionary; based on the Bert model and the identification of each word, obtain the word vector and sentence vector of the first text content.

9. An electronic device, characterized in that: The method comprises a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 4 is executed.

11. A computer program product comprising computer instructions, characterized in that: When the computer instruction is executed by a processor, the method described in any one of claims 1 to 4 is implemented.

Images