Text processing method and device

Abstract

The present application discloses a text processing method, apparatus, computer equipment, storage medium, and computer program product, wherein the method comprises: obtaining a target text, the target text comprising N phrases, N being a positive integer greater than 1; calling a text feature extraction model to perform feature extraction processing on the N phrases to obtain a first original feature sequence and a second original feature sequence, wherein the text feature extraction model comprises M hidden layers arranged in series, the first original feature sequence and the second original feature sequence being output feature sequences of any two hidden layers in the M hidden layers; performing pooling processing on the first original feature sequence and the second original feature sequence to obtain pooled features of the target text; performing whitening processing on the pooled features to obtain sentence features of the target text, and outputting the sentence features of the target text. Based on the method proposed in the present application, the efficiency of text feature extraction can be improved, and the feature accuracy of text features can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text processing method, device, and computer-readable storage medium. Background Art

[0002] Text matching is one of the most fundamental and important tasks in natural language processing, with crucial applications in information retrieval, question-answering, and recommendation systems. Examples include determining whether essay content is off-topic in essay grading, determining the accuracy of answers in subjective questions, and removing duplicate questions and recommending similar questions on resource platforms.

[0003] The key to text matching lies in processing the text and extracting sentence features. However, most current methods rely on manual feature extraction, which is not only inefficient but also inaccurate. Summary of the Invention

[0004] Embodiments of the present invention provide a text processing method, apparatus, computer device, storage medium, and computer program product, which can improve the efficiency of text feature extraction and enhance the feature accuracy of text features.

[0005] An embodiment of the present invention provides a text processing method, including:

[0006] Obtain a target text, where the target text includes N phrases, where N is a positive integer greater than 1;

[0007] Calling a text feature extraction model to perform feature extraction processing on the N phrases to obtain a first original feature sequence and a second original feature sequence, wherein the text feature extraction model includes M hidden layers arranged in series, and the first original feature sequence and the second original feature sequence are output feature sequences of any two hidden layers in the M hidden layers, where M is an integer not less than 2;

[0008] Performing pooling processing on the first original feature sequence and the second original feature sequence to obtain pooled features of the target text;

[0009] The pooled features are whitened to obtain the sentence features of the target text, and the sentence features of the target text are output.

[0010] An embodiment of the present invention provides a text processing device, including:

[0011] An acquisition unit, configured to acquire a target text, wherein the target text includes N phrases, where N is a positive integer greater than 1;

[0012] a processing unit, configured to call a text feature extraction model to perform feature extraction processing on the N phrases to obtain a first original feature sequence and a second original feature sequence, wherein the text feature extraction model includes M hidden layers arranged in series, and the first original feature sequence and the second original feature sequence are output feature sequences of any two hidden layers in the M hidden layers, where M is an integer not less than 2;

[0013] The processing unit is further configured to perform pooling processing on the first original feature sequence and the second original feature sequence to obtain pooled features of the target text;

[0014] The processing unit is also used to whiten the pooled features to obtain the sentence features of the target text;

[0015] Output unit, used to output the sentence features of the target text.

[0016] An embodiment of the present invention provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the above-mentioned text processing method.

[0017] An embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is read and executed by a processor of a computer device, the computer device executes the above-mentioned text processing method.

[0018] An embodiment of the present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above-described text processing method.

[0019] The present invention is automatically executed by a computer device without the need for human intervention, and has high execution efficiency. Furthermore, the first original feature sequence and the second original feature sequence obtained by the text feature extraction model are the output feature sequences of the two hidden layers before and after the text feature extraction model. The sentence features extracted based on these two feature sequences can take into account both high-level semantic features and low-level semantic features. In addition, the whitening process can enhance the isotropy of the sentence features and reduce the dimension of the sentence features, thereby reducing the storage space of the computer device. Therefore, the method proposed by the present invention is not only efficient, but the obtained sentence features can also more accurately represent the semantics of the target text. When the sentence features are applied in text matching, the accuracy of text matching can also be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 This is a schematic diagram of the architecture of a text processing system provided by an embodiment of the present invention;

[0022] Figure 2 This is a flowchart of a text processing method provided by an embodiment of the present invention;

[0023] Figure 3 Schematic diagram of the structure of a text feature extraction model provided by an embodiment of the present invention;

[0024] Figure 4 This is a flowchart of another text processing method provided by an embodiment of the present invention;

[0025] Figure 5 is a schematic diagram of a model training provided by an embodiment of the present invention;

[0026] Figure 6 This is a flowchart of another text processing method provided by an embodiment of the present invention;

[0027] Figure 7 is a schematic diagram of an interactive interface provided by an embodiment of the present invention;

[0028] Figure 8a is a schematic diagram of another interactive interface provided by an embodiment of the present invention;

[0029] Figure 8b is a schematic diagram of another interactive interface provided by an embodiment of the present invention;

[0030] Figure 9 This is a flowchart of a text matching process provided by an embodiment of the present invention;

[0031] Figure 10 This is a schematic structural diagram of an information identification device provided by an embodiment of the present invention;

[0032] Figure 11 It is a structural diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first" and "second" in the embodiments of the present invention are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, technical features designated as "first" or "second" may explicitly or implicitly include at least one such feature.

[0035] First, some terms involved in the embodiments of the present invention are explained to facilitate understanding by those skilled in the art.

[0036] BERT model: The full name is Bidirectional Encoder Representations from Transformers. It is a new language model proposed by Google. It pre-trains bidirectional deep representations (Embedding) by jointly adjusting the bidirectional transformers (Transformers) in all layers.

[0037] Text matching: One of the most basic and important tasks in the field of natural language processing, it has important applications in information retrieval, question-answering systems, and recommendation systems. The main method is to first calculate the feature vector (i.e., sentence vector) that can represent the semantic information of the sentence, and then calculate the similarity of the two sentence vectors in the representation space (usually cosine similarity) to measure the semantic relevance of the two sentences, thereby determining the matching score. In the field of education, text matching methods have many application scenarios, such as judging whether the content of an essay is off-topic during essay correction, judging whether the answer is accurate during subjective question correction, deduplication of questions on resource platforms, and recommending similar questions.

[0038] In order to enable the extracted sentence features to more accurately represent the semantics of the target text, an embodiment of the present invention proposes a text processing method.

[0039] The text processing method proposed in the embodiments of the present invention can be implemented based on artificial intelligence technology. Artificial intelligence (AI) technology is a comprehensive discipline covering a wide range of fields, encompassing both hardware and software technologies. Basic AI technologies generally include sensors, dedicated AI chips, cloud computing, cloud storage, big data processing technologies, operating / interactive systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0040] Natural language processing (NLP) is a key area of ​​research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to the study of linguistics. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0041] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields. The embodiments of the present invention may specifically involve technologies such as natural language processing technology in artificial intelligence technology when implementing the data processing method.

[0042] The method proposed in this application belongs to the field of text processing in natural language processing technology. By processing text data, a numerical vector that can represent the semantics of the text data is obtained. Text processing can convert semantic-related problems into mathematical problems for solution, and is usually used in scenarios such as text matching and text translation. For example, when a teacher corrects students' homework, he or she needs to compare the student's answer with the correct answer every time, which is time-consuming and laborious. However, by converting the answers and correct answers of two students into two numerical vectors, it is only necessary to calculate the similarity of the two numerical vectors to quickly determine whether the student's answer is correct, saving time and effort. For example, when a student does not understand the meaning of an English word when reading English literature, it takes a long time to look up the meaning of the word in a dictionary. If the English word is converted into a numerical vector, and then the numerical vector is converted into an expression that the student can understand, such as Chinese, this method can improve the efficiency of students in reading English literature.

[0043] See Figure 1 , Figure 1 FIG. 1 is a schematic diagram of the architecture of a text processing system according to an embodiment of the present invention. Figure 1As shown, the text processing system 100 may include multiple terminal devices 101 and a server 102. Of course, the text processing system 100 may also include one or more terminal devices 101 and multiple servers 102, which is not limited in this embodiment of the present invention. The terminal device 101 is primarily used to send a target text to the server 102 and receive one or more target texts sent by the server 102; the server 102 is primarily used to execute the relevant steps of the text processing method, obtain, and output sentence features corresponding to the target text. The terminal device 101 and the server 102 can achieve a communication connection, and the connection method may include a wired connection or a wireless connection, which is not limited here.

[0044] In one possible implementation, the terminal device 101 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart car, etc., but is not limited thereto; the server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0045] In combination with the above-mentioned text processing system, the text processing method of the embodiment of the present invention may generally include: obtaining the target text sent by the terminal device 101, the target text includes N phrases, N is a positive integer greater than 1; the server 102 calls the text feature extraction model to perform feature extraction processing on the N phrases to obtain a first original feature sequence and a second original feature sequence, wherein the text feature extraction model includes M hidden layers arranged in series, the first original feature sequence and the second original feature sequence are output feature sequences of any two hidden layers in the M hidden layers, and M is an integer not less than 2; then, the server 102 performs pooling processing on the first original feature sequence and the second original feature sequence to obtain pooled features of the target text; finally, the server 102 performs whitening processing on the pooled features to obtain sentence features of the target text, and outputs the sentence features of the target text. The sentence features obtained by this method can more accurately represent the semantics of the target text, and when the sentence features are applied in text matching, the accuracy of text matching can be improved.

[0046] It can be understood that the system architecture diagram described in the embodiment of the present invention is for the purpose of more clearly illustrating the technical solution of the embodiment of the present invention, and does not constitute a limitation on the technical solution provided by the embodiment of the present invention. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided by the embodiment of the present invention is also applicable to similar technical problems.

[0047] Based on the above description of the text processing system architecture, the present invention discloses a text processing method. Figure 2 , is a flowchart of a text processing method disclosed in an embodiment of the present invention. The text processing method can be executed by a computer device, which can specifically be the server 102 in the text processing system. The text processing method can specifically include steps S201 to S204:

[0048] S201. Obtain a target text, where the target text includes N phrases, where N is a positive integer greater than 1.

[0049] In the embodiment of the present invention, the target text may be a plurality of phrases, a sentence or an article, for example, the target text is "I am a student". Figure 1 The text data is sent by the terminal device 101 of the text processing system, or the text data stored by the computer device. The process of the computer device obtaining the target text can be: the computer device receives the audio data sent by the terminal device; then, the computer device converts the audio data into text data through speech recognition technology (Automatic Speech Recognition, ASR); finally, the computer device removes special characters from the text data to obtain the target text. For example, when the terminal device collects the user's audio data through a recorder, the terminal device sends the audio data to the computer device, and the computer device performs speech recognition on the audio data to generate text data, which is the target text.

[0050] S202. Call a text feature extraction model to perform feature extraction processing on the N phrases to obtain a first original feature sequence and a second original feature sequence, wherein the text feature extraction model includes M hidden layers arranged in series, and the first original feature sequence and the second original feature sequence are output feature sequences of any two hidden layers among the M hidden layers, where M is an integer not less than 2.

[0051] In a possible implementation, the text feature extraction model may be a Bert model. The main function of the Bert model is to represent the semantics of natural language through numerical vectors. The structure of the Bert model is as follows: Figure 3 As shown. It is divided into two parts: encoder and decoder, where Figure 3It only includes 2 encoders and 2 decoders. In actual applications, there can be multiple encoders and decoders, and the embodiment of the present invention does not limit this. Among them, the encoder includes a self-attention layer, a normalization layer, and a feed forward network. When processing the input sequence, the self-attention layer will combine the sequences corresponding to other phrases contained in the text corresponding to the current input sequence to improve the correlation between the various phrases in the text. For example, the input sentence is "I am a student", and the phrases corresponding to this sentence include "I", "I am", "one" and "student". When the self-attention layer processes the sequence corresponding to the phrase "student", it will combine the sequences corresponding to "I", "I am" and "one" for processing. The feed forward network is a feed forward neural network (FNN), which is composed of multiple layers of perceptrons (discontinuous nonlinear functions) and is used to process the input sequence. The feed forward network corresponding to each input position is independent and does not interfere with each other. The normalization layer adds a residual transform to the sequence after processing it in the feedforward network or self-attention layer, which can better optimize deep networks. After the encoder completes processing, the resulting encoded sequence is input to the decoder. The decoder structure includes a normalization layer, an encoder-decoder attention layer, and a feedforward network. The normalization layer and forward network in the decoder perform the same functions as those in the encoder. The encoder-decoder attention layer selects a small amount of important data from the large amount of input and output data, focusing on this important data when processing the sequence. Compared to the attention layer, the self-attention layer reduces its reliance on external information and is better at capturing internal correlations in data or features. The positional encoding in the model indicates the position of the corresponding word vector, enabling the BERT model to distinguish between word groups at different positions in a sentence. The linear layer and softmax layer convert the decoder's final output vector into words.

[0052] In addition, the text feature extraction model can also be the RoBERTa (A Robustly Optimized BERTPretraining Approach) model and the shuffle model (xlnet) model. Compared to the Bert model, RoBERTa uses more data training models, removes the next sentence prediction (NSP) objective function, and uses a longer sequence length (max_seq_length) for training. Compared to the Bert model, the xlnet model uses an autoregressive (AR) model instead of an autoencoding (AE) model to address the negative effects of masks. It also uses a dual attention flow mechanism and introduces relative position encoding and fragment recurrence mechanisms.

[0053] Optionally, the first original feature sequence is the output feature sequence of the first hidden layer in the M hidden layers, and the second original feature sequence is the output feature sequence of the last hidden layer in the M hidden layers. The text feature extraction model only applies the encoder part of the Bert model, and one encoder is one hidden layer. The text feature extraction model contains M encoders, and the first original feature sequence is the feature sequence output by the first encoder in the Bert model, for example Figure 3 The feature sequence output by encoder 1 in , the second original feature sequence is the feature sequence output by the last encoder in the Bert model, for example Figure 3 Extracting the output feature sequence of the first hidden layer among the M hidden layers and the output feature sequence of the last hidden layer among the M hidden layers helps the computer device take into account both high-level semantic features and low-level semantic features in subsequent processing.

[0054] S203 : Perform pooling processing on the first original feature sequence and the second original feature sequence to obtain pooled features of the target text.

[0055] In this embodiment of the present invention, the first and second original feature sequences are matrices of N*X dimensions, where the size of X is related to the text feature extraction model. For example, if the text feature extraction model is the Bert-base model, X is 768, and the first and second original feature sequences are matrices of N*768 dimensions. Pooling can compress feature sequences and extract their primary features, thereby reducing their dimensionality. Pooling can also reduce the spatial size of data, thereby reducing the amount of data processing computation and memory usage.

[0056] In one possible implementation, the pooling process is specifically average pooling, which is to average all values ​​in the local receptive field. Average pooling is conducive to preserving the overall characteristics of the data. For example, assuming that the matrix Perform average pooling, and the size of the pooled local receptive field is a 1×2 window, that is, (1, 3) belongs to one local receptive field, and (5, 7) belongs to another local receptive field. The matrix after pooling is [2 6]. In addition, the pooling process can also be a maximum pooling process or a random pooling process, which is not limited in this application.

[0057] The pooled feature obtained after the average pooling of the first original feature sequence and the second original feature sequence is a 1*X dimensional vector. That is, the average value of each column in the first original feature sequence and the second original feature sequence is obtained to obtain two 1*X dimensional vectors, and then the vectors corresponding to the first original feature sequence and the second original feature sequence are added together to obtain the average value. For example, the first original feature sequence is a matrix The second original feature sequence is the matrix The first and second original feature sequences are subjected to average pooling, and the vector corresponding to the first original feature sequence is obtained as

[35] , and the vector corresponding to the second original feature sequence is obtained as

[53] . The two are then added together to obtain the average, and the final pooled feature is obtained as

[44] . Pooling the first and second original feature sequences can take into account both high-level semantic features and low-level semantic features from the overall data.

[0058] S204: Perform whitening processing on the pooled features to obtain sentence features of the target text, and output the sentence features of the target text.

[0059] In an embodiment of the present invention, a computer device performs whitening processing on the pooled features to obtain the sentence features of the target text in the following specific implementation manner: the computer device performs principal component analysis (PCA) processing on the pooled features to generate a feature vector of the pooled features; the computer device performs standard deviation normalization processing on the feature vector of the pooled features to obtain the sentence features of the target text. PCA processing refers to converting the pooled features into linearly uncorrelated feature vectors through orthogonal transformation, and performing standard deviation normalization processing on the feature vector to obtain a sentence feature with a mean of 0 and a covariance of the unit matrix. At the same time, the sentence feature is also a vector with a dimension of 1*Y, where Y is less than X. Whitening processing can remove redundant information of the input data, and can also optimize memory storage and speed up processing.

[0060] See Figure 4, is a flow chart of another text processing method disclosed in an embodiment of the present invention. This text processing method can be executed by a computer device, specifically server 102 in a text processing system. This embodiment primarily illustrates the training process of a text feature extraction model. The text processing method may specifically include steps S401 to S409. Among them:

[0061] S401: Obtain sample text, perform data enhancement processing on the sample text, and obtain an enhanced feature sequence.

[0062] In the embodiment of the present invention, the sample text includes A phrases, where A is a positive integer. The sample text is a sentence or an article, etc., and is used to train the text feature extraction model.

[0063] In one possible implementation, a computer device performs data augmentation processing on a sample text to obtain an enhanced feature sequence, including: the computer device converts A phrases into A word vectors, into A position vectors, and into A sentence vectors; the computer device randomly selects a target vector from the A word vectors, the A position vector positions, and the A sentence vectors, and adjusts the value of the target vector; the computer device splices the positions of the phrases corresponding to the unselected vectors and the adjusted target vectors in the sample text into an enhanced feature sequence.

[0064] Among them, the word vector represents the semantics corresponding to the phrase, the position vector represents the position corresponding to the phrase in the sample text, and the clause vector represents the position of the sentence in which the phrase is located in the sample text. The training batch in which the sample text is located contains multiple sample texts, and each sample text corresponds to a number. For example, a training batch contains two sample texts (including the first sample text and the second sample text). If the clause vectors corresponding to a phrase are all [1], it can be inferred that the phrase is the first clause in the first sample text. If the position vector of the phrase is [4], it can be inferred that the phrase is the fourth phrase in the first clause of the sample text.

[0065] It should be noted that the specific implementation method of randomly selecting a target vector can be: randomly selecting multiple target vectors, wherein the multiple target vectors are any one of word vectors, position vectors or sentence vectors, that is, selecting from the perspective of features; or randomly selecting one or more target phrases from the L phrases included in the sample text, and using the word vectors, position vectors and sentence vectors corresponding to the target phrases as target vectors, that is, selecting from the perspective of tokens; or randomly selecting multiple vectors from A word vectors, A position vectors and A sentence vectors as target vectors. The specific implementation method of adjusting the value of the target vector can be: setting all the values ​​of the target vectors to preset values, such as 0; or randomly increasing or decreasing a certain value based on the value of the target vector. Since the target vector is selected based on a random method and the value of the target vector is adjusted, the semantics expressed by the enhanced feature sequence is the same or similar to the semantics expressed by the sample text.

[0066] It should also be noted that the following practical example introduces how to concatenate the A word vectors, A position vectors, and A sentence vectors corresponding to A phrases into a sequence: The A phrases in the sample text can be represented as: W1, W2, W3, ..., W A , a phrase W i Corresponding to a word vector W i1 , a position vector W i2 and a clause vector W i3 , i is an integer between [1, A]. Combine the A word vectors, A position vectors and A clause vectors corresponding to the A phrases to get the sequence: If the target vector is randomly selected as W 11 、W 12 、W 13 , adjust W 11 、W 12 、W 13 The value of the adjusted target vector is represented as W1'1, W1'2, and W1'3. The positions of the phrases corresponding to the unselected vectors and the adjusted target vectors in the sample text are spliced ​​into an enhanced feature sequence.

[0067] Optionally, the number of enhanced feature sequences is K, where K is an integer greater than 1. The K enhanced feature sequences are generated by randomly selecting K times. Since the randomly selected target vectors are different, the K enhanced feature sequences are different from each other. However, the K enhanced feature sequences are all obtained from the same sample text through data augmentation processing, and therefore represent the same or similar semantics.

[0068] In one possible implementation, the data augmentation processing method may also be a back-translation method, a CBERT method, or an adversarial attack method. The back-translation method involves a computer device using a machine translation model to translate a sample text into another language and then translate it back. The CBERT method involves a computer device randomly replacing some words in the sample text with a mask according to a certain probability, and then using the BERT model to predict the word corresponding to the mask. The adversarial attack method involves a computer device replacing some words in the sample text with synonyms. The embodiments of the present invention do not limit the specific data augmentation method employed.

[0069] S402 : Calling a text feature extraction model to perform feature extraction processing on the enhanced feature sequence to obtain a first feature sequence set and a second feature sequence set.

[0070] In the embodiment of the present invention, the specific implementation of step S402 is as follows: first, the text feature extraction model is called to perform feature extraction processing on the enhanced feature sequence, and a portion of parameters are randomly selected for masking according to a preset dropout rate to obtain a first feature sequence set E1, wherein the dropout rate refers to the ratio of the selected masked parameters to the total number of parameters in the entire model; then, the text feature extraction model is called again to perform feature extraction processing on the enhanced feature sequence, and a portion of parameters are randomly selected for masking according to the dropout rate to obtain a second feature sequence set E2. The first feature sequence set E2 includes a first sample feature sequence E11 and a second sample feature sequence E12, and the second feature sequence set E2 includes a third sample feature sequence E21 and a fourth sample feature sequence E22. The first sample feature sequence E11 and the third sample feature sequence E21 are both output feature sequences of the first hidden layer in the text feature extraction model, and the second sample feature sequence E12 and the fourth sample feature sequence E22 are both output feature sequences of the last hidden layer in the text feature extraction model. Because the parameters of the text feature extraction model corresponding to the first feature sequence set E1 are different from the parameters of the text feature extraction model corresponding to the second feature sequence set E2. Therefore, the first feature sequence set E1 and the second feature sequence set E2 are different, but since the first feature sequence set E1 and the second feature sequence set E2 are from the same sample text, the semantics represented by them are also the same or similar.

[0071] The specific structure of the text feature extraction model can be found in the description of step S202, and will not be elaborated herein in detail in this embodiment of the present invention.

[0072] S403 : Perform pooling processing and whitening processing on the first sample feature sequence and the second sample feature sequence to obtain a first sample sentence feature of the sample text.

[0073] S404 : Perform pooling processing and whitening processing on the third sample feature sequence and the fourth sample feature sequence to obtain a second sample sentence feature of the sample text.

[0074] Step S403 and step S404 can be performed simultaneously, or step S404 can be performed first and then step S403. The present embodiment does not limit the order of performing steps S403 and S404. The specific implementation of the pooling process can be referred to step S203, and the specific implementation of the whitening process can be referred to step S204. The present embodiment will not be described in detail here.

[0075] S405 : Combining the first sample sentence feature and the second sample sentence feature into a positive sample pair, and training a text feature extraction model based on the positive sample pair.

[0076] In an embodiment of the present invention, since the first sample sentence feature e1 and the second sample sentence feature e2 both come from the sample text, when the text feature model is called to perform text feature processing, when the feature enhancement feature sequence is subjected to feature extraction processing, the random text feature extraction model called will randomly mask some parameters, resulting in the first sample sentence feature and the second sample sentence feature being not completely identical. However, the first sample sentence feature e1 and the second sample sentence feature e2 come from the same sample text and represent the same or similar semantics. Therefore, the computer device can combine the first sample sentence feature e1 and the second sample sentence feature e2 into a positive sample pair to train the text feature extraction model. This method is conducive to enabling the feature sequence generated by the trained text feature extraction model when processing text data to more accurately represent the semantics of the text.

[0077] In one possible implementation, training a text feature extraction model based on positive sample pairs includes: obtaining sample sentence features to be combined, where the sample sentence features to be combined are the first sample sentence features d1 or the second sample sentence features d2 of the remaining sample texts in the same training batch as the sample text; combining the sample sentence features of the sample text with the sample sentence features to be combined to form negative sample pairs; and training the text feature extraction model based on the positive and negative sample pairs, where the sample sentence features of the sample text are the first sample sentence features or the second sample sentence features of the sample text. Because the two sentence features in the positive sample pair come from the same sample text, the semantics expressed by the sentence features in the positive sample pair are identical or similar; whereas, the two sentence features in the negative sample pair come from different sample texts and therefore express different semantics. In the final sentence vector space, the sentence vectors corresponding to two semantically similar sentences should be as close as possible, while the sentence vectors corresponding to two semantically different sentences should be as far apart as possible. A computer device then trains the text feature extraction model based on these features. Training the text feature extraction model based on the positive and negative sample pairs facilitates the feature sequence generated by the text feature extraction model when processing a target text to more accurately represent the semantics of the target text.

[0078] For example, in a training batch, two training texts are included (a first training text A and a second training text B). The first sample sentence feature A1 of the first training text A can form a positive sample pair with the second sample sentence feature A2 of the first training text, and the first sample sentence feature A1 of the first training text A can form a negative sample pair with the first sample sentence feature B1 or the second sample sentence feature B2 of the second training text B.

[0079] Optionally, a text feature extraction model is trained based on the positive sample pairs and the negative sample pairs, including: calculating the loss values ​​corresponding to the positive sample pairs and the negative samples through a loss function; and adjusting the parameters of the text matching model based on the loss values. Assume that the sample sentence feature set obtained from the unified training batch is Among them, h i is the sample sentence feature, h i and Corresponding to the same sample text, forming a positive sample pair, h i and They correspond to different sample texts and form negative sample pairs. The loss function is l i It can be expressed as:

[0080]

[0081] Where τ is a hyperparameter, sim(h1,h2) represents the cosine similarity of h1 and h2, and the calculation method is The computer equipment continuously trains the text feature extraction model and adjusts the parameters of the training text feature extraction model so that the loss value l corresponding to the positive sample pair and the negative sample pair is calculated through the loss function. i Based on this method, it is helpful to enable the feature sequence generated by the text feature extraction model when processing the target text to more accurately represent the semantics of the target text.

[0082] Below through Figure 5 The text feature extraction model training shown in FIG. 4 describes the model training process corresponding to steps S401 to S405. Figure 5 As shown, first, the computer device obtains X sample texts, where the X texts include sample text 1, sample text 2, sample text 3...sample text X; then, the computer device performs data enhancement processing on the sample texts to obtain enhanced feature sequences, where one sample text corresponds to two enhanced feature sequences (including enhanced feature sequence 1 and enhanced feature sequence 2); then, the computer device calls a text feature extraction model to extract features from the enhanced feature sequences, and obtains a first feature sequence set (including a first sample feature sequence and a second sample feature sequence) and a second feature sequence set (including a third sample feature sequence and a fourth sample feature sequence). Figure 5 In the example, the feature sequence sets corresponding to the enhanced feature sequence 1 are the first feature sequence set 1 and the second feature sequence set 1, and the feature sequence sets corresponding to the enhanced feature sequence 2 are the first feature sequence set 2 and the second feature sequence set 2; then, the computer device performs pooling processing and whitening processing on the first sample feature sequence and the second sample feature sequence to obtain sentence features of the sample text, and performs pooling processing and whitening processing on the third sample feature sequence and the fourth sample feature sequence to obtain sentence features of the sample text. Figure 5 In the example, the first feature sequence set 1 corresponds to sentence feature 1, the second feature sequence set 1 corresponds to sentence feature 2, the first feature sequence set 1 corresponds to sentence feature 1, and the second feature sequence set 1 corresponds to sentence feature 2; finally, the two sentence features corresponding to the same sample text are taken as a positive sample pair (for example, sentence feature 1 corresponding to sample text 1 and sentence feature 2 corresponding to sample text 1), and the two sentence features corresponding to two different sample texts are taken as a negative sample pair (for example, sentence feature 1 corresponding to sample text 1 and sentence feature 1 corresponding to sample text 2), and the text feature extraction model is trained based on the positive sample pairs and negative sample pairs.

[0083] S406. Obtain a target text, where the target text includes N phrases, where N is a positive integer greater than 1; call a text feature extraction model to perform feature extraction processing on the N phrases to obtain a first original feature sequence and a second original feature sequence, wherein the text feature extraction model includes M hidden layers arranged in series, and the first original feature sequence and the second original feature sequence are output feature sequences of any two hidden layers among the M hidden layers, where M is an integer not less than 2.

[0084] S407 , performing pooling processing on the first original feature sequence and the second original feature sequence to obtain pooled features of the target text; performing whitening processing on the pooled features to obtain sentence features of the target text, and outputting the sentence features of the target text.

[0085] Among them, the specific implementation of step S406 and step S407 is the same as that of the above steps S201 to S204, and is not described in detail in this embodiment of the present invention.

[0086] See Figure 6 , which is a flowchart of another text processing method disclosed in an embodiment of the present invention. This embodiment primarily illustrates the application of the text processing method. The text processing method can be executed by a computer device, specifically server 102 in a text processing system. The text processing method specifically includes steps S601 to S609. Figure 6 The corresponding embodiments can be Figure 4 Corresponding embodiments can be combined with each other.

[0087] S601. Obtain a target text, where the target text includes N phrases, where N is a positive integer greater than 1; call a text feature extraction model to perform feature extraction on the N phrases to obtain a first original feature sequence and a second original feature sequence, wherein the text feature extraction model includes M hidden layers arranged in series, and the first original feature sequence and the second original feature sequence are output feature sequences of any two hidden layers among the M hidden layers, where M is an integer not less than 2.

[0088] S602: Perform pooling processing on the first original feature sequence and the second original feature sequence to obtain pooled features of the target text; perform whitening processing on the pooled features to obtain sentence features of the target text, and output the sentence features of the target text.

[0089] Among them, the specific implementation of step S601 and step S602 is the same as that of step S201 to step S204, and is not described in detail in this embodiment of the present invention.

[0090] S603: Obtain a preset text, where the preset text includes L phrases, where L is a positive integer greater than 1.

[0091] In the embodiment of the present invention, the preset text may be a plurality of phrases, a sentence or an article, for example, the preset text is "I am a student". Figure 1 The text data is sent by the terminal device 101 of the text processing system in the text processing system, or the text data stored by the computer device. The process of obtaining the preset text can be the preset text obtained by converting the audio data through ASR, or the preset text can be obtained by removing special characters from the text data, which is not limited in the embodiment of the present invention. For example, when the terminal device collects the user's audio data through a recorder, the terminal device sends the audio data to the computer device, and the computer device performs speech recognition on the audio data to generate text data, which is the preset text.

[0092] S604. Call the text feature extraction model to perform feature extraction processing on the L phrases to obtain a first preset feature sequence and a second preset feature sequence. The first preset feature sequence is the output feature sequence of the first hidden layer in the text feature extraction model, and the second preset feature sequence is the output feature sequence of the last hidden layer in the text feature extraction model.

[0093] The specific structure of the text feature extraction model can be found in the description of step S202, and will not be elaborated herein in detail in this embodiment of the present invention.

[0094] S605 : Perform pooling processing and whitening processing on the first preset feature sequence and the second preset feature sequence to obtain sentence features corresponding to the preset text.

[0095] The specific implementation of the pooling process can refer to the description in step S203 , and the specific implementation of the whitening process can refer to the description in step S204 , which will not be described in detail in this embodiment of the present invention.

[0096] Among them, steps S601 to S605 only need to be executed before step S606. For example, steps S603 to S605 can be executed first, and then steps S601 and S602; or step S603 can be executed first, then steps S601 and S602, and then steps S604 and S605. The embodiment of the present invention does not limit the order of executing steps S601 to S605.

[0097] S606: Calculate the similarity between the sentence features corresponding to the preset text and the sentence features corresponding to the target text.

[0098] In one possible implementation, the similarity between the sentence features corresponding to the preset text and the sentence features corresponding to the target text can be expressed by the following formula:

[0099]

[0100] Among them, x represents the sentence features corresponding to the preset text, y represents the sentence features corresponding to the target text, and sim(x,y) represents the similarity between the sentence features corresponding to the preset text and the sentence features corresponding to the target text.

[0101] S607: Determine the matching result between the preset text and the target text according to the similarity.

[0102] In one possible implementation, the matching result includes matching text and non-matching text. When the similarity is greater than a preset threshold, the matching result is matching text; when the similarity is less than or equal to the preset threshold, the matching result is non-matching text. Alternatively, the matching result includes matching text, partially non-matching text, and completely non-matching text. When the similarity is greater than or equal to a first threshold, the matching result may be matching text; when the similarity is less than or equal to a second threshold, the matching result is completely non-matching text; when the similarity is less than or equal to the first threshold and greater than the second threshold, the matching result is partially non-matching text.

[0103] Optionally, the method further includes: if the similarity determines that the matching result between the preset text and the target text is an unmatched text, extracting the keyword group corresponding to the target text from the N phrases included in the target text; extracting the keyword group corresponding to the preset text from the L phrases included in the preset text; determining the reason for the mismatch based on the keyword group corresponding to the target text and the keyword group corresponding to the preset text, and outputting the reason for the mismatch. Among them, the reasons for the mismatch include missing keywords, wrong keywords, and incomplete answers. Among them, the keywords can be phrases of important parts of speech, such as verbs, nouns, quantifiers, adjectives, or adverbs. Based on this implementation method, it is helpful for users to understand the reasons for text mismatch and improve user experience.

[0104] For example, Figure 7As shown, the teacher pre-sets the question "How many red balls and white balls did Xiao Ming give Xiao Hong?" on the server platform and uploads the correct answer, which is the preset text "Xiao Ming gave Xiao Hong five red balls and eight white balls." When students log in to the platform through their terminal devices and answer the teacher's questions, they enter the target text "Xiao Ming gave Xiao Hong five red balls and four white balls." The server calls the text feature extraction model to process the target text and the preset text, obtaining a feature sequence corresponding to the target text and a feature sequence corresponding to the preset text. The server then performs pooling and whitening on the feature sequence corresponding to the target text and the feature sequence corresponding to the preset text to obtain the sentence features corresponding to the target text and the sentence features corresponding to the preset text. The server then calculates the similarity between the sentence features corresponding to the target text and the sentence features of the preset text. If the similarity is below the preset threshold, the match result is determined to be a mismatch. The server extracts the keywords "five red balls" and "eight white balls" in the preset text and the keywords "five red balls" and "four white balls" in the target text. After comparison, it determines that "eight white balls" and "four white balls" do not match. The server will then send the reason for the mismatch to the student's terminal device, and the student can learn the reason why their answer is wrong through the mismatch reason. If the target text entered by the student is "Xiao Ming gave Xiao Hong five red balls", when the server compares the keywords of the preset text and the target text, it determines that the target text is missing the keyword "eight white balls". At this time, the server outputs the reason for the mismatch as missing keywords. If the target text entered by the student is "Xiao Ming gave Xiao Hong five red balls and eight white balls" and it is the same as the preset text, the server calculates the sentence features corresponding to the target text and the sentence features of the preset text, and determines that the similarity between the sentence features corresponding to the target text and the sentence features of the preset text is higher than the preset threshold. The matching result output by the server is the matching text.

[0105] For example, the present application provides an educational platform. Figure 8a The following is a schematic diagram of the interactive interface of the platform, which includes functions such as picture description, theme composition, topic retelling and oral reading comprehension. The main function of picture description is: given a cartoon, the answerer is required to have a conversation based on the scene described in the cartoon. The main function of theme composition is: to write an oral composition around a given topic. The main function of topic retelling is: to play a recording, and require the answerer to retell the content of the recording in his own words after listening to the recording. The main function of oral reading comprehension is: after listening to a passage, the user orally answers the specified question. Students can choose a project, collect their own voice as audio data through the recorder of the terminal device, and transmit it to the platform. The platform can convert the audio uploaded by students into target text, convert the target text and the preset text corresponding to the correct answer into sentence features, and score the students' answers based on the similarity of the two sentence features. For example Figure 8bIn the example, the audio data input by the student was converted into "3000years", and the reference answer was "For more than4000years". The answer obtained by the platform after processing was "Error. Error type: Corresponding answer, but wrong answer", which means that the student's answer was a response to the question, but the answer did not match the correct answer.

[0106] Taking the education platform as an example, this paper introduces the overall process of the server processing text data. Figure 9 The corresponding process diagram is shown below. The teacher sets a speaking question on the education platform using a terminal device and submits the correct answer text S2 to the server. Students use their terminal devices to capture their own speech on the education platform as a voice file and send it to the server. The server uses ASR to convert this voice file into the student's answer S1. The server processes S1 and S2 to obtain the sentence vector h1 for S1 and the sentence vector h2 for S2. The server calculates the similarity sim between h1 and h2. If sim is greater than or equal to a first threshold, the match is considered correct. The first threshold is set to 0.98. If sim is less than or equal to a second threshold, the match is considered completely incorrect. The second threshold is set to 0.5. If sim is less than the first threshold but greater than the second threshold, the match is considered incorrect. The server then extracts keywords from S1 and S2 and analyzes the cause of the error through part-of-speech analysis, syntactic analysis, and completeness calculation. The server then sends the matching result to the student's corresponding terminal device.

[0107] Based on the above text processing method, an embodiment of the present invention provides a text processing device. Figure 10 , is a schematic diagram of the structure of a text processing device provided by an embodiment of the present invention. The text processing device 1000 can run the following units:

[0108] An acquisition unit 1001 is configured to acquire a target text, where the target text includes N phrases, where N is a positive integer greater than 1.

[0109] Processing unit 1002 is configured to call a text feature extraction model to perform feature extraction processing on the N phrases to obtain a first original feature sequence and a second original feature sequence, wherein the text feature extraction model includes M hidden layers arranged in series, and the first original feature sequence and the second original feature sequence are output feature sequences of any two hidden layers in the M hidden layers, where M is an integer not less than 2;

[0110] The processing unit 1002 is further configured to perform pooling processing on the first original feature sequence and the second original feature sequence to obtain pooled features of the target text;

[0111] The processing unit 1002 is further configured to perform whitening processing on the pooled features to obtain sentence features of the target text;

[0112] The output unit 1003 is used to output the sentence features of the target text.

[0113] In one embodiment, the first original feature sequence is the output feature sequence of the first hidden layer among the M hidden layers, and the second original feature sequence is the output feature sequence of the last hidden layer among the M hidden layers.

[0114] In one embodiment, the acquisition unit 1001 is further used to acquire sample text, perform data enhancement processing on the sample text, and obtain an enhanced feature sequence; the processing unit 1002 is further used to call the text feature extraction model to perform feature extraction processing on the enhanced feature sequence, and obtain a first feature sequence set and a second feature sequence set; wherein the first feature sequence set and the second feature sequence set are feature sequence sets extracted respectively after randomly shielding some parameters of the text feature extraction model, the first feature sequence set includes the first sample feature sequence and the second sample feature sequence, the second feature sequence set includes the third sample feature sequence and the fourth sample feature sequence, and the first sample feature sequence and the third sample feature sequence are both text feature sequences. The output feature sequence of the first hidden layer in this feature extraction model, the second sample feature sequence and the fourth sample feature sequence are all output feature sequences of the last hidden layer in the text feature extraction model; the processing unit 1002 is also used to perform pooling processing and whitening processing on the first sample feature sequence and the second sample feature sequence to obtain the first sample sentence feature of the sample text; the processing unit 1002 is also used to perform pooling processing and whitening processing on the third sample feature sequence and the fourth sample feature sequence to obtain the second sample sentence feature of the sample text; the processing unit 1002 is also used to combine the first sample sentence feature and the second sample sentence feature into a positive sample pair, and train the text feature extraction model based on the positive sample pair.

[0115] In one embodiment, when training a text feature extraction model based on a positive sample pair, the processing unit 1002 is specifically used to: obtain the sample sentence features to be combined, where the sample sentence features to be combined are the first sample sentence features or the second sample sentence features of the remaining sample texts in the same training batch as the sample text; combine the sample sentence features of the sample text with the sample sentence features to be combined into a negative sample pair, and train the text feature extraction model based on the positive sample pair and the negative sample pair, where the sample sentence features of the sample text are the first sample sentence features or the second sample sentence features of the sample text.

[0116] In one embodiment, the sample text includes A phrases, where A is a positive integer. The processing unit 1002 performs data enhancement processing on the sample text to obtain an enhanced feature sequence, which is specifically used to: convert the A phrases into A word vectors, A position vectors, and A sentence vectors; randomly select a target vector from the A word vectors, A position vector positions, and A sentence vectors, and adjust the value of the target vector to a preset vector; and splice the positions of the phrases corresponding to the unselected vectors and the adjusted target vectors in the sample text into an enhanced feature sequence.

[0117] In one embodiment, the number of enhanced feature sequences is K, where K is an integer greater than 1, and the K enhanced feature sequences are enhanced feature sequences generated after random selection K times.

[0118] In one embodiment, the processing unit 1002 performs whitening processing on the pooled features to obtain the sentence features of the target text, which is specifically used to: perform principal component analysis processing on the pooled features to generate a feature vector of the pooled features; and perform standard deviation normalization processing on the feature vector of the pooled features to obtain the sentence features of the target text.

[0119] In one embodiment, the acquisition unit 1001 is also used to acquire a preset text, which includes L phrases, where L is a positive integer greater than 1; the processing unit 1002 is also used to call a text feature extraction model to perform feature extraction processing on the L phrases to obtain a first preset feature sequence and a second preset feature sequence, where the first preset feature sequence is the output feature sequence of the first hidden layer in the text feature extraction model, and the second preset feature sequence is the output feature sequence of the last hidden layer in the text feature extraction model; the processing unit 1002 is also used to perform pooling processing and whitening processing on the first preset feature sequence and the second preset feature sequence to obtain sentence features corresponding to the preset text; the processing unit 1002 is also used to calculate the similarity between the sentence features corresponding to the preset text and the sentence features corresponding to the target text; the processing unit 1002 is also used to determine the matching result of the preset text and the target text based on the similarity.

[0120] In one embodiment, the processing unit 1002 is further used to extract the keyword group corresponding to the target text from the N phrases included in the target text if the similarity determines that the matching result between the preset text and the target text is an unmatched text; the processing unit 1002 is further used to extract the keyword group corresponding to the preset text from the L phrases included in the preset text; the processing unit 1002 is further used to determine the reason for the mismatch based on the keyword group corresponding to the target text and the keyword group corresponding to the preset text; the output unit 1003 is further used to output the reason for the mismatch.

[0121] Further, see Figure 11, is a structural diagram of a computer device provided in an embodiment of the present application. Figure 2 、 Figure 4 and Figure 6 The computer device in the corresponding embodiment may be the computer device 1100. Figure 11 As shown, the computer device 1100 may include: a user interface 1102, a processor 1104, an encoder 1106, and a memory 1108. A signal receiver 1116 is used to receive or send data via a cellular interface 1111, a WIFI interface 1112, ..., or an NFC interface 1114. The encoder 1106 encodes the received data into a data format for computer processing. A computer program is stored in the memory 1108, and the processor 1104 is configured to execute the steps of any of the above method embodiments through the computer program. The memory 1108 may include a volatile memory (e.g., a dynamic random access memory DRAM) and may also include a non-volatile memory (e.g., a one-time programmable read-only memory OTPROM). In some instances, the memory 1108 may further include a memory remotely located relative to the processor 1104, and these remote memories may be connected to the computer device 1100 via a network. The user interface 1102 may include: a keyboard 1118 and a display 1120.

[0122] exist Figure 11 In the computer device 1100 shown, the processor 1104 can be used to call the computer program stored in the memory 1108 to implement:

[0123] Obtain a target text, where the target text includes N phrases, where N is a positive integer greater than 1; call a text feature extraction model to perform feature extraction processing on the N phrases to obtain a first original feature sequence and a second original feature sequence, wherein the text feature extraction model includes M hidden layers arranged in series, and the first original feature sequence and the second original feature sequence are output feature sequences of any two hidden layers in the M hidden layers, where M is an integer not less than 2; perform pooling processing on the first original feature sequence and the second original feature sequence to obtain pooled features of the target text; perform whitening processing on the pooled features to obtain sentence features of the target text, and output the sentence features of the target text.

[0124] In one embodiment, the first original feature sequence is the output feature sequence of the first hidden layer among the M hidden layers, and the second original feature sequence is the output feature sequence of the last hidden layer among the M hidden layers.

[0125] In one embodiment, the processor 1104 further performs the following steps: obtaining a sample text, performing data enhancement processing on the sample text, and obtaining an enhanced feature sequence; calling a text feature extraction model to perform feature extraction processing on the enhanced feature sequence, and obtaining a first feature sequence set and a second feature sequence set; wherein the first feature sequence set and the second feature sequence set are feature sequence sets extracted respectively after randomly shielding some parameters of the text feature extraction model, the first feature sequence set includes a first sample feature sequence and a second sample feature sequence, the second feature sequence set includes a third sample feature sequence and a fourth sample feature sequence, the first sample feature sequence and the third sample feature sequence are both output feature sequences of the first hidden layer in the text feature extraction model, and the second sample feature sequence and the fourth sample feature sequence are both output feature sequences of the last hidden layer in the text feature extraction model; performing pooling processing and whitening processing on the first sample feature sequence and the second sample feature sequence to obtain a first sample sentence feature of the sample text; performing pooling processing and whitening processing on the third sample feature sequence and the fourth sample feature sequence to obtain a second sample sentence feature of the sample text; combining the first sample sentence feature and the second sample sentence feature into a positive sample pair, and training the text feature extraction model based on the positive sample pair.

[0126] In one embodiment, when the processor 1104 trains a text feature extraction model based on positive sample pairs, it is specifically used to: obtain the sample sentence features to be combined, where the sample sentence features to be combined are the first sample sentence features or the second sample sentence features of the remaining sample texts in the same training batch as the sample text; combine the sample sentence features of the sample text with the sample sentence features to be combined into a negative sample pair, and train the text feature extraction model based on the positive sample pairs and the negative sample pairs, where the sample sentence features of the sample text are the first sample sentence features or the second sample sentence features of the sample text.

[0127] In one embodiment, the sample text includes A phrases, where A is a positive integer. The processor 1104 performs data enhancement processing on the sample text to obtain an enhanced feature sequence, which is specifically used to: convert the A phrases into A word vectors, into A position vectors, and into A sentence vectors; randomly select a target vector from the A word vectors, the A position vector positions, and the A sentence vectors, and adjust the value of the target vector to a preset vector; and splice the positions of the phrases corresponding to the unselected vectors and the adjusted target vectors in the sample text into an enhanced feature sequence.

[0128] In one embodiment, the number of enhanced feature sequences is K, where K is an integer greater than 1, and the K enhanced feature sequences are enhanced feature sequences generated after random selection K times.

[0129] In one embodiment, the processor 1104 performs whitening processing on the pooled features to obtain sentence features of the target text, specifically for: performing principal component analysis processing on the pooled features to generate a feature vector of the pooled features; performing standard deviation normalization processing on the feature vector of the pooled features to obtain sentence features of the target text.

[0130] In one embodiment, the processor 1104 further performs the following steps: obtaining a preset text, the preset text including L phrases, where L is a positive integer greater than 1; calling a text feature extraction model to perform feature extraction processing on the L phrases to obtain a first preset feature sequence and a second preset feature sequence, the first preset feature sequence being the output feature sequence of the first hidden layer in the text feature extraction model, and the second preset feature sequence being the output feature sequence of the last hidden layer in the text feature extraction model; performing pooling processing and whitening processing on the first preset feature sequence and the second preset feature sequence to obtain sentence features corresponding to the preset text; calculating the similarity between the sentence features corresponding to the preset text and the sentence features corresponding to the target text; and determining the matching result of the preset text and the target text based on the similarity.

[0131] In one embodiment, the matching result includes unmatched text, and the processor 1104 further performs the following steps: if the similarity determines that the matching result of the preset text and the target text is unmatched text, extracting the keyword group corresponding to the target text from the N phrases included in the target text; extracting the keyword group corresponding to the preset text from the L phrases included in the preset text; determining the reason for the mismatch based on the keyword group corresponding to the target text and the keyword group corresponding to the preset text, and outputting the reason for the mismatch.

[0132] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to execute all or part of the steps of the above methods in each embodiment of the present application. Among them, the aforementioned storage medium may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (English: Read-Only Memory, abbreviated: ROM) or random access memory (English: Random Access Memory, abbreviated: RAM) and other media that can store program code.

[0133] Those skilled in the art will appreciate that the units and steps of each example described in conjunction with the embodiments disclosed in this application can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0134] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer storage medium or transmitted via a computer storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0135] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A text processing method, characterized in that: The method comprises: Obtaining a target text, wherein the target text includes N phrases, where N is a positive integer greater than 1; Calling a text feature extraction model to perform feature extraction processing on the N phrases to obtain a first original feature sequence and a second original feature sequence, wherein the text feature extraction model includes M hidden layers arranged in series, the first original feature sequence and the second original feature sequence are output feature sequences of any two hidden layers in the M hidden layers, and M is an integer not less than 2; the first original feature sequence is the output feature sequence of the first hidden layer in the M hidden layers, and the second original feature sequence is the output feature sequence of the last hidden layer in the M hidden layers; Performing pooling processing on the first original feature sequence and the second original feature sequence to obtain pooled features of the target text; The pooled features are whitened to obtain sentence features of the target text, and the sentence features of the target text are output.

2. The method according to claim 1, characterized in that The method further comprises: Obtain sample text, perform data enhancement processing on the sample text, and obtain an enhanced feature sequence; Calling a text feature extraction model to perform feature extraction processing on the enhanced feature sequence to obtain a first feature sequence set and a second feature sequence set; wherein the first feature sequence set and the second feature sequence set are feature sequence sets respectively extracted after randomly shielding some parameters of the text feature extraction model, the first feature sequence set includes a first sample feature sequence and a second sample feature sequence, the second feature sequence set includes a third sample feature sequence and a fourth sample feature sequence, the first sample feature sequence and the third sample feature sequence are both output feature sequences of the first hidden layer in the text feature extraction model, and the second sample feature sequence and the fourth sample feature sequence are both output feature sequences of the last hidden layer in the text feature extraction model; Performing pooling processing and whitening processing on the first sample feature sequence and the second sample feature sequence to obtain a first sample sentence feature of the sample text; Performing pooling and whitening processing on the third sample feature sequence and the fourth sample feature sequence to obtain the second sample sentence feature of the sample text; The features of the first sample sentence and the features of the second sample sentence are combined into a positive sample pair, and a text feature extraction model is trained based on the positive sample pair.

3. The method according to claim 2, characterized in that The training text feature extraction model based on positive samples includes: Acquire features of sample sentences to be combined, where the features of the sample sentences to be combined are features of first sample sentences or features of second sample sentences of other sample texts in the same training batch as the sample text; The sample sentence feature of the sample text and the sample sentence feature to be combined are combined into a negative sample pair, and a text feature extraction model is trained based on the positive sample pair and the negative sample pair. The sample sentence feature of the sample text is the first sample sentence feature or the second sample sentence feature of the sample text.

4. The method according to claim 2, characterized in that The sample text includes A phrases, where A is a positive integer. The sample text is subjected to data enhancement processing to obtain an enhanced feature sequence, including: Convert A phrases into A word vectors, into A position vectors, and into A clause vectors; Randomly select a target vector from A word vectors, A position vector positions, and A sentence vectors, and adjust the value of the target vector; The positions of the phrases corresponding to the unselected vectors and the adjusted target vector in the sample text are spliced ​​into the enhanced feature sequence.

5. The method according to claim 4, characterized in that The number of the enhanced feature sequences is K, where K is an integer greater than 1, and the K enhanced feature sequences are enhanced feature sequences generated after being randomly selected K times.

6. The method according to claim 1, characterized in that The whitening process is performed on the pooled features to obtain the sentence features of the target text, including: Performing principal component analysis on the pooled features to generate a feature vector of the pooled features; The feature vector of the pooled feature is normalized by standard deviation to obtain the sentence feature of the target text.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining a preset text, wherein the preset text includes L phrases, where L is a positive integer greater than 1; Calling a text feature extraction model to perform feature extraction processing on the L phrases to obtain a first preset feature sequence and a second preset feature sequence, where the first preset feature sequence is an output feature sequence of a first hidden layer in the text feature extraction model, and the second preset feature sequence is an output feature sequence of a last hidden layer in the text feature extraction model; Performing pooling processing and whitening processing on the first preset feature sequence and the second preset feature sequence to obtain sentence features corresponding to the preset text; Calculating the similarity between the sentence features corresponding to the preset text and the sentence features corresponding to the target text; A matching result between the preset text and the target text is determined according to the similarity.

8. The method according to claim 7, characterized in that The matching result includes unmatched text, and the method further includes: If the matching result between the preset text and the target text is an unmatched text, extracting a keyword group corresponding to the target text from the N phrases included in the target text; Extracting a keyword group corresponding to the preset text from the L phrases included in the preset text; A mismatch reason is determined based on the keyword group corresponding to the target text and the keyword group corresponding to the preset text, and the mismatch reason is output.

9. A text processing device, characterized in that: The device comprises: an acquisition unit, configured to acquire a target text, wherein the target text includes N phrases, where N is a positive integer greater than 1; a processing unit, configured to call a text feature extraction model to perform feature extraction processing on the N phrases to obtain a first original feature sequence and a second original feature sequence, wherein the text feature extraction model includes M hidden layers arranged in series, the first original feature sequence and the second original feature sequence are output feature sequences of any two hidden layers in the M hidden layers, and M is an integer not less than 2; the first original feature sequence is the output feature sequence of a first hidden layer in the M hidden layers, and the second original feature sequence is the output feature sequence of a last hidden layer in the M hidden layers; The processing unit is further configured to perform pooling processing on the first original feature sequence and the second original feature sequence to obtain pooled features of the target text; The processing unit is further configured to perform whitening processing on the pooled features to obtain sentence features of the target text; The output unit is used to output the sentence features of the target text.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the text processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, wherein when the computer program is read and executed by a processor of a computer device, the computer device executes the text processing method according to any one of claims 1 to 8.

12. A computer program product, comprising computer instructions, wherein the computer instructions are stored in a computer-readable storage medium, and a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, so that the computer device executes the text processing method according to any one of claims 1 to 8.

Images