Text auditing method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of deep learning, and discloses a text auditing method and device, computer equipment and a storage medium, and the method comprises the steps: inputting a problem text into an embedded layer of a first network model, and obtaining a first text vector; inputting the first text vector into a first decoder module of the first network model to obtain a first feature vector which is used for generating a reply text; performing dimension reduction processing on the first text vector and the first feature vector to obtain a second text vector and a second feature vector; and after feature fusion is performed on the second text vector and the second feature vector, inputting the second text vector and the second feature vector into a second decoder module of the second network model to obtain an audit result of the reply text. It can be seen that in the text auditing process of the second network model, the feature extraction capacity of the reasoning capacity of the first network model is fully utilized, and compared with the mode that the reply text is directly input into the second network model for text auditing, the purpose of improving the text auditing precision can be achieved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a text review method, device, computer equipment and storage medium. Background Art

[0002] Large Language Models, also known as large-scale language models, are widely used in text understanding, text generation, sentiment classification, etc. When large language models are used to generate text, the text generated by the large language models needs to be reviewed to avoid sensitive content in the text.

[0003] At present, while ensuring the efficiency of review, a common method is to set up a small classification model separately from the large language model. The classification model is used to review the text output by the large language model. After confirming that the text does not contain sensitive content, the text is presented to the user.

[0004] However, due to the limitation of the size of small classification models, there is a problem of low audit accuracy. Summary of the invention

[0005] The embodiments of the present invention provide a text review method, apparatus, computer equipment and storage medium to solve the problem of low review accuracy in existing text review methods.

[0006] A text review method, the method comprising: Input the question text into the embedding layer of the first network model to obtain a first text vector of the first dimension; Inputting the first text vector into a first decoder module of the first network model to obtain a first feature vector of a first dimension, wherein the first feature vector is used by the first network model to generate a reply text; Perform dimensionality reduction processing on the first text vector and the first feature vector respectively to obtain a second text vector of a second dimension and a second feature vector of a second dimension; After feature fusion of the second text vector and the second feature vector, the second text vector and the second feature vector are input into the second decoder module of the second network model to obtain the review result of the reply text. The first network model and the second network model constitute a two-way text processing model.

[0007] In the above method, optionally, the two-way text processing model is constructed in the following way: Connecting the output of the embedding layer in the first network model to the input of the first decoder module and the input of the feature fusion module respectively; Connecting the output of the first decoder module to the input of the feature fusion module, wherein the feature fusion module is used to perform feature fusion on the first text vector output by the embedding layer and the first feature vector output by the first decoder module; The output of the feature fusion module is connected to the input of the second decoder module to obtain the two-way text processing model. A downsampling module is respectively arranged between the embedding layer and the feature fusion module, and between the first decoder module and the feature fusion module. The downsampling module is used to perform the dimensionality reduction processing on the first text vector output by the embedding layer and the first feature vector output by the first decoder module.

[0008] In the above method, optionally, the first decoder module includes a first number of trunk decoders, and the second decoder module includes a second number of side decoders, where the second number of layers is equal to the first number of layers; The step of connecting the output of the feature fusion module to the input of the second decoder module to obtain the two-way text processing model includes: The output of the embedding layer is connected to the first layer of the backbone decoder and the first feature fusion module respectively, and the downsampling module is respectively arranged between the embedding layer and the first layer of the backbone decoder and the first feature fusion module; According to the arrangement order of the backbone decoders in the first decoder module, the output of the backbone decoder of the nth layer and the output of the side decoder of the n-1th layer are respectively connected to the input of the nth feature fusion module, and the downsampling module is respectively arranged between the output of the backbone decoder of the nth layer and the input of the nth feature fusion module; The output of the nth feature fusion module is connected to the input of the nth layer of the side branch decoder to construct the dual-path text processing model.

[0009] In the above method, optionally, the first decoder module includes x layers of main shallow decoders, and the second decoder module includes x layers of side branch shallow decoders; The step of connecting the output of the trunk decoder at the nth layer and the output of the side decoder at the n-1th layer to the input of the nth feature fusion module respectively according to the arrangement order of the trunk decoders in the first decoder module, comprising: According to the arrangement order of the trunk shallow decoder in the first decoder module, the output of the nth layer of the trunk shallow decoder and the output of the n-1th layer of the side branch shallow decoder are respectively connected to the input of the nth feature fusion module, and the downsampling module is respectively arranged between the output of the nth layer of the trunk shallow decoder and the input of the nth feature fusion module.

[0010] In the above method, optionally, the step of inputting the question text into the embedding layer of the first network model to obtain a first text vector of the first dimension includes: Preprocess the question text to obtain a token ID card corresponding to the question text; Input the token ID card into the embedding layer to obtain the word vector corresponding to the question text; Performing position encoding on the word vector to obtain a position vector corresponding to the word vector; The word vector and the corresponding position vector are added to obtain the first text vector.

[0011] In the above method, optionally, the reply text is obtained by: Converting the first feature vector into a vocabulary probability distribution; The reply text is generated by using the vocabulary with the highest probability in the probability distribution of the vocabulary.

[0012] In the above method, optionally, the step of fusing the second text vector and the second feature vector and inputting the result of the feature fusion into a second decoder module of a second network model to obtain the review result of the reply text includes: Inputting the second text vector and the second feature vector into the feature fusion module to obtain a fused feature vector; The fused feature vector is input into the second decoder module to obtain the audit result.

[0013] A text review device, comprising: A vector generation module, used for inputting the question text into the embedding layer of the first network model to obtain a first text vector of a first dimension; A feature extraction module, used for inputting the first text vector into a first decoder module of the first network model to obtain a first feature vector of a first dimension, wherein the first feature vector is used for the first network model to generate a reply text; A vector dimension reduction module, used for performing dimension reduction processing on the first text vector and the first feature vector respectively, to obtain a second text vector of a second dimension and a second feature vector of a second dimension; The text review module is used to perform feature fusion on the second text vector and the second feature vector, and then input the result into the second decoder module of the second network model to generate the review result of the reply text. The first network model and the second network model constitute a two-way text processing model.

[0014] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned text review methods when executing the computer program.

[0015] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements any of the above-mentioned text review methods.

[0016] The above-mentioned text review method, device, computer equipment and storage medium input the question text into the embedding layer of the first network model to obtain the first text vector of the first dimension; input the first text vector into the first decoder module of the first network model to obtain the first feature vector of the first dimension, and the first feature vector is used for the first network model to generate a reply text; respectively perform dimensionality reduction processing on the first text vector and the first feature vector to obtain the second text vector of the second dimension and the second feature vector of the second dimension; after feature fusion of the second text vector and the second feature vector, input them into the second decoder module of the second network model to obtain the review result of the reply text. It can be seen that the second network model in the present invention fully utilizes the feature extraction capability of the first network model's own reasoning capability in the process of text review of the reply text, and can achieve the purpose of improving the accuracy of text review compared to directly inputting the reply text into the second network model for text review. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0018] Figure 1 It is a flow chart for implementing a text review method disclosed in an embodiment of the present invention; Figure 2 It is a partial implementation flow chart of a text review method disclosed in one embodiment of the present invention; Figure 3 It is a partial implementation flow chart of a text review method disclosed in one embodiment of the present invention; Figure 4 It is a structural schematic diagram of a two-way text processing model disclosed in one embodiment of the present invention; Figure 5 It is a partial implementation flow chart of a text review method disclosed in one embodiment of the present invention; Figure 6is another structural schematic diagram of a two-way text processing model disclosed in one embodiment of the present invention; Figure 7 It is another structural schematic diagram of a dual-path text processing model disclosed in one embodiment of the present invention; Figure 8 It is a partial implementation flow chart of a text review method disclosed in one embodiment of the present invention; Fig. 9 It is a partial implementation flow chart of a text review method disclosed in one embodiment of the present invention; Fig.10 It is a structural schematic diagram of a text review device in one embodiment of the present invention; Fig.11 It is a structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0021] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0022] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]", depending on the context.

[0023] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" etc. described in the present specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0025] The present invention discloses a text review method, device, computer equipment and storage medium, which obtains a question text, inputs the question text into the embedding layer of a first network model, obtains a first feature vector of a first dimension, generates a reply text based on the first feature vector, performs dimensionality reduction processing on the first feature vector, obtains a second feature vector of a second dimension, inputs the second feature vector into a second network model, obtains a review result of the reply text, and the first network model and the second network model constitute a dual-path text processing model. It can be seen that in the present invention, the first network model and the second network model constitute a dual-path text processing model, which fully utilizes the feature extraction capability of the first network model's own reasoning capability, and can achieve a more accurate text review of the reply text generated by the first network model. This is explained below through specific embodiments.

[0026] like Figure 1 As shown, it is a flowchart of a text review method disclosed in an embodiment of the present invention. The method is applicable to electronic devices with data processing capabilities, such as mobile phones, tablet computers, laptops, personal computers, servers and other devices. The method in this embodiment specifically includes the following steps: S101: Input the question text into the embedding layer of the first network model to obtain a first text vector of the first dimension.

[0027] The question text may be manually input text or text obtained through voice recognition. The method for obtaining the question text is not limited in this embodiment.

[0028] For example, taking a mobile phone as an example, when the voice assistant in the mobile phone receives a wake-up command, it performs voice recognition on the next received voice signal, and then recognizes voice text from the voice signal, and the voice text is the question text.

[0029] For another example, taking a computer as an example, start the browser in the computer and enter text in the input bar of the browser. At this time, the question text can be obtained.

[0030] In a specific implementation, the first network model in this embodiment is a text generation model, such as GPT-2, GPT-3, etc. The specific model structure of the first network model is not limited in this embodiment. The question text is input into the embedding layer of the first network model, and the question text is feature extracted through the embedding layer to obtain the first text vector of the first dimension.

[0031] Furthermore, the first text vector in this embodiment can be obtained by the following steps, specifically: Figure 2 As shown: S201: Preprocess the question text to obtain a token ID card corresponding to the question text.

[0032] It should be understood that the question text is text, and the model cannot process text directly. Therefore, it is necessary to pre-process it to obtain a token ID that the model can recognize, and then input the token ID into the model so that the model can process the token ID. Among them, pre-processing can include word segmentation and dictionary matching. Among them, word segmentation is to divide the question text from a complete sentence into single words, namely tokens, and then convert each word into a corresponding token ID, namely token id.

[0033] In a specific implementation, in this embodiment, the word segmentation tool can be used to segment the question text to obtain individual words, and then each word is converted based on each word to obtain a token ID card for each word. Among them, the word segmentation tools in this embodiment include but are not limited to the jieba word segmentation tool, the SnowNLP word segmentation tool, the THULAC word segmentation tool, and the NLPIR word segmentation tool. In addition, in this embodiment, each word can be converted into a corresponding token ID card through self.vocab.w2i.

[0034] For example, taking the question text "ultra-light-soled casual women's shoes" as an example, after segmenting the question text through the word segmentation tool, we get ['super', 'light', 'soles', 'rest', 'leisure', 'women', 'shoes'], and then convert the token to a token id, we get [6631, 6768, 2419, 828, 7312, 1957, 7490]. Thus, we can get the token ID card corresponding to the question text.

[0035] S202: Input the token ID card into the embedding layer to obtain the word vector corresponding to the question text.

[0036] S203: Perform position encoding on the word vector to obtain a position vector corresponding to the word vector.

[0037] S204: Add the word vector and the corresponding position vector to obtain a first text vector.

[0038] In the specific implementation, in this embodiment, the token ID card can be converted into a word vector through the embedding layer, that is, each token ID card is represented by a word vector, each token ID card corresponds to a different word vector, these word vectors are positionally encoded to obtain the position vector corresponding to each word vector, and then the text vector and the corresponding position vector are added to obtain the first feature vector.

[0039] It can be understood that the word vector in this embodiment represents the text content of each word, and the position vector represents the position of each word in the question text. After the word vector and the position vector are concatenated together, the first text vector is obtained.

[0040] S102: Input the first text vector into the first decoder module of the first network model to obtain a first feature vector of the first dimension, and the first feature vector is used by the first network model to generate a reply text.

[0041] The first text vector is input into the first decoder module of the first network model, and the first decoder module performs feature extraction on the first text vector to obtain a first feature vector. The first network model generates a reply text based on the first feature vector.

[0042] S103: Perform dimensionality reduction processing on the first text vector and the first feature vector respectively to obtain a second text vector of a second dimension and a second feature vector of a second dimension.

[0043] It can be understood that the second dimension is smaller than the first dimension, that is, the dimension of the second feature vector is smaller than the dimension of the first feature vector, and the dimension of the second text vector is smaller than the dimension of the first text vector.

[0044] In a specific implementation, in this embodiment, a downsampling module can be used to perform dimensionality reduction processing on the first feature vector to obtain a second feature vector of the second dimension. The downsampling coefficient of the downsampling module can be set according to actual needs, such as 6, 8 or 10, etc. In this embodiment, the downsampling coefficient of the downsampling module is not limited. When the downsampling coefficient is 8, it means that the dimension of the first feature vector is 8 times that of the second feature vector.

[0045] S104: After feature fusion of the second text vector and the second feature vector, the second text vector and the second feature vector are input into a second decoder module of the second network model to generate a review result of the reply text.

[0046] Among them, the first network model and the second network model constitute a two-way text processing model.

[0047] It should be noted that in this embodiment, the first network model can be used as the backbone network, and the second network model can be used as the side network, thereby constructing a two-way text processing model, wherein the first network model (i.e., the backbone network) is used to generate a reply text based on the first feature vector, and the second network model (i.e., the side network) is used to generate a review result of the reply text based on the second text vector and the fusion feature vector of the second feature vector, and the review result is used to characterize whether the reply text complies with the review rules. The review rule can be whether there are sensitive words in the reply text. If the review result indicates that there are no sensitive words, the reply text is output to the user. If the review result indicates that there are sensitive words, the reply text is not output to the user.

[0048] In one embodiment, when the backbone network gradually generates a reply text based on the first feature vector, if the text with the highest predicted probability does not comply with the review rules, then the words that comply with the review rules are selected in order of probability to generate a reply text to be output to the user.

[0049] It should be noted that the first network model in this embodiment is a pre-trained model, while the second network model is a model that needs to be trained. Therefore, during the training process of the second network model, the parameters of the first network model need to be frozen and only the parameters of the second network model need to be updated. Therefore, the introduction of the second network model can avoid negative impact on the reply text generation quality of the first network model.

[0050] In a specific implementation, in this embodiment, the training question text required for training and the classification labels corresponding to the training question text can be prepared in advance, and the training question text can be input into the first network model and the second network model for training, so as to continuously update the model parameters of the second network model during the training process. After the training question text is input into the first network model, the first network model outputs the reply text corresponding to the training question text, and the second network model outputs the review result corresponding to the reply text. The classification loss value (such as the cross entropy loss function) of the second network model is calculated based on the review result and the classification label. When the classification loss value is lower than the preset loss threshold, it is determined that the second network model is fully trained and the model parameters of the second network model are stopped from being updated. In other embodiments, the classification loss value can also be other loss function values, such as autoregressive loss value, contrast loss value, etc.

[0051] In summary, the present invention discloses a text review method, which includes inputting a question text into the embedding layer of a first network model to obtain a first text vector of a first dimension; inputting the first text vector into the first decoder module of the first network model to obtain a first feature vector of the first dimension, and the first feature vector is used by the first network model to generate a reply text; performing dimensionality reduction processing on the first text vector and the first feature vector respectively to obtain a second text vector of a second dimension and a second feature vector of a second dimension; performing feature fusion on the second text vector and the second feature vector, and then inputting the second decoder module of the second network model to obtain the review result of the reply text. It can be seen that the second network model in the present invention fully utilizes the feature extraction capability of the first network model's own reasoning capability in the process of text reviewing the reply text, and can achieve the purpose of improving the accuracy of text review compared to directly inputting the reply text into the second network model for text review.

[0052] In one embodiment, the two-way text processing model in this embodiment is constructed in the following manner: S301: Connecting the output of the embedding layer in the first network model to the input of the first decoder module and the input of the feature fusion module respectively; S302: Connecting the output of the first decoder module to the input of a feature fusion module, where the feature fusion module is used to perform feature fusion on the first text vector output by the embedding layer and the first feature vector output by the first decoder module; S303: Connect the output of the feature fusion module to the input of the second decoder module to obtain a two-way text processing model. A downsampling module is respectively provided between the embedding layer and the feature fusion module, and between the first decoder module and the feature fusion module. The downsampling module is used to perform dimensionality reduction processing on the first text vector output by the embedding layer and the first feature vector output by the first decoder module.

[0053] like Figure 4 As shown, it is a structural schematic diagram of the dual-path text processing model in this embodiment, wherein Q represents the question text, R represents the reply text, E represents the audit result, V1 represents the first text vector, V2 represents the second text vector, + represents the feature fusion module, W1 represents the first feature vector, and W2 represents the second feature vector. In practical applications, in this embodiment, after the question text is input into the embedding layer, a first text vector of a first dimension is obtained, the first text vector is input into the downsampling module, and the first feature vector is subjected to dimensionality reduction processing by the downsampling module to obtain the second text vector of the second dimension, the first text vector is input into the first decoder module to obtain the first feature vector, the first feature vector is subjected to dimensionality reduction processing by the downsampling module to obtain the second feature vector of the second dimension, the second text vector and the second feature vector are subjected to feature fusion and input into the second decoder module to obtain the audit result.

[0054] In another embodiment, the first decoder module in this embodiment includes a first number of trunk decoders, and the second decoder module includes a second number of side decoders, where the second number of layers is equal to the first number of layers. On this basis, the two-way text processing model in this embodiment can be constructed as follows: Figure 5 As shown: S501: The output of the embedding layer is connected to the first layer backbone decoder and the first feature fusion module respectively, and a downsampling module is provided between the embedding layer and the first layer backbone decoder and the first feature fusion module respectively.

[0055] S502: According to the arrangement order of the backbone decoders in the first decoder module, the output of the n-th layer backbone decoder and the output of the n-1-th layer side branch decoder are respectively connected to the input of the n-th feature fusion module, and a downsampling module is respectively arranged between the output of the n-th layer backbone decoder and the input of the n-th feature fusion module.

[0056] S503: Connect the output of the nth feature fusion module to the input of the nth layer side branch decoder to construct a dual-path text processing model.

[0057] like Figure 6 As shown, it is another structural schematic diagram of the dual-path text processing model in this embodiment, Q represents the question text, R represents the reply text, E represents the audit result, V1 represents the first text vector, V2 represents the second text vector, + represents the feature fusion module, W1 represents the first feature vector, and W2 represents the second feature vector. In practical applications, in this embodiment, after the output of the nth layer backbone decoder in the first network model is processed by the downsampling module for dimensionality reduction, it is necessary to perform feature fusion with the output of the corresponding n-1th layer side branch decoder, and then the obtained fused feature vector is input into the nth layer side branch decoder to finally obtain the audit result.

[0058] To sum up, the second network model in this embodiment utilizes the reasoning ability and feature extraction ability of the first network model itself during the review of the reply text. Compared with the use of the second network model alone to review the reply text, the purpose of improving the review accuracy of the second network model can be achieved.

[0059] based on Figure 5 In the specific implementation, the first decoder module in this embodiment includes x layers of trunk shallow decoders, and the second decoder module includes x layers of side branch shallow decoders. On this basis, step S502 can be implemented as follows: According to the arrangement order of the trunk shallow decoder in the first decoder module, the output of the n-th trunk shallow decoder and the output of the n-1-th side branch shallow decoder are respectively connected to the input of the n-th feature fusion module, and a downsampling module is respectively arranged between the output of the n-th trunk shallow decoder and the input of the n-th feature fusion module.

[0060] It should be noted that there is a main shallow decoder in the first decoder module, and there is also a main deep decoder. The number of layers of the main shallow decoder plus the number of layers of the main deep decoder equals the total number of layers of the decoder in the first decoder module. Similarly, there is a side branch shallow decoder in the second decoder module, and there is also a side branch deep decoder. The number of layers of the side branch shallow decoder plus the number of layers of the side branch deep decoder equals the total number of layers of the decoder in the second decoder module.

[0061] like Figure 7 As shown, it is another structural diagram of the dual-path text processing model in this embodiment. It can be seen that the first decoder module and the second decoder module in this embodiment only perform feature fusion in the shallow network part, but do not perform feature fusion in the deep network part. That is to say, the output features of the nth layer trunk shallow decoder in the first decoder module are fused with the output features of the n-1th layer side branch shallow decoder in the second decoder module, and then the fused feature vector is input into the nth layer side branch shallow decoder. In the side branch deep decoder part, except for the 1st layer side branch deep decoder whose input is the fused feature vector, the other side branch deep decoders only use the output of the previous layer side branch deep decoder as input.

[0062] To sum up, the second network model and the first network model in this embodiment only perform feature fusion in the shallow network, but not in the deep network. This not only makes full use of the first network model's own reasoning ability and feature extraction ability, but also avoids the problem that the first network model and the second network model affect the second network model due to their different tasks, thereby achieving the purpose of improving the text review accuracy of the second network model.

[0063] In one embodiment, the reply text can be obtained by the following steps: Figure 8 As shown: S801: Convert the first feature vector into vocabulary probability distribution.

[0064] It should be understood that in the text generation model, that is, the first network model in this embodiment, when generating a reply text, it will gradually predict according to the order of words, and finally obtain a complete reply text. That is to say, when predicting the first word, the vocabulary probability distribution of the first word is obtained, and after the first word is determined according to the vocabulary probability distribution, the second word is predicted, and the vocabulary probability distribution of the second word is obtained, and so on, until the reply text is predicted.

[0065] In a specific implementation, in this embodiment, the first feature vector can be converted into a vocabulary probability distribution through a linear layer and a Softmax layer. The vocabulary probability distribution represents the probability of generating the next word. In the vocabulary, the word with the highest probability is usually selected as the next word, and so on, until the complete reply text is predicted.

[0066] S802: Generate a reply text using the word with the highest probability in the probability distribution of the vocabulary.

[0067] In one embodiment, the vocabulary probability distribution may be as shown in Table 1: Among them, the probability of word 2 is the largest, so word 2 is selected as the next word, and so on. The next word is predicted based on the predicted word, and the vocabulary probability distribution of the next word is obtained, and then the next word is determined according to the vocabulary probability distribution.

[0068] It should be noted that the above-mentioned method of generating the reply text by greedy decoding in this embodiment is only a partial implementation method in this embodiment. This embodiment can also use beam decoding and prefix beam decoding to generate the reply text. This embodiment does not limit the decoding method used to generate the reply text.

[0069] In one embodiment, the audit result can be obtained through the following steps: Fig. 9 As shown: S901: Input the second text vector and the second feature vector into a feature fusion module to obtain a fused feature vector.

[0070] Among them, the feature fusion module is a learnable module, which is used for weight matching when the second text vector and the second feature vector are fused. The second text vector and the second feature vector are fused according to the weight matching to obtain a fused feature vector.

[0071] In a specific implementation, in this embodiment, the weight ratio of feature fusion in the feature fusion module can be updated as a learnable parameter during the training of the second network model, that is, the weight ratio of feature fusion in the feature fusion module is updated during the training of the second network model, and when the second network model is fully trained, the weight ratio of the feature fusion module is obtained. In addition, in this embodiment, the weight ratio of the feature fusion module can also be manually set according to actual needs, and the method for obtaining the weight ratio in the feature fusion module is not limited in this embodiment.

[0072] S902: Input the fused feature vector into the second decoder module to obtain the audit result.

[0073] The fused feature vector is input into the second decoder module to obtain a third feature vector, and the third feature vector is audited and classified to obtain an audit result.

[0074] In a specific implementation, in this embodiment, a binary classification head can be used to audit and classify the third feature vector to obtain an audit result. The audit result is used to characterize whether the reply text meets the audit specifications. For example, if the audit result is "1", it indicates that the reply text does not meet the audit specifications. If the audit result is "0", it indicates that the reply text meets the audit specifications. At the user level, the audit result can be output by outputting yes (corresponding to 1) or no (corresponding to 0), such as compliance or non-compliance.

[0075] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0076] like Fig.10 The figure shows a schematic diagram of the structure of a text review device disclosed in the present invention, which is applicable to electronic devices with data processing capabilities, such as mobile phones, tablet computers, laptops, personal computers, servers and other devices. The method in this embodiment may specifically include the following steps: A vector generation module 1001 is used to input the question text into the embedding layer of the first network model to obtain a first text vector of a first dimension; A feature extraction module 1002 is used to input the first text vector into a first decoder module of a first network model to obtain a first feature vector of a first dimension, and the first feature vector is used by the first network model to generate a reply text; A vector dimension reduction module 1003, used to perform dimension reduction processing on the first text vector and the first feature vector respectively, to obtain a second text vector of a second dimension and a second feature vector of a second dimension; The text review module 1004 is used to perform feature fusion on the second text vector and the second feature vector, and then input the result into the second decoder module of the second network model to generate the review result of the reply text. The first network model and the second network model constitute a two-way text processing model.

[0077] In summary, the present invention discloses a text review device, which inputs a question text into the embedding layer of a first network model to obtain a first text vector of a first dimension; inputs the first text vector into the first decoder module of the first network model to obtain a first feature vector of the first dimension, and the first feature vector is used by the first network model to generate a reply text; performs dimensionality reduction processing on the first text vector and the first feature vector respectively to obtain a second text vector of a second dimension and a second feature vector of a second dimension; after feature fusion of the second text vector and the second feature vector, the second decoder module of the second network model is input to obtain the review result of the reply text. It can be seen that the second network model in the present invention fully utilizes the feature extraction capability of the first network model's own reasoning capability in the process of text review of the reply text, and can achieve the purpose of improving the accuracy of text review compared to directly inputting the reply text into the second network model for text review.

[0078] In one implementation, the two-way text processing model is constructed as follows: Connect the output of the embedding layer in the first network model to the input of the first decoder module and the input of the feature fusion module respectively; Connecting the output of the first decoder module to the input of a feature fusion module, where the feature fusion module is used to perform feature fusion on the first text vector output by the embedding layer and the first feature vector output by the first decoder module; The output of the feature fusion module is connected to the input of the second decoder module to obtain a two-way text processing model. A downsampling module is respectively arranged between the embedding layer and the feature fusion module, and between the first decoder module and the feature fusion module. The downsampling module is used to perform dimensionality reduction processing on the first text vector output by the embedding layer and the first feature vector output by the first decoder module.

[0079] In one implementation, the first decoder module includes a first number of trunk decoders, and the second decoder module includes a second number of side decoders, where the second number of layers is equal to the first number of layers; The output of the feature fusion module is connected to the input of the second decoder module to obtain a two-way text processing model, including: The output of the embedding layer is connected to the first-layer backbone decoder and the first feature fusion module respectively, and a downsampling module is set between the embedding layer and the first-layer backbone decoder and the first feature fusion module respectively; According to the arrangement order of the backbone decoders in the first decoder module, the output of the n-th layer backbone decoder and the output of the n-1-th layer side branch decoder are respectively connected to the input of the n-th feature fusion module, and a downsampling module is respectively arranged between the output of the n-th layer backbone decoder and the input of the n-th feature fusion module; The output of the nth feature fusion module is connected to the input of the nth layer side branch decoder to construct a two-way text processing model.

[0080] In one implementation, the first decoder module includes x layers of main shallow decoders, and the second decoder module includes x layers of side branch shallow decoders; According to the arrangement order of the backbone decoders in the first decoder module, the output of the n-th layer backbone decoder and the output of the n-1-th layer side branch decoder are respectively connected to the input of the n-th feature fusion module, including: According to the arrangement order of the trunk shallow decoder in the first decoder module, the output of the n-th trunk shallow decoder and the output of the n-1-th side branch shallow decoder are respectively connected to the input of the n-th feature fusion module, and a downsampling module is respectively arranged between the output of the n-th trunk shallow decoder and the input of the n-th feature fusion module.

[0081] In one implementation, the vector generation module 902 is used to: Preprocess the question text to obtain the token ID card corresponding to the question text; Input the token ID card into the embedding layer to obtain the word vector corresponding to the question text; Perform position encoding on the word vector to obtain the position vector corresponding to the word vector; The word vector and the corresponding position vector are added to obtain the first text vector.

[0082] In one implementation, the reply text is obtained by: Convert the first feature vector,into a vocabulary probability distribution; Generate the reply text using the word with the highest probability in the vocabulary probability distribution.

[0083] In one implementation, after the second text vector and the second feature vector are feature-fused, the second text vector and the second feature vector are input into the second decoder module of the second network model to obtain the review result of the reply text, including: Inputting the second text vector and the second feature vector into a feature fusion module to obtain a fused feature vector; The fused feature vector is input into the second decoder module to obtain the audit result.

[0084] For the specific limitations of the text audit device, please refer to the relevant limitations of the text audit method above, which will not be repeated here. The various modules in the above-mentioned text audit device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0085] The embodiment of the present application also discloses a computer device, which may be a server, and its internal structure diagram may be as shown in Fig.11 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a text audit method is implemented.

[0086] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: Input the question text into the embedding layer of the first network model to obtain a first text vector of the first dimension; Inputting the first text vector into a first decoder module of a first network model to obtain a first feature vector of a first dimension, wherein the first feature vector is used by the first network model to generate a reply text; Perform dimensionality reduction processing on the first text vector and the first feature vector respectively to obtain a second text vector of a second dimension and a second feature vector of a second dimension; After feature fusion of the second text vector and the second feature vector, the vector is input into the second decoder module of the second network model to obtain the review result of the reply text. The first network model and the second network model constitute a two-way text processing model.

[0087] In summary, the present invention discloses a computer device, which inputs a question text into the embedding layer of a first network model to obtain a first text vector of a first dimension; inputs the first text vector into the first decoder module of the first network model to obtain a first feature vector of the first dimension, and the first feature vector is used by the first network model to generate a reply text; performs dimensionality reduction processing on the first text vector and the first feature vector respectively to obtain a second text vector of a second dimension and a second feature vector of a second dimension; after feature fusion of the second text vector and the second feature vector, the second decoder module of the second network model is input to obtain the review result of the reply text. It can be seen that the second network model in the present invention fully utilizes the feature extraction capability of the first network model's own reasoning capability in the process of text review of the reply text, and can achieve the purpose of improving the accuracy of text review compared to directly inputting the reply text into the second network model for text review.

[0088] In one implementation, the two-way text processing model is constructed as follows: Connect the output of the embedding layer in the first network model to the input of the first decoder module and the input of the feature fusion module respectively; Connecting the output of the first decoder module to the input of a feature fusion module, where the feature fusion module is used to perform feature fusion on the first text vector output by the embedding layer and the first feature vector output by the first decoder module; The output of the feature fusion module is connected to the input of the second decoder module to obtain a two-way text processing model. A downsampling module is respectively arranged between the embedding layer and the feature fusion module, and between the first decoder module and the feature fusion module. The downsampling module is used to perform dimensionality reduction processing on the first text vector output by the embedding layer and the first feature vector output by the first decoder module.

[0089] In one implementation, the first decoder module includes a first number of trunk decoders, and the second decoder module includes a second number of side decoders, where the second number of layers is equal to the first number of layers; The output of the feature fusion module is connected to the input of the second decoder module to obtain a two-way text processing model, including: The output of the embedding layer is connected to the first-layer backbone decoder and the first feature fusion module respectively, and a downsampling module is set between the embedding layer and the first-layer backbone decoder and the first feature fusion module respectively; According to the arrangement order of the backbone decoders in the first decoder module, the output of the n-th layer backbone decoder and the output of the n-1-th layer side branch decoder are respectively connected to the input of the n-th feature fusion module, and a downsampling module is respectively arranged between the output of the n-th layer backbone decoder and the input of the n-th feature fusion module; The output of the nth feature fusion module is connected to the input of the nth layer side branch decoder to construct a two-way text processing model.

[0090] In one implementation, the first decoder module includes x layers of main shallow decoders, and the second decoder module includes x layers of side branch shallow decoders; According to the arrangement order of the backbone decoders in the first decoder module, the output of the n-th layer backbone decoder and the output of the n-1-th layer side branch decoder are respectively connected to the input of the n-th feature fusion module, including: According to the arrangement order of the trunk shallow decoder in the first decoder module, the output of the n-th trunk shallow decoder and the output of the n-1-th side branch shallow decoder are respectively connected to the input of the n-th feature fusion module, and a downsampling module is respectively arranged between the output of the n-th trunk shallow decoder and the input of the n-th feature fusion module.

[0091] In one implementation, the question text is input into the embedding layer of the first network model to obtain a first text vector of the first dimension, including: Preprocess the question text to obtain the token ID card corresponding to the question text; Input the token ID card into the embedding layer to obtain the word vector corresponding to the question text; Perform position encoding on the word vector to obtain the position vector corresponding to the word vector; The word vector and the corresponding position vector are added to obtain the first text vector.

[0092] In one implementation, the reply text is obtained by: Convert the first feature vector,into a vocabulary probability distribution; Generate the reply text using the word with the highest probability in the vocabulary probability distribution.

[0093] In one implementation, after the second text vector and the second feature vector are feature-fused, the second text vector and the second feature vector are input into the second decoder module of the second network model to obtain the review result of the reply text, including: Inputting the second text vector and the second feature vector into a feature fusion module to obtain a fused feature vector; The fused feature vector is input into the second decoder module to obtain the audit result.

[0094] The present application also discloses a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor in a computer device, the computer device can perform the steps of any embodiment of a text review method disclosed in the present invention. The computer-readable storage medium can be non-volatile or volatile.

[0095] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: Input the question text into the embedding layer of the first network model to obtain a first text vector of the first dimension; Inputting the first text vector into a first decoder module of a first network model to obtain a first feature vector of a first dimension, wherein the first feature vector is used by the first network model to generate a reply text; Perform dimensionality reduction processing on the first text vector and the first feature vector respectively to obtain a second text vector of a second dimension and a second feature vector of a second dimension; After feature fusion of the second text vector and the second feature vector, the vector is input into the second decoder module of the second network model to obtain the review result of the reply text. The first network model and the second network model constitute a two-way text processing model.

[0096] In one implementation, the two-way text processing model is constructed as follows: Connect the output of the embedding layer in the first network model to the input of the first decoder module and the input of the feature fusion module respectively; Connecting the output of the first decoder module to the input of a feature fusion module, where the feature fusion module is used to perform feature fusion on the first text vector output by the embedding layer and the first feature vector output by the first decoder module; The output of the feature fusion module is connected to the input of the second decoder module to obtain a two-way text processing model. A downsampling module is respectively arranged between the embedding layer and the feature fusion module, and between the first decoder module and the feature fusion module. The downsampling module is used to perform dimensionality reduction processing on the first text vector output by the embedding layer and the first feature vector output by the first decoder module.

[0097] In one implementation, the first decoder module includes a first number of trunk decoders, and the second decoder module includes a second number of side decoders, where the second number of layers is equal to the first number of layers; The output of the feature fusion module is connected to the input of the second decoder module to obtain a two-way text processing model, including: The output of the embedding layer is connected to the first-layer backbone decoder and the first feature fusion module respectively, and a downsampling module is set between the embedding layer and the first-layer backbone decoder and the first feature fusion module respectively; According to the arrangement order of the backbone decoders in the first decoder module, the output of the n-th layer backbone decoder and the output of the n-1-th layer side branch decoder are respectively connected to the input of the n-th feature fusion module, and a downsampling module is respectively arranged between the output of the n-th layer backbone decoder and the input of the n-th feature fusion module; The output of the nth feature fusion module is connected to the input of the nth layer side branch decoder to construct a two-way text processing model.

[0098] In one implementation, the first decoder module includes x layers of main shallow decoders, and the second decoder module includes x layers of side branch shallow decoders; According to the arrangement order of the backbone decoders in the first decoder module, the output of the n-th layer backbone decoder and the output of the n-1-th layer side branch decoder are respectively connected to the input of the n-th feature fusion module, including: According to the arrangement order of the trunk shallow decoder in the first decoder module, the output of the n-th trunk shallow decoder and the output of the n-1-th side branch shallow decoder are respectively connected to the input of the n-th feature fusion module, and a downsampling module is respectively arranged between the output of the n-th trunk shallow decoder and the input of the n-th feature fusion module.

[0099] In one implementation, the question text is input into the embedding layer of the first network model to obtain a first text vector of the first dimension, including: Preprocess the question text to obtain the token ID card corresponding to the question text; Input the token ID card into the embedding layer to obtain the word vector corresponding to the question text; Perform position encoding on the word vector to obtain the position vector corresponding to the word vector; The word vector and the corresponding position vector are added to obtain the first text vector.

[0100] In one implementation, the reply text is obtained by: Convert the first feature vector,into a vocabulary probability distribution; Generate the reply text using the word with the highest probability in the vocabulary probability distribution.

[0101] In one implementation, after the second text vector and the second feature vector are feature-fused, the second text vector and the second feature vector are input into the second decoder module of the second network model to obtain the review result of the reply text, including: Inputting the second text vector and the second feature vector into a feature fusion module to obtain a fused feature vector; The fused feature vector is input into the second decoder module to obtain the audit result.

[0102] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0103] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0104] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A text review method, characterized in that: The method comprises: Input the question text into the embedding layer of the first network model to obtain a first text vector of the first dimension; Inputting the first text vector into a first decoder module of the first network model to obtain a first feature vector of a first dimension, wherein the first feature vector is used by the first network model to generate a reply text; Perform dimensionality reduction processing on the first text vector and the first feature vector respectively to obtain a second text vector of a second dimension and a second feature vector of a second dimension; After feature fusion of the second text vector and the second feature vector, the second text vector and the second feature vector are input into the second decoder module of the second network model to obtain the review result of the reply text. The first network model and the second network model constitute a two-way text processing model.

2. The text review method according to claim 1, characterized in that: The two-way text processing model is constructed in the following way: Connecting the output of the embedding layer in the first network model to the input of the first decoder module and the input of the feature fusion module respectively; Connecting the output of the first decoder module to the input of the feature fusion module, wherein the feature fusion module is used to perform feature fusion on the first text vector output by the embedding layer and the first feature vector output by the first decoder module; The output of the feature fusion module is connected to the input of the second decoder module to obtain the two-way text processing model. A downsampling module is respectively arranged between the embedding layer and the feature fusion module, and between the first decoder module and the feature fusion module. The downsampling module is used to perform the dimensionality reduction processing on the first text vector output by the embedding layer and the first feature vector output by the first decoder module.

3. The text review method according to claim 2, characterized in that: The first decoder module includes a first number of trunk decoders, and the second decoder module includes a second number of side decoders, where the second number of layers is equal to the first number of layers; The step of connecting the output of the feature fusion module to the input of the second decoder module to obtain the two-way text processing model includes: The output of the embedding layer is connected to the first layer of the backbone decoder and the first feature fusion module respectively, and the downsampling module is respectively arranged between the embedding layer and the first layer of the backbone decoder and the first feature fusion module; According to the arrangement order of the backbone decoders in the first decoder module, the output of the backbone decoder of the nth layer and the output of the side decoder of the n-1th layer are respectively connected to the input of the nth feature fusion module, and the downsampling module is respectively arranged between the output of the backbone decoder of the nth layer and the input of the nth feature fusion module; The output of the nth feature fusion module is connected to the input of the nth layer of the side branch decoder to construct the dual-path text processing model.

4. The text review method according to claim 3, characterized in that: The first decoder module includes x layers of main shallow decoders, and the second decoder module includes x layers of side branch shallow decoders; The step of connecting the output of the trunk decoder at the nth layer and the output of the side decoder at the n-1th layer to the input of the nth feature fusion module respectively according to the arrangement order of the trunk decoders in the first decoder module, comprising: According to the arrangement order of the trunk shallow decoder in the first decoder module, the output of the nth layer of the trunk shallow decoder and the output of the n-1th layer of the side branch shallow decoder are respectively connected to the input of the nth feature fusion module, and the downsampling module is respectively arranged between the output of the nth layer of the trunk shallow decoder and the input of the nth feature fusion module.

5. The text review method according to claim 1, characterized in that: The step of inputting the question text into the embedding layer of the first network model to obtain a first text vector of a first dimension includes: Preprocess the question text to obtain a token ID card corresponding to the question text; Input the token ID card into the embedding layer to obtain the word vector corresponding to the question text; Performing position encoding on the word vector to obtain a position vector corresponding to the word vector; The word vector and the corresponding position vector are added to obtain the first text vector.

6. The text review method according to claim 2, characterized in that: The reply text is obtained in the following manner: Converting the first feature vector into a vocabulary probability distribution; The reply text is generated by using the vocabulary with the highest probability in the probability distribution of the vocabulary.

7. The text review method according to claim 2, characterized in that: After the second text vector and the second feature vector are subjected to feature fusion, the second text vector and the second feature vector are input into a second decoder module of a second network model to obtain the review result of the reply text, including: Inputting the second text vector and the second feature vector into the feature fusion module to obtain a fused feature vector; The fused feature vector is input into the second decoder module to obtain the audit result.

8. A text review device, characterized in that: The device comprises: A vector generation module, used for inputting the question text into the embedding layer of the first network model to obtain a first text vector of a first dimension; A feature extraction module, used for inputting the first text vector into a first decoder module of the first network model to obtain a first feature vector of a first dimension, wherein the first feature vector is used for the first network model to generate a reply text; A vector dimension reduction module, used for performing dimension reduction processing on the first text vector and the first feature vector respectively, to obtain a second text vector of a second dimension and a second feature vector of a second dimension; The text review module is used to perform feature fusion on the second text vector and the second feature vector, and then input the result into the second decoder module of the second network model to generate the review result of the reply text. The first network model and the second network model constitute a two-way text processing model.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the text review method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the text review method according to any one of claims 1 to 7 is implemented.

Images