Method, medium, apparatus and computing device for determining importance degree of words in text

Abstract

Embodiments of the present disclosure provide a method, medium, device and computing equipment for determining the importance of words in text. A text recognition model having a neural network algorithm structure is used to recognize a text vector of a target file, and a probability value output by the text recognition model to represent the recognition result is used to complete forward propagation of the text vector. Then, the probability value is back propagated from an output layer to an input layer of the neural network layer by layer.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of artificial intelligence, and more particularly, embodiments of the present disclosure relate to a method, medium, device and computing device for determining importance degree of words in text. BACKGROUND

[0002] Currently, some text processing tasks involve a subtask of determining important words in text. The so-called important words can refer to words that contribute more to the meaning of the text.

[0003] For example, in the task of generating an adversarial text based on a standard text, one of the subtasks involves determining one or more important words in the standard text.

[0004] Therefore, there is a need for a technical solution for determining the importance degree of words in text. SUMMARY

[0005] In this context, embodiments of the present disclosure aim to provide a method, medium, device and computing device for determining the importance degree of words in text, so as to obtain more effective adversarial text for optimizing the training of a text recognition model.

[0006] In a first aspect of the embodiments of the present disclosure, a method for determining the importance degree of words in text is provided, comprising the following steps:

[0007] obtaining a text vector of a target text and a probability value for representing a recognition result of a text recognition model on the text vector, wherein the text vector is obtained by merging the same dimensions of word vectors of each word in the target text based on a vector merging algorithm, and the algorithm structure of the text recognition model comprises a neural network, which comprises an input layer, a plurality of intermediate layers and an output layer;

[0008] According to the contribution proportion of each neuron in a previous intermediate layer of the output layer to the output layer, the probability value is proportionally assigned to each neuron in the intermediate layer, and the assignment result is a contribution value corresponding to each neuron, wherein the contribution proportion of each neuron in the intermediate layer to the output layer comprises the parameter value of each neuron in the intermediate layer multiplied by the connection weight value between the neuron and the output layer.

[0009] select any intermediate layer, and proportionally distribute the contribution value corresponding to each neuron in the intermediate layer to each neuron in the previous layer according to the contribution proportion of each neuron in the previous layer to each neuron in the intermediate layer, wherein the previous layer includes the input layer or other intermediate layers, and the contribution proportion of a first neuron in the previous layer to a second neuron in the intermediate layer includes the parameter value of the first neuron multiplied by the connection weight value between the first neuron and the second neuron;

[0010] According to the inverse operation of the vector merging algorithm, the contribution value of each dimension is distributed to the dimension of the word vector of each word, wherein the contribution value corresponding to each neuron in the input layer is the contribution value of the dimension corresponding to the neuron;

[0011] The contribution values corresponding to each dimension of the word vector of each word are summed respectively, and the importance degree representation value of the word is obtained according to the calculation result.

[0012] In an embodiment of the present disclosure, wherein the step of obtaining the importance degree representation value of the word according to the calculation result further comprises the following steps:

[0013] The absolute value of the calculation result is taken as the importance degree representation value of the word.

[0014] In an embodiment of the present disclosure, the target text is a standard text for training a text recognition model, and the method further comprises an adversarial text generation step, which comprises:

[0015] Based on the importance degree representation value of each word in the standard text, select several important words as replacement words;

[0016] According to the words before and after each replacement word in the standard text, predict several candidate words corresponding to each replacement word;

[0017] According to the candidate words corresponding to at least part of the replacement words, perform a word replacement operation on the standard text to obtain several candidate texts;

[0018] Use the text recognition model to recognize each candidate text respectively, and determine several candidate texts meeting the adversarial condition as adversarial texts;

[0019] Wherein, the adversarial condition is that the recognition result of the text recognition model on the candidate text is different from the recognition result on the standard text.

[0020] In an embodiment of the present disclosure,

[0021] Wherein, the step of predicting several candidate words corresponding to each replacement word further comprises:

[0022] performing the following steps for each word to be replaced:

[0023] masking the position of the word to be replaced in the standard text, inputting the standard text into a Bert model, and outputting a set of prediction probability values corresponding to the mask;

[0024] determining, in order of prediction probability values from large to small, a plurality of words other than the word to be replaced as candidate words,

[0025] wherein the set of prediction probability values includes a plurality of prediction probability values corresponding to a plurality of words, and each prediction probability value corresponds to the probability of the word appearing at the position corresponding to the mask.

[0026] In an embodiment of the present disclosure, the text recognition model is used to recognize text in a target business scenario.

[0027] Before inputting the standard text into the Bert model, the method further includes:

[0028] training the Bert model in a target business scenario.

[0029] In an embodiment of the present disclosure, the method further includes the following steps: using the standard text as a target text, and using the first word to be replaced as a target word to be replaced in order of each word to be replaced, wherein the step of determining the adversarial text further includes the following steps:

[0030] for each candidate word corresponding to the target word to be replaced, replacing the target word to be replaced in the target text with the candidate word to obtain a candidate text corresponding to the target text;

[0031] using a text recognition model to recognize each candidate text corresponding to the target text to obtain a recognition result of each candidate text corresponding to the target text; and adding the candidate text meeting the adversarial condition in each candidate text corresponding to the target text to a set of available candidate texts;

[0032] if the set of available candidate texts is not empty, determining an adversarial text based on the set of available candidate texts, and ending the iteration;

[0033] if the set of available candidate texts is empty, reusing the candidate text meeting the preset difference condition in each candidate text corresponding to the target text as the target text, reusing the next word to be replaced as the target word to be replaced, and starting the next iteration;

[0034] The preset difference condition is that a difference between a prediction probability value corresponding to a recognition result of each candidate text in the target text and a prediction probability value corresponding to a recognition result of the standard text is maximum.

[0035] In an embodiment of the present disclosure, the standard text is a Chinese text, and the step of determining the candidate word further comprises:

[0036] For at least part of the word to be replaced, a plurality of Martian words corresponding to the word to be replaced are obtained as candidate words corresponding to the word to be replaced.

[0037] In an embodiment of the present disclosure, the standard text is a Chinese text, and the step of determining the candidate word further comprises:

[0038] For at least part of the word to be replaced, a plurality of Martian words corresponding to the word to be replaced are obtained as candidate words corresponding to the word to be replaced.

[0039] In an embodiment of the present disclosure, the standard text is a Chinese text, and the step of determining the candidate word further comprises:

[0040] For at least part of the word to be replaced, a plurality of Martian words corresponding to the word to be replaced are obtained as candidate words corresponding to the word to be replaced.

[0041] In an embodiment of the present disclosure, the standard text is a Chinese text, and the step of determining the candidate word further comprises:

[0042] For at least part of the word to be replaced, the word to be replaced is converted in form, including: at least part of the Chinese characters contained in the word to be replaced are split into components, and each component split out is taken as an independent character;

[0043] The word to be replaced after form conversion is taken as a corresponding candidate word.

[0044] In an embodiment of the present disclosure, the standard text is a Chinese text;

[0045] Before the word replacement operation is performed on the standard text according to the candidate words corresponding to at least part of the word to be replaced, the method further comprises:

[0046] For at least part of the word to be replaced, a Chinese character picture corresponding to each Chinese character included in the word to be replaced is obtained;

[0047] For each Chinese character picture corresponding to the word to be replaced, the Chinese character picture is input into a convolutional neural network, and a feature representation corresponding to the Chinese character picture is output;

[0048] The feature representation corresponding to the Chinese character picture is input into a Chinese character picture classification model; the Chinese character picture classification model takes different Chinese characters as different categories;

[0049] The prediction probability of the Chinese character picture belonging to each category output by the Chinese character picture classification model is obtained, and a plurality of Chinese characters other than the to-be-replaced word are sequentially determined as candidate words in descending order of prediction probability.

[0050] In a second aspect of the embodiments of the present disclosure, a device for determining the importance degree of a word in a text is provided, comprising:

[0051] An acquisition module is configured to acquire a text vector of a target text and a probability value for representing a recognition result of the text recognition model on the text vector, wherein the text vector is obtained by merging the same dimensions of word vectors of each word in the target text based on a vector merging algorithm, and the algorithm structure of the text recognition model comprises a neural network, and the neural network comprises an input layer, a plurality of intermediate layers, and an output layer;

[0052] A first distribution module is configured to distribute the probability value to each neuron in a previous intermediate layer of the output layer in proportion to the contribution proportion of each neuron in the intermediate layer to the output layer, and the distribution result is a contribution value corresponding to each neuron, wherein the contribution proportion of each neuron in the intermediate layer to the output layer comprises the parameter value of each neuron in the intermediate layer multiplied by the connection weight value between the neuron and the output layer;

[0053] A second distribution module is configured to select any intermediate layer, and distribute the contribution value corresponding to each neuron in the intermediate layer to each neuron in a previous layer of the intermediate layer in proportion to the contribution proportion of each neuron in the previous layer to each neuron in the intermediate layer, wherein the previous layer comprises the input layer or other intermediate layers, and the contribution proportion of a first neuron in the previous layer to a second neuron in the intermediate layer comprises the parameter value of the first neuron multiplied by the connection weight value between the first neuron and the second neuron;

[0054] A third distribution module is configured to distribute the contribution value of each dimension to the dimension of the word vector of each word according to the inverse operation of the vector merging algorithm, wherein the contribution value corresponding to each neuron in the input layer is the contribution value of the dimension corresponding to the neuron;

[0055] A determination module is configured to perform summation calculation on the contribution values corresponding to each dimension of the word vector of each word respectively, and obtain an importance degree representation value of the word according to the calculation result.

[0056] In a third aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program, and the program is executed by a processor to implement a method for determining the importance degree of a word in a text.

[0057] In a fourth aspect of the embodiments of the present disclosure, a computing device is provided, comprising a memory, a processor; the memory is configured to store computer instructions executable on the processor, and the processor is configured to implement the method for determining the importance degree of a word in a text when executing the computer instructions.

[0058] In the above technical solution, the text recognition model having the neural network algorithm structure is used to recognize the text vector of the target file, the probability value output by the text recognition model for representing the recognition result is used to complete the forward propagation of the text vector. Then, the probability value is reversely propagated layer by layer from the output layer of the neural network to the input layer, and then reversely propagated from the input layer to each vector dimension.

[0059] In the reverse propagation process, the probability value is actually proportionally distributed according to the contribution proportion of each neuron in the previous intermediate layer of the output layer to the output layer, to obtain the contribution value (which can be understood as the contribution to the forward propagation) corresponding to each neuron in the previous intermediate layer. Then, for any intermediate layer, the contribution value corresponding to each neuron in the intermediate layer is further distributed to the neurons in the previous layer according to the contribution proportion of each neuron in the previous layer to each neuron in the intermediate layer, so that the probability value can be reversely propagated layer by layer to the input layer to obtain the contribution value corresponding to each neuron in the input layer. Further, by using the one-to-one correspondence between each neuron in the input layer and each dimension, the contribution value corresponding to each neuron in the input layer is assigned to the corresponding dimension to complete the reverse propagation.

[0060] Since the text vector is obtained by merging the same dimensions of the word vectors of each word in the target text based on the vector merging algorithm, the contribution value corresponding to each dimension can be distributed to the dimension of the word vector of each word based on the inverse operation of the vector merging algorithm. Further, it can be considered that the sum of the contribution values corresponding to each dimension of the word vector of each word can represent the contribution degree of the word to the recognition result, and the greater the contribution degree, the more important the word in the target text. BRIEF DESCRIPTION OF DRAWINGS

[0061] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0062] Figure 1 An exemplary method for determining the importance degree of a word in a text is provided;

[0063] Figure 2 An exemplary structure of a neural network is provided;

[0064] Figure 3 An exemplary structure of another neural network is provided.

[0065] Figure 4 An exemplary flow of a method for generating an adversarial text is provided.

[0066] Figure 5 An exemplary flow of a method for determining an adversarial text is provided.

[0067] Figure 6 An exemplary structure of a device for determining the importance degree of a word in a text is provided.

[0068] Figure 7 is a schematic diagram of a computer-readable storage medium provided by the present disclosure.

[0069] Figure 8 is a structural schematic diagram of a computing device provided by the present disclosure.

[0070] In the drawings, identical or corresponding reference signs indicate identical or corresponding parts. The number of elements in the drawings is used for illustration only and not for limitation, and any naming is only used for differentiation and does not have any limiting meaning. DETAILED DESCRIPTION

[0071] The principles and spirits of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only to enable those skilled in the art to better understand and implement the present disclosure, and do not limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0072] Those skilled in the art know that the embodiments of the present disclosure can be implemented as a system, a device, an apparatus, a method or a computer program product. Therefore, the present disclosure can be embodied in the form of a complete hardware, a complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0073] According to the embodiments of the present disclosure, a method, medium, device and computing device for determining the importance degree of a word in a text are provided.

[0074] The principles and spirits of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only to enable those skilled in the art to better understand and implement the present disclosure, and do not limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0075] In the method for determining the importance degree of a word in a text provided in the present disclosure, a text recognition model having a neural network algorithm structure is used to recognize the text vector of a target file, and the probability value output by the text recognition model to represent the recognition result is subjected to forward propagation. Then, the probability value is subjected to backward propagation from the output layer to the input layer of the neural network, and then from the input layer to each vector dimension.

[0076] In the backward propagation process, the probability value is actually proportionally distributed according to the contribution proportion of each neuron in the previous intermediate layer of the output layer to the output layer, to obtain the contribution value (which can be understood as the contribution to the forward propagation) corresponding to each neuron in the previous intermediate layer. Then, for any intermediate layer, the contribution value corresponding to each neuron in the intermediate layer is proportionally distributed to the neurons in the previous layer according to the contribution proportion of each neuron in the previous layer to each neuron in the intermediate layer, so that the probability value can be backward propagated layer by layer to the input layer to obtain the contribution value corresponding to each neuron in the input layer. Furthermore, by using the one-to-one correspondence between each neuron in the input layer and each dimension, the contribution value corresponding to each neuron in the input layer is assigned to the corresponding dimension, and the backward propagation is completed.

[0077] Since the text vector is obtained by merging the same dimensions of the word vectors of each word in the target text based on the vector merging algorithm, the contribution value corresponding to each dimension can be distributed to the dimension of the word vector of each word based on the inverse operation of the vector merging algorithm. Furthermore, it can be considered that the sum of the contribution values corresponding to each dimension of the word vector of each word can represent the contribution degree of the word to the recognition result, and the greater the contribution degree, the more important the word in the target text.

[0078] Figure 1 An exemplary method for determining the importance degree of a word in a text is provided, which includes:

[0079] S100: obtaining a text vector of a target text and a probability value representing the recognition result of the text vector by a text recognition model.

[0080] S102: proportionally distributing the probability value to each neuron in the previous intermediate layer of the output layer according to the contribution proportion of each neuron in the intermediate layer to the output layer, and the distribution result is a contribution value corresponding to each neuron.

[0081] S104: selecting any intermediate layer, and proportionally distributing the contribution value corresponding to each neuron in the intermediate layer to each neuron in the previous layer according to the contribution proportion of each neuron in the previous layer to each neuron in the intermediate layer.

[0082] S106: According to the inverse operation of the vector merging algorithm, the contribution value of each dimension is assigned to the dimension of the word vector of each word.

[0083] S108: The contribution values corresponding to each dimension of the word vector of each word are summed up respectively, and the importance degree representation value of the word is obtained according to the calculation result.

[0084] Generally, the target text includes multiple words, and the same dimensions of the word vectors of each word in the target text can be merged based on the vector merging algorithm to obtain a text vector corresponding to the target text. For example, each word vector includes N dimensions, and the text vector also includes N dimensions, and set i = 1, 2, …, N, the i-th dimension of the text vector is obtained by merging the i-th dimensions of each word vector. In this paper, the vector merging algorithm is not limited, and the common vector merging algorithm can be understood as that the same dimensions of each word vector are weighted and calculated according to certain weights, and the weights of the same dimensions in different word vectors are usually different.

[0085] Figure 1 The text recognition model in the method can have a neural network algorithm structure. The neural network includes an input layer, a plurality of intermediate layers (or called hidden layers), and an output layer arranged in sequence.

[0086] The process of inputting the text vector into the text recognition model for recognition can be considered as a process of forward propagation of the text vector. In the forward propagation process, the input layer can map the text vector into a feature vector and input it to the next intermediate layer. Each intermediate layer is used to analyze the input feature vector to obtain a new feature vector, and input the new feature vector to the next intermediate layer or the output layer. And the output layer can calculate the probability value representing the recognition result of the model according to the input feature vector.

[0087] Generally, it can be considered that the output layer includes a single neuron, and the input layer and the intermediate layer include a plurality of neurons. The neuron can be understood as a processing unit for processing vectors, and each neuron has its own parameter value.

[0088] Each neuron in the previous intermediate layer of the output layer has a connection weight value with the output layer. Each neuron in the input layer has a one-to-one connection relationship with each neuron in the next intermediate layer, and the two connected neurons also have a connection weight value. Each neuron in the adjacent previous intermediate layer also has a one-to-one connection relationship with each neuron in the next intermediate layer, and the two connected neurons also have a connection weight value.

[0089] During the forward propagation process, each neuron calculates the input vector according to its own parameter value and the connection weight value between itself and the neurons in the next layer, and outputs the calculation result. Among them, the input of the neuron in the input layer is the text vector, and the output is the feature vector. The input of the neuron in the intermediate layer is the feature vector, and the output is also the feature vector. The input of the neuron in the output layer is the feature vector, and the output is the recognition result.

[0090] Figure 2 An exemplary structure of a neural network is provided. As shown in Figure 2 , the neural network includes an input layer, one intermediate layer, and an output layer. The input layer includes three neurons, the intermediate layer includes three neurons, and the output layer includes one neuron. Each neuron in the input layer can have a connection relationship with each neuron in the intermediate layer, and each neuron in the intermediate layer can also have a connection relationship with each neuron in the output layer. Each set of neurons has a connection weight value corresponding to the connection relationship.

[0091] Figure 3 An exemplary structure of another neural network is provided. Figure 3 Based on Figure 2 , a one-to-one correspondence relationship between each dimension in the text vector and each neuron in the input layer is added.

[0092] After introducing the above-mentioned forward propagation process, the concept of the method shown in Figure 1 is explained as follows:

[0093] The importance degree of each word in the target text is actually the contribution size of each word in the target text to the meaning of the target text. The word with greater contribution is more important, and the probability value output by the text recognition model to represent the recognition result of the target text can be understood as the classification probability of the meaning of the target text, for example, the classification probability includes whether the meaning expression of the target text is positive or negative emotion. Therefore, if a method can be designed to find out the influence degree of each word in the target text on the probability value output by the text recognition model, the importance degree of the word can be represented by the influence degree.

[0094] According to the further analysis in the above method, it is found that, in the process of the text recognition model recognizing the target text (i.e. in the forward propagation process), if it is considered that different mathematical representations of the target text are converted and forward propagated for different layers, different neurons in the same layer have certain contributions to the forward propagation, and since the parameter values of different neurons in the same layer and the connection weight values of neurons in the next layer are different, the contribution degrees of different neurons in the same layer to the forward propagation are different. Then, by reverse thinking, the probability value output by the text recognition model for representing the recognition result can be regarded as the sum of the contribution values of each layer of neurons, and the probability value is back propagated from the output layer to the text vector. In the back propagation process, the probability value is distributed to different neurons in the same layer. Among them, the distribution rule is that the neuron in the same layer that has a greater contribution proportion to the next layer gets more contribution value.

[0095] Therefore, for the contribution proportion of each neuron of the last intermediate layer to the output layer, the following rule can be used to calculate: the parameter value of each neuron in the intermediate layer is multiplied by the connection weight value between the neuron and the output layer.

[0096] In addition, for the input layer and the non-last intermediate layer, the contribution proportion of each first neuron to a second neuron in the next layer can be calculated according to the following rule: the parameter value of the first neuron is multiplied by the connection weight value between the first neuron and the second neuron.

[0097] After obtaining the contribution value corresponding to each neuron in the input layer, since each neuron in the input layer corresponds to each dimension, the contribution value corresponding to each dimension in the text vector can also be obtained, wherein the each dimension is the dimension of the vector.

[0098] Since each dimension in the text vector of the target text is based on the merging of the same dimensions of the word vectors, the contribution value corresponding to each dimension of the text vector can be distributed to the dimension of each word vector based on the inverse operation of the vector merging algorithm. For example, if the vector merging algorithm is understood as the weighted sum calculation of the same dimensions of the word vectors according to certain weights, the inverse operation of the vector merging algorithm can be understood as: distributing the contribution value according to the different weights of the same dimensions in different word vectors.

[0099] Then, after obtaining the contribution value corresponding to each dimension in the word vector of each word, the sum of the contribution values of each dimension in the word vector of the same word can be taken as the contribution value corresponding to the word.

[0100] In some embodiments, the contribution value corresponding to each word can be positive, and thus the contribution value corresponding to the word can be taken as the importance degree representation value of the word.

[0101] In some other embodiments, the contribution value corresponding to each word can be positive or negative. A positive contribution value can be understood as a contribution degree of increasing the probability value of the model output, and a negative contribution value can be understood as a contribution degree of reducing the probability value of the model output. However, it is not necessary to care whether the contribution of each word increases or reduces the probability value, but only to care about the influence size of the contribution of the word on the probability value, and thus the absolute value of the summation calculation result (contribution value) corresponding to each word can be taken as the importance degree representation value of the word.

[0102] In addition, the present disclosure also provides a generation scheme of an adversarial text.

[0103] An adversarial text generally needs to meet two requirements. One of the requirements is that the adversarial text is a text obtained by adding subtle interference to a standard text. The subtle interference described herein can refer to interference that is not easily perceived by humans, in other words, the understanding of the meaning of the standard text by humans is basically consistent with the understanding of the meaning of the adversarial text. For example, the standard text is “the room is a bit old and should be redecorated, otherwise it will go downhill”, and the adversarial sample obtained based on the standard text can be “the room is a bit old and should be redecorated, otherwise it will go downhill”. It can be seen that in the above example, the meaning of the standard text is almost consistent with the meaning of the adversarial text.

[0104] The other requirement that the adversarial text generally needs to meet is that the adversarial text can cause the text recognition model to make a mistake, that is, the recognition result of the text recognition model for the standard text is different from the recognition result of the text recognition model for the adversarial text. Still following the above example, assuming that the text recognition model is used to classify whether the emotion expressed by the text is positive or negative, then if the recognition result of the text recognition model for the standard text “the room is a bit old and should be redecorated, otherwise it will go downhill” is negative emotion, the recognition result of the text recognition model for the adversarial text “the room is a bit old and should be redecorated, otherwise it will go downhill” should be positive emotion.

[0105] Therefore, only the adversarial text meeting the above two requirements is a qualified adversarial text.

[0106] In order to obtain qualified adversarial text, the technical solution provided by the present disclosure is to take a plurality of words in the standard text used for training the text recognition model as replacement words, and predict a plurality of candidate words that can be used to replace the replacement words according to the context of the replacement words in the standard text. After replacing the replacement words with the candidate words, the candidate text is obtained, and the candidate text is input into the text recognition model for recognition. If the recognition result of the candidate text by the text recognition model is different from the recognition result of the standard text, it indicates that the candidate text is a qualified adversarial text.

[0107] Through the above technical solution, since the context of the replacement words in the standard text is considered to determine the candidate words, the meaning of the obtained candidate text is very close to that of the standard text, and the candidate text meeting the adversarial condition is taken as the adversarial text. On the one hand, such adversarial text is very close in meaning to the standard text. On the other hand, such adversarial text is also sufficient to cause the text recognition model before optimization to make a mistake. Using such adversarial text to optimize and train the text recognition model makes the text recognition model often able to accurately distinguish between two different categories of text with very close meanings and output different recognition results for the two different categories of text, respectively.

[0108] Figure 4 An exemplary method for generating adversarial text is provided, including the following steps:

[0109] S400: obtaining a standard text used for training a text recognition model, and determining a plurality of replacement words from each word included in the standard text.

[0110] S402: predicting a plurality of candidate words corresponding to each replacement word according to the words before and after each replacement word in the standard text.

[0111] S404: performing a word replacement operation on the standard text according to the candidate words corresponding to at least part of the replacement words to obtain a plurality of candidate texts.

[0112] S406: using the text recognition model to recognize each candidate text, respectively, and determining a plurality of candidate texts meeting the adversarial condition as adversarial texts.

[0113] The text recognition model described herein can be any artificial intelligence model capable of implementing a text recognition task. A common text recognition task is a text classification task, which can be a binary classification task or a multi-classification task. The present specification does not make specific limitations on this.

[0114] The algorithm structure of the text recognition model is also not specifically limited in the present specification. For example, the algorithm structure of the text recognition model can include a TEXTCNN convolutional neural network and a single-layer bidirectional LSTM network.

[0115] The standard text mentioned in this article can refer to text that conforms to natural language rules and is used to train a text recognition model. Conforming to natural language rules usually means that the text is fluent and free of typos.

[0116] Before generating adversarial text based on standard text, it is usually necessary to perform word segmentation on the standard text. In step S400, there can be various rules for determining replacement words from the various words included in the standard text. For example, one or more replacement words can be randomly determined from the various words included in the standard text.

[0117] In some embodiments, words that contribute significantly to the meaning of the standard text can be identified from among the words included in the standard text as words to be replaced. These words, which contribute significantly to the meaning of the standard text, refer to important words or keywords in the standard text. Since text recognition models typically analyze text based on important words, replacing important words in the standard text makes it easier for the text recognition model to misjudge the text.

[0118] Specifically, the importance level representation value of each word in the standard text can be calculated, where the importance level representation value of each word is positively correlated with the importance of the word to the content meaning of the standard text; then, a number of words can be determined as words to be replaced in descending order of importance level representation values.

[0119] Those skilled in the art can conceive of various ways to calculate the importance value of each word in a standard text. Besides... Figure 4 The method for determining the importance of words in the text shown (in which case the text recognition model needs to have a neural network algorithm structure) is illustrated in this disclosure. This disclosure also exemplarily provides a method for calculating the importance representation values ​​of each word included in a standard text (in which case the text recognition model may or may not have a neural network algorithm structure):

[0120] This disclosure calculates the training loss matrix of the text recognition model based on the recognition results of the standard text and the recognition labels of the annotated text, and then calculates the gradient matrix based on the training loss matrix. Next, this disclosure calculates the importance representation vector by performing a L2 norm calculation on the gradient matrix. The importance representation vector contains dimension values ​​that correspond one-to-one with each word in the standard text, with each dimension value serving as a representation of the importance level of the corresponding word.

[0121] Furthermore, the gradient matrix can be input into the sign function to obtain the output sign matrix, and then the sign matrix can be used as the input for the L2 norm calculation to obtain the importance representation vector.

[0122] The method for calculating the importance representation value in the above example can be understood as utilizing the FGSM algorithm. In step S402, several candidate words are predicted for each word to be replaced based on its context in the standard text. Each candidate word can be considered as a word that can play a similar role in the meaning of the word to be replaced in the standard text. Using such candidate words to replace the word to be replaced can minimize the change in the meaning of the text.

[0123] There are various methods that can be used to implement step S402. This disclosure provides an exemplary method for predicting several candidate words corresponding to each word to be replaced based on the words preceding and following each word in the standard text. The following steps can be performed for each word to be replaced:

[0124] After masking the position of the word to be replaced in the standard text, the standard text is input into the BERT model, which outputs a set of predicted probability values ​​corresponding to the masking mark. The set of predicted probability values ​​includes predicted probability values ​​for multiple words, where each word's predicted probability is used to predict the probability that the word will appear at the position corresponding to the masking mark. Several words other than the word to be replaced can be selected as candidate words in descending order of predicted probability values.

[0125] In the example above, it can be understood that an MLM task is implemented using a BERT-based model. The position of the word to be replaced in the text input to the BERT model is masked, and BERT predicts the word appearing at the masked position. For example, if the original text is "The room is a bit old and should be renovated, otherwise it will decline," after masking the position of "renovate" using a masking tag, the original text is input into the BERT model for MLM generation. The BERT model can then output the prediction of the word appearing at the masked position in the original text.

[0126] Furthermore, the text recognition model can be used to recognize text within a target business scenario. Therefore, before inputting the standard text into the BERT model, the BERT model can be trained using text from the target business scenario for scenario-adaptive training. In this way, the BERT model can have predictive capabilities that are more suitable for the target business scenario.

[0127] In step S404, some or all of the words to be replaced in the standard text can be replaced with corresponding candidate words to obtain several candidate texts. It should be noted that the fewer words to be replaced, the closer the meaning of the candidate text is to the standard text.

[0128] The alternative text is not necessarily a counter-text, so the text recognition model does not necessarily misjudge the alternative text. Therefore, in step S406, each alternative text can be input into the text recognition model for recognition, and if the recognition result is misjudged, it means that the alternative text can be a counter-text. The recognition result is misjudged, which means that the recognition result of the text recognition model for the alternative text is different from the recognition result of the standard text.

[0129] As shown in Figure 5 To obtain qualified counter-texts with as few replaced words as possible, the present disclosure exemplarily provides a method for determining counter-texts, which comprises the following steps:

[0130] The standard text is taken as the target text, and the first to-be-replaced word is taken as the target to-be-replaced word in the order of the to-be-replaced words. The following steps are iteratively performed:

[0131] For each alternative word corresponding to the target to-be-replaced word, the target to-be-replaced word in the target text is replaced with the alternative word to obtain one alternative text corresponding to the target text;

[0132] The text recognition model is used to recognize each alternative text corresponding to the target text respectively to obtain the recognition result of each alternative text corresponding to the target text. The alternative texts in the target text that meet the counter condition are added to the available alternative text set;

[0133] If the available alternative text set is not empty, a counter-text is determined based on the available alternative text set, and the iteration is ended;

[0134] If the available alternative text set is empty, the alternative text in the target text that meets the preset difference condition is taken as the target text again, and the next to-be-replaced word is taken as the target to-be-replaced word to start the next iteration;

[0135] The preset difference condition is that the difference between the prediction probability value corresponding to the recognition result of the alternative text and the prediction probability value corresponding to the recognition result of the standard text is the largest among the alternative texts corresponding to the target text.

[0136] The counter-text obtained by the method shown in Figure 5 is obtained based on the minimum modification of the standard text. Such counter-texts can not only meet the requirement of making the text recognition model misjudge, but also have as little difference as possible from the meaning of the standard text. Therefore, the above counter-texts are relatively high-quality counter-texts.

[0137] In addition, when the standard text is in Chinese, in order to obtain adversarial text that is also in Chinese, the following embodiments can be further extended based on the Figure 4 method flow shown. For the word replacement methods used in the following embodiments, from the perspective of humans, the meaning of the text will not change significantly, but for the text recognition model, the expression form of the text will change greatly, which is likely to cause great interference to the model's recognition and lead to misjudgment by the model.

[0138] In some embodiments, before performing the word replacement operation on the standard text according to the alternative words corresponding to at least some of the words to be replaced, for at least some of the words to be replaced, several Martian words corresponding to the word to be replaced can be obtained as the alternative words corresponding to the word to be replaced.

[0139] For example, if the standard text is "It's just that breakfast needs to be improved", after replacing "just" with the Martian alternative word "僦湜", the alternative text obtained is "僦湜 breakfast needs to be improved".

[0140] In some embodiments, before performing the word replacement operation on the standard text according to the alternative words corresponding to at least some of the words to be replaced, for at least some of the words to be replaced, the pinyin corresponding to the word to be replaced can be obtained as the alternative word corresponding to the word to be replaced.

[0141] For example, if the standard text is "You are very handsome", after replacing "handsome" with the pinyin alternative word "shuai", the alternative text obtained is "You are very shuai".

[0142] In some embodiments, before performing the word replacement operation on the standard text according to the alternative words corresponding to at least some of the words to be replaced, for at least some of the words to be replaced, several homophonic words corresponding to the word to be replaced can be obtained as the alternative words corresponding to the word to be replaced.

[0143] For example, if the standard text is "Have dinner at 6 o'clock", after replacing "dinner" with the homophonic alternative word "碗", the alternative text obtained is "Have 碗 at 6 o'clock".

[0144] In some embodiments, before performing the word replacement operation on the standard text according to the alternative words corresponding to at least some of the words to be replaced, for at least some of the words to be replaced, the form of the word to be replaced can be converted. The form conversion includes: splitting at least some of the Chinese characters included in the word to be replaced into components, and taking each split component as an independent character. Then, the word to be replaced after form conversion can be used as the corresponding alternative word.

[0145] For example, if the standard text is "He is sweating", after splitting "汗" into components to get "氵干", the alternative text obtained is "He is out of 氵干".

[0146] In some embodiments, before performing the word replacement operation on the standard text according to the candidate words corresponding to at least part of the words to be replaced, the method can further include: obtaining a picture of each Chinese character included in the word to be replaced; inputting the picture of each Chinese character corresponding to the word to be replaced into a convolutional neural network to output a feature representation corresponding to the picture of each Chinese character; inputting the feature representation corresponding to the picture of each Chinese character into a Chinese character picture classification model, wherein the Chinese character picture classification model classifies different Chinese characters as different classes; obtaining a predicted probability of the picture of each Chinese character belonging to each class output by the Chinese character picture classification model; and determining, in descending order of the predicted probability, a plurality of Chinese characters other than the word to be replaced as candidate words.

[0147] In this way, the word replacement is actually based on the features of the Chinese characters in the visual aspect. Further, the convolutional neural network described above can be a Tianzi grid-convolutional neural network, in which the output of the last hidden layer is a 2*2 matrix representing the Tianzi grid structure of a Chinese character.

[0148] Figure 6 An exemplary structure of a device for determining the importance of a word in a text is provided, including:

[0149] The obtaining module 601 obtains a text vector of a target text and a probability value representing the recognition result of the text recognition model on the text vector, wherein the text vector is obtained by merging the word vectors of each word in the target text in the same dimension based on a vector merging algorithm, and the algorithm structure of the text recognition model includes a neural network, which includes an input layer, a plurality of intermediate layers, and an output layer.

[0150] The first distribution module 602 distributes the probability value to each neuron in the previous intermediate layer of the output layer in proportion to the contribution ratio of each neuron in the intermediate layer to the output layer, and the distribution result is a contribution value corresponding to each neuron, wherein the contribution ratio of each neuron in the intermediate layer to the output layer includes the parameter value of each neuron in the intermediate layer multiplied by the connection weight value between the neuron and the output layer.

[0151] The second distribution module 603 selects any intermediate layer and distributes the contribution value corresponding to each neuron in the intermediate layer to each neuron in the previous layer in proportion to the contribution ratio of each neuron in the previous layer to each neuron in the intermediate layer, wherein the previous layer includes the input layer or other intermediate layers, and the contribution ratio of a first neuron in the previous layer to a second neuron in the intermediate layer includes the parameter value of the first neuron multiplied by the connection weight value between the first neuron and the second neuron.

[0152] a third distribution module 604 configured to distribute the contribution value of each dimension to the dimension of the word vector of each word according to an inverse operation of the vector merging algorithm, wherein the contribution value corresponding to each neuron in the input layer is the contribution value of the dimension corresponding to the neuron;

[0153] a determination module 605 configured to sum the contribution values corresponding to each dimension of the word vector of each word respectively, and obtain a representation value of the importance degree of the word according to a result of the calculation.

[0154] In some embodiments, the determination module 605 takes an absolute value of the result of the calculation as the representation value of the importance degree of the word.

[0155] In some embodiments, the target text is a standard text used for training a text recognition model, and the apparatus further comprises:

[0156] an adversarial text generation module 606 configured to select a plurality of important words as replacement words based on the representation value of the importance degree of each word in the standard text, predict a plurality of candidate words corresponding to each replacement word according to the words before and after each replacement word in the standard text, perform a word replacement operation on the standard text according to the candidate words corresponding to at least part of the replacement words, obtain a plurality of candidate texts, and identify each candidate text using the text recognition model to determine a plurality of candidate texts satisfying an adversarial condition as adversarial texts.

[0157] The adversarial condition is that the recognition result of the text recognition model on the candidate text is different from the recognition result on the standard text.

[0158] In some embodiments, the adversarial text generation module 606 performs the following steps for each replacement word: after marking the position of the replacement word in the standard text with a mask, inputs the standard text into a Bert model to output a set of prediction probability values corresponding to the mask, and in descending order of the prediction probability values, determines a plurality of words other than the replacement word as candidate words,

[0159] The set of prediction probability values includes a plurality of prediction probability values corresponding to a plurality of words, wherein each prediction probability value corresponding to a word is used to predict a probability that the word appears at the position corresponding to the mask.

[0160] In some embodiments, the text recognition model is used to recognize text in a target business scenario, and the adversarial text generation module 606 performs scenario adaptive training on the Bert model using text in the target business scenario before inputting the standard text into the Bert model.

[0161] In some embodiments, the adversarial text generation module 606 uses the standard text as the target text, and takes the first to-be-replaced word as the target to-be-replaced word in the order of the to-be-replaced words, and the step of determining the adversarial text further comprises the following steps which are iteratively performed:

[0162] for each candidate word corresponding to the target to-be-replaced word, replacing the target to-be-replaced word in the target text with the candidate word to obtain one candidate text corresponding to the target text;

[0163] using a text recognition model to recognize each candidate text corresponding to the target text respectively to obtain a recognition result of each candidate text corresponding to the target text, and adding the candidate texts in the target texts corresponding to the target text that meet the adversarial condition to a set of available candidate texts,

[0164] if the set of available candidate texts is not empty, determining the adversarial text based on the set of available candidate texts, and ending the iteration;

[0165] if the set of available candidate texts is empty, reusing the candidate text in the target texts corresponding to the target text that meets the preset difference condition as the target text, reusing the next to-be-replaced word as the target to-be-replaced word, and starting the next iteration,

[0166] wherein the preset difference condition is that among the candidate texts corresponding to the target text, the difference between the predicted probability value corresponding to the recognition result of the candidate text and the predicted probability value corresponding to the recognition result of the standard text is the largest.

[0167] In some embodiments, the standard text belongs to Chinese text, and the adversarial text generation module 606 obtains a number of Martian language words corresponding to at least part of the to-be-replaced words as candidate words corresponding to the to-be-replaced words.

[0168] In some embodiments, wherein the standard text belongs to Chinese text, the adversarial text generation module 606 obtains a number of Martian language words corresponding to at least part of the to-be-replaced words as candidate words corresponding to the to-be-replaced words.

[0169] In some embodiments, the standard text belongs to Chinese text, and the adversarial text generation module 606 obtains a number of Martian language words corresponding to at least part of the to-be-replaced words as candidate words corresponding to the to-be-replaced words.

[0170] In some embodiments, the standard text belongs to a Chinese text, and the adversarial text generation module 606 performs form conversion on at least part of the word to be replaced, and the form conversion includes: performing component splitting on at least part of the Chinese characters contained in the word to be replaced, and taking each component as an independent character; and taking the word to be replaced after form conversion as a corresponding candidate word.

[0171] In some embodiments, the standard text belongs to a Chinese text, and the adversarial text generation module 606, for at least part of the word to be replaced, obtains a Chinese character picture corresponding to each Chinese character included in the word to be replaced; for each Chinese character picture corresponding to the word to be replaced, inputs the Chinese character picture into a convolutional neural network to output a feature representation corresponding to the Chinese character picture; inputs the feature representation corresponding to the Chinese character picture into a Chinese character picture classification model, where the Chinese character picture classification model takes different Chinese characters as different classes; obtains a predicted probability that the Chinese character picture belongs to each class output by the Chinese character picture classification model, and sequentially determines a number of Chinese characters other than the word to be replaced as candidate words in descending order of the predicted probability.

[0172] It should be noted that although several units / modules or sub-units / modules of the apparatus are mentioned in the foregoing detailed description, such division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into a plurality of units / modules.

[0173] Figure 7 is a schematic diagram of a computer readable storage medium provided by the present disclosure, the medium 140 stores a computer program, and the program is executed by a processor to implement the method of any embodiment of the present disclosure.

[0174] The present disclosure also provides a computing device, including a memory, a processor; the memory is used to store computer instructions executable on the processor, and the processor is used to implement the method of any embodiment of the present disclosure when executing the computer instructions.

[0175] Figure 8 is a structural schematic diagram of a computing device provided by the present disclosure, the computing device 15 can include but is not limited to: a processor 151, a memory 152, a bus 153 connecting different system components including the memory 152 and the processor 151.

[0176] The memory 152 stores computer instructions that can be executed by the processor 131 to enable the processor 151 to perform the methods of any of the embodiments of the present disclosure. The memory 152 can include a random access memory unit RAM 1521, a cache memory unit 1522, and / or a read-only memory unit ROM 1523. The memory 152 can also include a program tool 1525 having a set of program modules 1524 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, in combination, which can include implementation of a network environment.

[0177] The bus 153 can include, for example, a data bus, an address bus, and a control bus, among others. The computing device 15 can also communicate with external devices 155, such as a keyboard, a Bluetooth device, and the like, through the I / O interface 154. The computing device 15 can also communicate with one or more networks, such as a local area network, a wide area network, a public network, and the like, through the network adapter 156. The network adapter 156 can also communicate with other modules of the computing device 15 through the bus 153.

[0178] Further, although operations of the methods of the present disclosure are described in a particular order in the figures, this is not required or implied in any way as to the order of the operations or that all operations be performed to achieve desirable results. Additionally or alternatively, certain steps can be omitted, combined into a single step, and / or separated into multiple steps.

[0179] While the spirit and principles of the present disclosure have been described with reference to several specific implementations, it is to be understood that the present disclosure is not limited to the specific implementations disclosed and that the division into aspects is not meant to imply that features from the aspects cannot be combined to benefit from the disclosure, but is merely for ease of presentation. The present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method of determining the degree of importance of a word in a text, wherein, The method comprises the following steps: obtaining a text vector of a target text and a probability value for representing a recognition result of the text vector by a text recognition model, wherein the text vector is obtained by merging the same dimensions of word vectors of each word in the target text based on a vector merging algorithm, and the algorithm structure of the text recognition model comprises a neural network, and the neural network comprises an input layer, a plurality of intermediate layers and an output layer; according to the contribution proportion of each neuron in a previous intermediate layer of the output layer to the output layer, the probability value is proportionally distributed to each neuron in the intermediate layer, and the distribution result is a contribution value corresponding to each neuron, wherein the contribution proportion of each neuron in the intermediate layer to the output layer comprises the parameter value of each neuron in the intermediate layer multiplied by the connection weight value between the neuron and the output layer; selecting any intermediate layer, and proportionally distributing the contribution value corresponding to each neuron in the intermediate layer to each neuron in a previous layer of the intermediate layer according to the contribution proportion of each neuron in the previous layer to each neuron in the intermediate layer, wherein the previous layer comprises the input layer or other intermediate layers, and the contribution proportion of a first neuron in the previous layer to a second neuron in the intermediate layer comprises the parameter value of the first neuron multiplied by the connection weight value between the first neuron and the second neuron; according to the inverse operation of the vector merging algorithm, the contribution value of each dimension is distributed to the dimension of the word vector of each word, wherein the contribution value corresponding to each neuron in the input layer is the contribution value of the dimension corresponding to the neuron; summing the contribution values corresponding to each dimension of the word vector of each word respectively, and obtaining an importance degree representation value of the word according to the calculation result.

2. The method of claim 1, wherein, The step of obtaining the importance degree representation value of the word according to the calculation result further comprises the following steps: taking the absolute value of the calculation result as the importance degree representation value of the word.

3. The method of claim 1 or 2, wherein the target text is a standard text for training the text recognition model, and the method further comprises an adversarial text generation step, which comprises: selecting a plurality of important words as replacement words based on the importance degree representation value of each word in the standard text; predicting a plurality of candidate words corresponding to each replacement word according to the words before and after each replacement word in the standard text; performing a word replacement operation on the standard text according to the candidate words corresponding to at least part of the replacement words to obtain a plurality of candidate texts; identifying each candidate text using the text recognition model to determine a plurality of candidate texts satisfying an adversarial condition as adversarial texts; wherein the adversarial condition is that the recognition result of the text recognition model on the candidate text is different from the recognition result on the standard text.

4. The method of claim 3, wherein, The step of predicting a plurality of candidate words corresponding to each replacement word further comprises: for each replacement word, performing the following steps: after marking the position of the replacement word in the standard text with a masking mark, inputting the standard text into a Bert model to output a set of prediction probability values corresponding to the masking mark; determining, in order of descending predicted probability values, a plurality of words other than the to-be-replaced word as candidate words, wherein the set of predicted probability values comprises predicted probability values corresponding to a plurality of words respectively, and each predicted probability value corresponds to a probability of a word appearing at a position corresponding to the mask mark.

5. The method of claim 4, wherein, The text recognition model is used for recognizing text in a target business scenario, wherein before inputting the standard text into the Bert model, the method further comprises: training the Bert model in the target business scenario.

6. The method of claim 3, wherein, The method further comprises the following steps: using the standard text as the target text, and taking the first to-be-replaced word as the target to-be-replaced word in order of the to-be-replaced words, wherein the step of determining the adversarial text further comprises the following steps which are iteratively executed: for each candidate word corresponding to the target to-be-replaced word, replacing the target to-be-replaced word in the target text with the candidate word to obtain one candidate text corresponding to the target text; using the text recognition model to recognize each candidate text corresponding to the target text respectively to obtain a recognition result of each candidate text corresponding to the target text, and adding a candidate text meeting the adversarial condition in the target text to a set of available candidate texts, if the set of available candidate texts is not empty, determining the adversarial text based on the set of available candidate texts, and ending the iteration; if the set of available candidate texts is empty, taking a candidate text meeting a preset difference condition in the target text as the target text again, taking the next to-be-replaced word as the target to-be-replaced word again, and starting the next iteration, wherein the preset difference condition is that among the candidate texts corresponding to the target text, a difference between a predicted probability value corresponding to a recognition result of the candidate text and a predicted probability value corresponding to a recognition result of the standard text is maximum.

7. The method of claim 3, wherein, The standard text is a Chinese text, and the step of determining the candidate words further comprises: for at least part of the to-be-replaced words, obtaining a plurality of Martian words corresponding to the to-be-replaced word as candidate words corresponding to the to-be-replaced word.

8. The method of claim 3, wherein, The standard text is a Chinese text, and the step of determining the candidate words further comprises: for at least part of the to-be-replaced words, obtaining a plurality of Martian words corresponding to the to-be-replaced word as candidate words corresponding to the to-be-replaced word.

9. The method of claim 3, wherein, The standard text is a Chinese text, and the step of determining the candidate words further comprises: for at least part of the to-be-replaced words, obtaining a plurality of Martian words corresponding to the to-be-replaced word as candidate words corresponding to the to-be-replaced word.

10. The method of claim 3, wherein the standard text is a Chinese text, and the step of determining the candidate words further comprises: for at least part of the to-be-replaced words, performing form conversion on the to-be-replaced word, the form conversion comprising: performing component splitting on at least part of Chinese characters contained in the to-be-replaced word, and taking each component split out as an independent character; taking the to-be-replaced word after form conversion as a corresponding candidate word.

11. The method of claim 3, wherein, The standard text is a Chinese text, and the step of determining the candidate words further comprises: For at least part of the word to be replaced, obtain a Chinese character picture corresponding to each Chinese character included in the word to be replaced; For each Chinese character picture corresponding to the word to be replaced, input the Chinese character picture into a convolutional neural network to output a feature representation corresponding to the Chinese character picture; Input the feature representation corresponding to the Chinese character picture into a Chinese character picture classification model, wherein the Chinese character picture classification model takes different Chinese characters as different classes; Obtain a predicted probability that the Chinese character picture belongs to each class output by the Chinese character picture classification model, and in descending order of the predicted probability, sequentially determine a plurality of Chinese characters other than the word to be replaced as candidate words.

12. A device for determining the importance degree of a word in a text, comprising: an obtaining module, which obtains a text vector of a target text and a probability value for representing a recognition result of a text recognition model on the text vector, wherein the text vector is obtained by merging dimensions of word vectors of respective words in the target text based on a vector merging algorithm, and the algorithm structure of the text recognition model comprises a neural network, which comprises an input layer, a plurality of intermediate layers, and an output layer; a first distribution module, which distributes the probability value to each neuron in a previous intermediate layer of the output layer in proportion to a contribution ratio of the neuron to the output layer, and the distribution result is a contribution value corresponding to each neuron, wherein the contribution ratio of each neuron in the intermediate layer to the output layer comprises a parameter value of the neuron multiplied by a connection weight value between the neuron and the output layer; a second distribution module, which selects any intermediate layer, and distributes the contribution value corresponding to each neuron in the intermediate layer to each neuron in a previous layer of the intermediate layer in proportion to a contribution ratio of each neuron in the previous layer to each neuron in the intermediate layer, wherein the previous layer comprises the input layer or another intermediate layer, and the contribution ratio of a first neuron in the previous layer to a second neuron in the intermediate layer comprises a parameter value of the first neuron multiplied by a connection weight value between the first neuron and the second neuron; a third distribution module, which distributes the contribution value of each dimension to a dimension of a word vector of a respective word based on an inverse operation of the vector merging algorithm, wherein the contribution value corresponding to each neuron in the input layer is a contribution value of a dimension corresponding to the neuron; a determination module, which performs summation calculation on the contribution values respectively corresponding to the dimensions of the word vector of each word, and obtains an importance degree representation value of the word based on a calculation result.

13. The device of claim 12, wherein the determination module takes an absolute value of the calculation result as the importance degree representation value of the word.

14. The device of claim 12 or 13, wherein the target text is a standard text for training the text recognition model, and the device further comprises: an adversarial text generation module, which selects a plurality of important words as words to be replaced based on the importance degree representation value of each word in the standard text. According to the words before and after each to-be-replaced word in the standard text, a plurality of candidate words corresponding to each to-be-replaced word are predicted; according to the candidate words corresponding to at least part of the to-be-replaced words, a word replacement operation is performed on the standard text to obtain a plurality of candidate texts; each candidate text is identified using the text recognition model, and a candidate text satisfying an adversarial condition is determined as an adversarial text; The adversarial condition is that the identification result of the text recognition model on the candidate text is different from the identification result on the standard text.

15. The apparatus of claim 14, wherein the adversarial text generation module performs the following steps for each to-be-replaced word: after marking the position of the to-be-replaced word in the standard text with a mask, inputting the standard text into a Bert model to output a set of prediction probability values corresponding to the mask; and in descending order of the prediction probability values, sequentially determining a plurality of words other than the to-be-replaced word as candidate words. wherein The set of prediction probability values includes a plurality of prediction probability values corresponding to the words, wherein each prediction probability value corresponds to the probability that the word appears at the position corresponding to the mask.

16. The apparatus of claim 15, wherein, The text recognition model is used to recognize text in a target business scenario. Before inputting the standard text into the Bert model, the adversarial text generation module performs scene adaptive training on the Bert model using text in the target business scenario.

17. The apparatus of claim 14, wherein the counter text generating module uses the standard text as a target text, and sequentially takes a first word to be replaced as a target word to be replaced, among the words to be replaced. The step of determining the adversarial text further includes the following steps which are iteratively performed: For each candidate word corresponding to the target to-be-replaced word, the target to-be-replaced word in the target text is replaced with the candidate word to obtain one candidate text corresponding to the target text; Each candidate text corresponding to the target text is identified using the text recognition model to obtain an identification result of each candidate text corresponding to the target text, and a candidate text satisfying the adversarial condition in each candidate text corresponding to the target text is added to a set of available candidate texts, If the set of available candidate texts is not empty, an adversarial text is determined based on the set of available candidate texts, and the iteration ends; If the set of available candidate texts is empty, a candidate text satisfying a preset difference condition in each candidate text corresponding to the target text is re-used as the target text, the next to-be-replaced word is re-used as the target to-be-replaced word, and the next iteration is started, The preset difference condition is that the difference between the prediction probability value corresponding to the identification result of the candidate text and the prediction probability value corresponding to the identification result of the standard text is the largest among the candidate texts corresponding to the target text.

18. The apparatus of claim 14, wherein, The standard text is a Chinese text, and the adversarial text generation module obtains a plurality of Martian language vocabularies corresponding to at least part of the to-be-replaced words as candidate words corresponding to the to-be-replaced words.

19. The apparatus of claim 14, wherein, The standard text is a Chinese text, and the adversarial text generation module obtains pinyin corresponding to at least part of the to-be-replaced words as candidate words corresponding to the to-be-replaced words.

20. The apparatus of claim 14, wherein, The standard text is Chinese text, and the adversarial text generation module obtains a plurality of homophonic words corresponding to the to-be-replaced word as candidate words corresponding to the to-be-replaced word.

21. The apparatus of claim 14, wherein the standard text is Chinese text, and wherein the adversarial text generation module is configured to perform a form transformation on at least some of the words to be replaced, and wherein the form transformation comprises: At least part of the Chinese characters included in the to-be-replaced word are decomposed into components, and each decomposed component is taken as an independent character. The to-be-replaced word after form conversion is taken as a corresponding candidate word.

22. The apparatus of claim 14, wherein, The standard text is Chinese text, and the adversarial text generation module obtains a picture of each Chinese character corresponding to the to-be-replaced word included in the to-be-replaced word; inputs the picture of each Chinese character corresponding to the to-be-replaced word into a convolutional neural network to output a feature representation corresponding to the picture of each Chinese character; inputs the feature representation corresponding to the picture of each Chinese character into a Chinese character picture classification model, wherein the Chinese character picture classification model takes different Chinese characters as different categories; obtains a predicted probability that the picture of each Chinese character belongs to each category output by the Chinese character picture classification model, and sequentially determines a plurality of Chinese characters other than the to-be-replaced word as candidate words in descending order of the predicted probability. 23.A computer readable storage medium having stored thereon a computer program, the program being executed by a processor to implement the method of any one of claims 1 to 11. 24.A computing device comprising a memory and a processor, the memory being configured to store computer instructions executable on the processor, and the processor being configured to implement the method of any one of claims 1 to 11 when executing the computer instructions.

Images