Text processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a text processing method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the steps of obtaining a target text, and extracting a preset number of first text units from the target text; first text unit features corresponding to the first text units are extracted respectively; extracting a second text unit feature corresponding to the first text unit based on an association relationship between other text units and each first text unit and the first text unit feature of the first text unit; based on each second text unit feature, determining a category label of the target text; the category label is used for representing whether the target text contains sensitive information or not. When the text unit features of each text unit are extracted, more reference information is provided for model output in combination with the incidence relation between the text units, the influence of training gradient disappearance in the model neural network deepening process is made up, and therefore the recognition effect of sensitive events is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a text processing method, device, electronic device and storage medium. Background Art

[0002] In the fast-paced modern society, the Internet is full of sensitive content. At the same time, a large number of daily news reports on sensitive Internet content can cover different types of sensitive content and important information, but also contain a lot of redundant or overlapping data, which makes information processing more and more difficult; at the same time, in the face of massive Internet data, sensitive information is overwhelmed by a large amount of other information. In order to better detect sensitive content events, an accurate and real-time method is needed to detect and classify network sensitive event texts.

[0003] With the development of deep learning technology, artificial neural networks have received more and more attention in the field of sensitive event recognition. The current solution is to use a specific deep learning model for sensitive event recognition, which has higher portability, but the specific model generally uses a single neural network, and there are many parameters that need to be adjusted during the training process of the specific model. With the increase in the number of iterations, the training gradient is prone to dispersion, and the single neural network framework has certain limitations. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above technical defects. The technical solutions provided by the embodiments of this application are as follows: In a first aspect, an embodiment of the present application provides a text processing method, comprising: Acquire a target text, and extract a preset number of first text units from the target text; Extracting first text unit features corresponding to each first text unit respectively; For each first text unit, based on the association relationship between other text units and the first text unit and the first text unit feature of the first text unit, the second text unit feature corresponding to the first text unit is extracted by the first feature extraction model; the other text units are other first text units in each first text unit except the first text unit; Based on the features of each second text unit, a category label of the target text is determined; the category label is used to indicate whether the target text contains sensitive information.

[0005] In an optional embodiment of the present application, the first feature extraction model includes a preset number of sub-feature extraction models cascaded in sequence; Extracting the second text unit feature corresponding to the first text unit by the first feature extraction model specifically includes: For each first text unit, a second text unit feature corresponding to the first text unit is extracted by using a corresponding sub-feature extraction model.

[0006] In an optional embodiment of the present application, for each sub-feature extraction model, the sub-feature extraction model includes a plurality of layer structures cascaded in sequence; each layer structure includes a first cycle unit and a second cycle unit; Extracting the second text unit feature corresponding to the first text unit through the corresponding sub-feature extraction model specifically includes: The first text unit is input into the first layer structure of the sub-feature extraction model until the output of the last layer structure of the sub-feature extraction model is obtained, and the output of the last layer structure is used as the second text unit feature.

[0007] In an optional embodiment of the present application, the output of each layer structure is obtained by: For each layer structure of the sub-feature extraction model, a first input feature corresponding to a first cycle unit of the layer structure is obtained, and a second input feature corresponding to a second cycle unit of the layer structure is obtained; Extracting a first input feature through a first cyclic unit of the layer structure to obtain a first output feature; Extracting the second input feature through the second cyclic unit of the layer structure to obtain a second output feature; The first output feature and the second output feature are merged to obtain a third output feature; The input features of the layer structure and the third output features are fused to obtain the output of the layer structure.

[0008] In an optional embodiment of the present application, for the first sub-feature extraction model, the first input feature of the first layer structure is the corresponding first text unit feature, and the first input feature of the other layer structures except the first layer structure is the output feature of the previous layer structure; for other feature extraction modules except the first sub-feature extraction model, the first input feature of the first layer structure is the corresponding first text unit feature and the first output feature of the first layer structure of the previous sub-feature extraction model, and the first input feature of the other layer structures except the first layer structure is the output feature of the previous layer structure and the first output feature of the layer structure corresponding to the previous sub-feature extraction model; For the last sub-feature extraction model, the second input feature of the first layer structure is the corresponding first text unit feature, and the second input feature of other layer structures except the first layer structure is the output feature of the previous layer structure; for other feature extraction modules except the last sub-feature extraction model, the second input feature of the first layer structure is the corresponding first text unit feature and the second output feature of the first layer structure of the next sub-feature extraction model, and the second input feature of other layer structures except the first layer structure is the output feature of the previous layer structure and the second output feature of the corresponding layer structure of the next sub-feature extraction model.

[0009] In an optional embodiment of the present application, extracting a preset number of first text units from the target text specifically includes: Segmenting the target text to obtain at least one second text unit; The obtained number of second text units is compared with a preset number, and if the number of second text units is not less than the preset number, the preset number of second text units are used as first text units according to the order in which the second text units appear in the target text; If the number of second text units is less than the preset number, third text units of the difference between the preset number and the second text units are obtained, and each second text unit and each third text unit are taken together as the first text unit.

[0010] In an optional embodiment of the present application, extracting first text unit features corresponding to each first text unit respectively includes: For each first text unit, input the first text unit into a preset word vector conversion model to obtain a first text vector matrix corresponding to the first text unit; The first text vector matrix is ​​convolved using a preset convolution kernel to obtain a first text unit feature of the first text unit.

[0011] In an optional embodiment of the present application, determining the category label of the target text based on each second text unit feature specifically includes: Based on each second text unit feature, obtaining text features corresponding to the target text; The text features are calculated through a preset activation function to obtain the text type probability distribution of the target text, and the category label of the target text is output based on the text type probability distribution.

[0012] In an optional embodiment of the present application, outputting a category label of the target text based on the text type probability distribution specifically includes: Obtain the maximum probability index value in the text type probability, and match the maximum probability index value with each category label stored in the preset database; The category label matched from the preset database is used as the category label of the target text.

[0013] In a second aspect, an embodiment of the present application provides a text processing device, including: A text unit extraction module, used to obtain a target text and extract a preset number of first text units from the target text; A first text unit feature extraction module, used to extract first text unit features corresponding to each first text unit respectively; A second text unit feature extraction module is used to extract, for each first text unit, the second text unit feature corresponding to the first text unit through the first feature extraction model based on the association relationship between other text units and the first text unit and the first text unit feature of the first text unit; the other text units are other first text units in each first text unit except the first text unit; The text category determination module is used to determine the category label of the target text based on the features of each second text unit; the category label is used to indicate whether the target text contains sensitive information.

[0014] In an optional embodiment of the present application, the first feature extraction model includes a preset number of sub-feature extraction models cascaded in sequence; The second text unit feature extraction module is specifically used for: For each first text unit, a second text unit feature corresponding to the first text unit is extracted by using a corresponding sub-feature extraction model.

[0015] In an optional embodiment of the present application, for each sub-feature extraction model, the sub-feature extraction model includes a plurality of layer structures cascaded in sequence; each layer structure includes a first cycle unit and a second cycle unit; The second text unit feature extraction module is further used for: The first text unit is input into the first layer structure of the sub-feature extraction model until the output of the last layer structure of the sub-feature extraction model is obtained, and the output of the last layer structure is used as the second text unit feature.

[0016] In an optional embodiment of the present application, the device further includes a layer structure output acquisition module, which is specifically used to: For each layer structure of the sub-feature extraction model, a first input feature corresponding to a first cycle unit of the layer structure is obtained, and a second input feature corresponding to a second cycle unit of the layer structure is obtained; Extracting a first input feature through a first cyclic unit of the layer structure to obtain a first output feature; Extracting the second input feature through the second cyclic unit of the layer structure to obtain a second output feature; The first output feature and the second output feature are merged to obtain a third output feature; The input features of the layer structure and the third output features are fused to obtain the output of the layer structure.

[0017] In an optional embodiment of the present application, for the first sub-feature extraction model, the first input feature of the first layer structure is the corresponding first text unit feature, and the first input feature of the other layer structures except the first layer structure is the output feature of the previous layer structure; for other feature extraction modules except the first sub-feature extraction model, the first input feature of the first layer structure is the corresponding first text unit feature and the first output feature of the first layer structure of the previous sub-feature extraction model, and the first input feature of the other layer structures except the first layer structure is the output feature of the previous layer structure and the first output feature of the layer structure corresponding to the previous sub-feature extraction model; For the last sub-feature extraction model, the second input feature of the first layer structure is the corresponding first text unit feature, and the second input feature of other layer structures except the first layer structure is the output feature of the previous layer structure; for other feature extraction modules except the last sub-feature extraction model, the second input feature of the first layer structure is the corresponding first text unit feature and the second output feature of the first layer structure of the next sub-feature extraction model, and the second input feature of other layer structures except the first layer structure is the output feature of the previous layer structure and the second output feature of the corresponding layer structure of the next sub-feature extraction model.

[0018] In an optional embodiment of the present application, the text unit extraction module is specifically used to: Segmenting the target text to obtain at least one second text unit; The obtained number of second text units is compared with a preset number, and if the number of second text units is not less than the preset number, the preset number of second text units are used as first text units according to the order in which the second text units appear in the target text; If the number of second text units is less than the preset number, third text units of the difference between the preset number and the second text units are obtained, and each second text unit and each third text unit are taken together as the first text unit.

[0019] In an optional embodiment of the present application, the first text unit feature extraction module is specifically used to: For each first text unit, input the first text unit into a preset word vector conversion model to obtain a first text vector matrix corresponding to the first text unit; The first text vector matrix is ​​convolved using a preset convolution kernel to obtain a first text unit feature of the first text unit.

[0020] In an optional embodiment of the present application, the text category determination module is specifically used to: Based on each second text unit feature, obtaining text features corresponding to the target text; The text features are calculated through a preset activation function to obtain the text type probability distribution of the target text, and the category label of the target text is output based on the text type probability distribution.

[0021] In an optional embodiment of the present application, the text category determination module is further used to: Obtain the maximum probability index value in the text type probability, and match the maximum probability index value with each category label stored in the preset database; The category label matched from the preset database is used as the category label of the target text.

[0022] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory; The processor executes the computer program to implement the method provided in the embodiment of the first aspect or any optional embodiment of the first aspect.

[0023] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method provided in the embodiment of the first aspect or any optional embodiment of the first aspect is implemented.

[0024] The beneficial effects of the technical solution provided by the embodiment of the present application are: The solution provided by the present application, after acquiring the target text, first extracts a certain number of text units from the target text, and extracts the text unit features of each text unit through a feature extraction model based on the association relationship between the text units, and finally determines the category to which the target text belongs based on the text unit features of each text unit. When extracting the text unit features of each text unit, the association relationship between each text unit is combined to provide more reference information for the output of the model, making up for the influence of the disappearance of the training gradient in the process of deepening the model neural network, thereby improving the recognition effect of sensitive events. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0026] Figure 1 A flowchart of a text processing method provided in an embodiment of the present application; Figure 2A schematic diagram of the structure of a feature extraction model in an example of an embodiment of the present application; Figure 3 A schematic diagram of a process of extracting a first text unit feature in an example of an embodiment of the present application; Figure 4 This is a schematic diagram of the overall flow of a text processing method in an example of an embodiment of the present application; Figure 5 A structural block diagram of a text processing device provided in an embodiment of the present application; Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0028] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B".

[0029] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0030] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0031] Figure 1A flowchart of a text processing method is provided for an embodiment of the present application. The execution subject of the method may be a terminal (such as a computer, a mobile phone, etc.). Figure 1 As shown, the method may include: Step S101, obtaining a target text, and extracting a preset number of first text units from the target text.

[0032] The target text may be web page information published on the Internet or text content input by a user, which is not limited in the present embodiment. The first text unit may be a single word or phrase, or a short sentence, letter, symbol, number, etc., which is not limited in the present embodiment.

[0033] Specifically, when it is necessary to classify the target text, the target text needs to be input into the terminal first. The input method can be to input the target text content into the terminal by itself, or to send an operation instruction to the terminal to crawl the target text from the network. After obtaining the target text, the target text needs to be preprocessed. Specifically, the target text can be segmented first to obtain a preset number of target words (that is, the first text unit). The segmentation method can be to use the jieba (jieba word segmentation) tool (a Chinese word segmentation tool based on Python (a programming language)). When processing the target text, if stop words are detected in the target text, these stop words can be removed before performing subsequent word segmentation operations.

[0034] Step S102: extract first text unit features corresponding to each first text unit respectively.

[0035] The first text unit feature may be in the form of a word vector.

[0036] Specifically, after the target text is segmented to obtain the first text units, these text units need to be further converted into word vectors (i.e., first text unit features) that can be recognized by the model. Specifically, the word vectors of each first text unit can be obtained by training the Word2Vec model (a group of related models used to generate word vectors). ∈ ( =1,2,…, ), where m is the preset number, and the training method can adopt the Skip-Gram model (a model that allows skipping certain symbols).

[0037] Step S103, for each first text unit, based on the association relationship between other text units and the first text unit and the first text unit features of the first text unit, extract the second text unit features corresponding to the first text unit through the first feature extraction model; the other text units are other first text units in each first text unit except the first text unit.

[0038] Among them, in the solution provided in the embodiment of the present application, the first feature extraction model can be a BiGRU (bidirectional gated recurrent unit) model combined with a residual structure. The characteristic of this model is that it combines the contextual association relationship between each first text unit through the residual structure, that is, each text unit will affect the feature extraction of other text units during feature extraction.

[0039] Step S104, determining the category label of the target text based on the features of each second text unit; the category label is used to indicate whether the target text contains sensitive information.

[0040] Specifically, after obtaining the features of each second text unit, these second text unit features can be combined to obtain the deep context semantic feature matrix of the target text: S , then based on the matrix S Analyze the corresponding category labels, and finally judge whether the target text belongs to sensitive text based on the category labels, and output the final judgment result.

[0041] The solution provided by the present application, after acquiring the target text, first extracts a certain number of text units from the target text, and extracts the text unit features of each text unit through a feature extraction model based on the association relationship between the text units, and finally determines the category to which the target text belongs based on the text unit features of each text unit. When extracting the text unit features of each text unit, the association relationship between each text unit is combined to provide more reference information for the output of the model, making up for the influence of the disappearance of the training gradient in the process of deepening the model neural network, thereby improving the recognition effect of sensitive events.

[0042] In an optional embodiment of the present application, the first feature extraction model includes a preset number of sub-feature extraction models cascaded in sequence; Extracting the second text unit feature corresponding to the first text unit by the first feature extraction model specifically includes: For each first text unit, a second text unit feature corresponding to the first text unit is extracted by using a corresponding sub-feature extraction model.

[0043] Among them, in the embodiment of the present application, the first feature extraction module is composed of multiple sub-feature extraction models, each sub-feature extraction model corresponds to a first text unit, so when obtaining the first text unit of the target text, the same number of first text units will be extracted according to the number of sub-feature extraction models.

[0044] Each first text unit can be simultaneously input into the first feature extraction model, and the first feature extraction model includes several sub-feature extraction models. Each first text unit will be input into each sub-feature extraction model respectively, and each sub-feature extraction model will then output the second text unit features of its corresponding first text unit.

[0045] In an optional embodiment of the present application, for each sub-feature extraction model, the sub-feature extraction model includes a plurality of layer structures cascaded in sequence; each layer structure includes a first cycle unit and a second cycle unit; Extracting the second text unit feature corresponding to the first text unit through the corresponding sub-feature extraction model specifically includes: The first text unit is input into the first layer structure of the sub-feature extraction model until the output of the last layer structure of the sub-feature extraction model is obtained, and the output of the last layer structure is used as the second text unit feature.

[0046] Among them, Figure 2 As shown, in the embodiment of the present application, each sub-feature extraction model is composed of multiple layer structures, and structures 01, 02, and 04 in the figure are all sub-feature extraction models, and structure 03 is a layer structure in a sub-feature extraction model. The layer structure contains two GRUs (Recurrent Neural Network, RNN, gated recurrent units), one of which is forward (the first one from left to right in the figure, i.e., the first recurrent unit), and the other is reverse (the second one from left in the figure, i.e., the second recurrent unit).

[0047] Specifically, for each first text unit, the first text unit is input into the first layer structure of the corresponding sub-feature extraction model according to the order of appearance of each first text unit in the target text. The sub-feature extraction model will process the first text unit through each cascaded layer structure. Specifically, the output of each layer structure is used as the output of the next adjacent cascaded layer structure to realize the sequential transmission of data until the output of the last layer structure is obtained. The output of the last layer structure is the second text unit feature.

[0048] In an optional embodiment of the present application, the output of each layer structure is obtained by: For each layer structure of the sub-feature extraction model, a first input feature corresponding to a first cycle unit of the layer structure is obtained, and a second input feature corresponding to a second cycle unit of the layer structure is obtained; Extracting a first input feature through a first cyclic unit of the layer structure to obtain a first output feature; Extracting the second input feature through the second cyclic unit of the layer structure to obtain a second output feature; The first output feature and the second output feature are merged to obtain a third output feature; The input features of the layer structure and the third output features are fused to obtain the output of the layer structure.

[0049] Among them, the first output feature can also be called a forward memory unit, and the second output feature can also be called a backward memory unit.

[0050] Specifically, Figure 2 As shown, the two cyclic units of each layer structure will be fused after outputting the first output feature and the second output feature to obtain the third output feature, and then the output of the layer structure is obtained based on the third output feature and the input of the layer structure, and then the output of the layer structure is used as the input of the next adjacent cascaded layer structure.

[0051] In an optional embodiment of the present application, for the first sub-feature extraction model, the first input feature of the first layer structure is the corresponding first text unit feature, and the first input feature of the other layer structures except the first layer structure is the output feature of the previous layer structure; for other feature extraction modules except the first sub-feature extraction model, the first input feature of the first layer structure is the corresponding first text unit feature and the first output feature of the first layer structure of the previous sub-feature extraction model, and the first input feature of the other layer structures except the first layer structure is the output feature of the previous layer structure and the first output feature of the layer structure corresponding to the previous sub-feature extraction model; For the last sub-feature extraction model, the second input feature of the first layer structure is the corresponding first text unit feature, and the second input feature of other layer structures except the first layer structure is the output feature of the previous layer structure; for other feature extraction modules except the last sub-feature extraction model, the second input feature of the first layer structure is the corresponding first text unit feature and the second output feature of the first layer structure of the next sub-feature extraction model, and the second input feature of other layer structures except the first layer structure is the output feature of the previous layer structure and the second output feature of the corresponding layer structure of the next sub-feature extraction model.

[0052] Specifically, in the embodiment of the present application, the feature extraction model can also be called a bidirectional recurrent neural network model with a fused residual structure. The characteristic of the model is that the corresponding layer structures between different sub-feature extraction models will affect each other, such as Figure 2 As shown, the first cycle unit in the layer structure 03 affects the first cycle unit of the corresponding layer structure in the sub-feature extraction model 04, and the second cycle unit in the layer structure corresponding to the layer structure 03 in the sub-feature extraction model 04 also affects the second cycle unit in the layer structure 03. The specific influence method can be as follows Figure 2 As shown, for the first cycle unit, the output features of the first cycle units of the sub-feature extraction models other than the last sub-feature extraction model will be used as the input features of the first cycle unit of the corresponding next sub-feature extraction model; and for the second cycle unit, the output features of the second cycle units of the sub-feature extraction models other than the first sub-feature extraction model will be used as the input features of the second cycle unit of the corresponding previous sub-feature extraction model.

[0053] It should be noted that each sub-feature extraction model can also influence each other at different times, and thus the output features at different times can be obtained. Before introducing the specific calculation method, some special terms in recurrent neural networks are introduced first: Forget gate: It is used to decide which information should be forgotten. It decides which information should be discarded from the memory cell by looking at the current input and the hidden state of the previous time step.

[0054] Input gate: responsible for deciding what new information should be added to the memory cell. It determines the extent of the new information by combining the current input and the hidden state of the previous time step. This mechanism helps control the update of information and ensures that only important new information is added to the memory cell, thereby enhancing the memory capacity of the model.

[0055] Output gate: The output gate controls the output of the memory cell. It determines which information should be output from the memory cell to the hidden state of the current time step. This mechanism helps control the transfer of information and ensures that only relevant information is passed to the next time step, thereby enhancing the predictive power of the model.

[0056] Reset gate: determines how much past information needs to be forgotten. The reset gate is adjusted by an activation function, whose value is between 0 and 1, and determines how much historical information the model should retain when processing sequence data.

[0057] Update gate: determines how much information in the previous hidden state is transferred to the current hidden state, or how much information between the previous moment and the current moment needs to be transferred. When the output value of the update gate is close to 0, it means that the information of the previous hidden state is forgotten in the hidden layer; when the output value is close to 1, it means that the information is retained in the current hidden layer. In the embodiment of the present application, the update gate can be composed of the input gate and the forget gate.

[0058] The specific calculation method is as follows: 1) Features of the first text unit After input, it will be combined with the sequence information of the previous moment on the hidden layer of the sub-feature extraction model The update signal is obtained by the operation of the update gate (the update gate is the combination of the input gate and the forget gate) and the reset gate. and reset signal , the specific formula is as follows:

[0059]

[0060] in, , represents the preset weight, Represents the activation function.

[0061] 2) By Value determination Importance of candidate memory cells. and Candidate memory cells By weight And activation function tanh. The candidate memory unit calculation formula is:

[0062] Among them, W is the preset weight.

[0063] 3) By calculation , and ,Sure The weight of passing the next state to obtain the memory unit at the current moment The calculation formula is as follows:

[0064] 4) Perform the above steps on the first cycle unit and the second cycle unit respectively, and you can get the first output feature of the first cycle unit at time t And the second output feature of the second cycle unit at time t At this time, the first output feature and the second output feature are combined to obtain the third output feature , and then connect the third output feature with the first text unit feature to obtain , forming the output of this layer structure , and use the output as the input feature of the next layer structure.

[0065] It should be noted that, in the embodiment of the present application, in order to further improve the feature learning effect of the network, before connecting the third output feature with the first text unit feature, a further convolution operation may be performed on the first text unit feature. go through The shape is After convolution with the convolution kernel, we get the vector , and connect the vector as the new first text unit feature to the third output feature.

[0066] In an optional embodiment of the present application, extracting a preset number of first text units from the target text specifically includes: Segmenting the target text to obtain at least one second text unit; The obtained number of second text units is compared with a preset number, and if the number of second text units is not less than the preset number, the preset number of second text units are used as first text units according to the order in which the second text units appear in the target text; If the number of second text units is less than the preset number, third text units of the difference between the preset number and the second text units are obtained, and each second text unit and each third text unit are taken together as the first text unit.

[0067] The second text unit may be the corresponding target words obtained after word segmentation of the target text. The third text unit may be a specific filling mark, which is consistent with the input requirements of the preset word vector conversion model, and usually a mark (such as a zero vector) that has no practical significance for the training of the preset word vector conversion model is selected.

[0068] Specifically, after the target text is processed by the word segmentation model to obtain multiple second text units of the target text, further processing can be performed according to the number of the obtained second text units. Specifically, when the number of second text units is greater than the preset number (i.e., the number of sub-feature extraction models), only the preset number of second text units that appear earlier in the target text are obtained as the first text units; when the number of second text units is equal to the preset number, all second text units can be directly used as the first text units; and when the number of second text units is less than the preset number, the second text units cannot be input into all sub-feature extraction models. At this time, in order to ensure the accuracy of feature extraction, some meaningless target words (i.e., third text units) can be used as fillers to make the number of text units reach the preset number, and then the second text units obtained from the word segmentation of the target text and the third text units used for filling are input into each sub-feature extraction model as the first text unit, thereby ensuring the accuracy of feature extraction.

[0069] In an optional embodiment of the present application, extracting first text unit features corresponding to each first text unit respectively includes: For each first text unit, input the first text unit into a preset word vector conversion model to obtain a first text vector matrix corresponding to the first text unit; The first text vector matrix is ​​convolved using a preset convolution kernel to obtain a first text unit feature of the first text unit.

[0070] Among them, the preset convolution kernel can be a structure matrix, and the structure can be expressed as The preset convolution kernel has a corresponding convolution kernel field of view. By sliding the convolution kernel field of view on the vector matrix, the corresponding text unit features can be extracted.

[0071] Specifically, Figure 3 As shown, after obtaining the first text vector matrix corresponding to each first text unit through the preset word vector conversion model, one or more convolution kernels (i.e., the box A1 in the figure, each box represents a convolution kernel) can be scanned sequentially on the first text vector matrix, and then the corresponding first text feature (i.e., part A2 in the figure) can be obtained after calculation through the intermediate nonlinear activation function f. Each first text feature after the convolution operation can be expressed as follows:

[0072] in, Represents the first text vector matrix obtained through the convolution kernel field of view when each convolution kernel scans Composed of Matrix (i.e. the first text vector matrix), Represents the convolution kernel, the number can be r, for dimensional bias vector parameter.

[0073] Furthermore, by using the convolution kernel to scan each first text vector matrix one by one, the local features of the word vector of the entire sentence can be obtained:

[0074] in .

[0075] In an optional embodiment of the present application, determining the category label of the target text based on each second text unit feature specifically includes: Based on each second text unit feature, obtaining text features corresponding to the target text; The text features are calculated through a preset activation function to obtain the text type probability distribution of the target text, and the category label of the target text is output based on the text type probability distribution.

[0076] Specifically, after each sub-feature extraction model outputs its own second text unit feature, the text feature of the target text can be obtained by combining all the second text unit features. Then, the text feature of the target text is input into the preset softmax activation function to calculate the probability distribution P of the text type. The specific calculation method is as follows:

[0077] in, represents the weight, represents the offset vector, representing text features of the target text at the corresponding moment; ∈ , ∈ , , is the category number of the text type). After the probability distribution of each text type is calculated, the category label of the target text is determined according to the probability distribution of each text type and is output as the result.

[0078] In an optional embodiment of the present application, outputting a category label of the target text based on the text type probability distribution specifically includes: Obtain the maximum probability index value in the text type probability, and match the maximum probability index value with each category label stored in the preset database; The category label matched from the preset database is used as the category label of the target text.

[0079] Specifically, when obtaining the maximum probability index value, the arg_max function can be used ( ), through which the probability distribution can be obtained P The maximum probability index value , then Match the index values ​​corresponding to each text type in the preset database, and use the text type corresponding to the index value that matches the result closest to the target text as the category label of the target text. Optionally, you can use a pre-trained matching model to match each index value (i.e., input , output category label).

[0080] Optionally, during the training process of the matching model, the cross entropy cost function can be used as the loss function, and the loss value is loss. The loss is calculated as follows:

[0081] Among them, label and tf represent the corresponding label parameters, which can be continuously changed during the model training process until the loss value is less than a custom preset threshold, at which time the training of the model is terminated.

[0082] Combine the following Figure 4 , introduce the overall process of the solution provided by this application, such as Figure 4 As shown, the target text is first crawled through the network or other means, and then the target text is preprocessed. The preprocessing method includes word segmentation and stop word removal of the target text, etc., to obtain each first text unit, and each first text unit is input into the word vector conversion model to obtain the first text vector matrix corresponding to each first text unit, and then each first text unit feature is input into the convolutional neural network for convolution processing to obtain each first text unit feature, and finally each first text unit feature is input into the trained feature extraction model to output the category label of the target text.

[0083] Figure 5 A structural block diagram of a text processing device provided in an embodiment of the present application, such as Figure 5 As shown, the text processing device 500 may include: a text unit extraction module 501, a first text unit feature extraction module 502, a second text unit feature extraction module 503 and a text category determination module 504, wherein: The text unit extraction module 501 is used to obtain a target text and extract a preset number of first text units from the target text; The first text unit feature extraction module 502 is used to extract first text unit features corresponding to each first text unit respectively; The second text unit feature extraction module 503 is used to extract the second text unit feature corresponding to each first text unit through the first feature extraction model based on the association relationship between other text units and the first text unit and the first text unit feature of the first text unit; the other text units are other first text units in each first text unit except the first text unit; The text category determination module 504 is used to determine the category label of the target text based on the features of each second text unit; the category label is used to indicate whether the target text contains sensitive information.

[0084] The solution provided by the present application, after acquiring the target text, first extracts a certain number of text units from the target text, and extracts the text unit features of each text unit through a feature extraction model based on the association relationship between the text units, and finally determines the category to which the target text belongs based on the text unit features of each text unit. When extracting the text unit features of each text unit, the association relationship between each text unit is combined to provide more reference information for the output of the model, making up for the influence of the disappearance of the training gradient in the process of deepening the model neural network, thereby improving the recognition effect of sensitive events.

[0085] In an optional embodiment of the present application, the first feature extraction model includes a preset number of sub-feature extraction models cascaded in sequence; The second text unit feature extraction module is specifically used for: For each first text unit, a second text unit feature corresponding to the first text unit is extracted by using a corresponding sub-feature extraction model.

[0086] In an optional embodiment of the present application, for each sub-feature extraction model, the sub-feature extraction model includes a plurality of layer structures cascaded in sequence; each layer structure includes a first cycle unit and a second cycle unit; The second text unit feature extraction module is further used for: The first text unit is input into the first layer structure of the sub-feature extraction model until the output of the last layer structure of the sub-feature extraction model is obtained, and the output of the last layer structure is used as the second text unit feature.

[0087] In an optional embodiment of the present application, the device further includes a layer structure output acquisition module, which is specifically used to: For each layer structure of the sub-feature extraction model, a first input feature corresponding to a first cycle unit of the layer structure is obtained, and a second input feature corresponding to a second cycle unit of the layer structure is obtained; Extracting a first input feature through a first cyclic unit of the layer structure to obtain a first output feature; Extracting the second input feature through the second cyclic unit of the layer structure to obtain a second output feature; The first output feature and the second output feature are merged to obtain a third output feature; The input features of the layer structure and the third output features are fused to obtain the output of the layer structure.

[0088] In an optional embodiment of the present application, for the first sub-feature extraction model, the first input feature of the first layer structure is the corresponding first text unit feature, and the first input feature of the other layer structures except the first layer structure is the output feature of the previous layer structure; for other feature extraction modules except the first sub-feature extraction model, the first input feature of the first layer structure is the corresponding first text unit feature and the first output feature of the first layer structure of the previous sub-feature extraction model, and the first input feature of the other layer structures except the first layer structure is the output feature of the previous layer structure and the first output feature of the layer structure corresponding to the previous sub-feature extraction model; For the last sub-feature extraction model, the second input feature of the first layer structure is the corresponding first text unit feature, and the second input feature of other layer structures except the first layer structure is the output feature of the previous layer structure; for other feature extraction modules except the last sub-feature extraction model, the second input feature of the first layer structure is the corresponding first text unit feature and the second output feature of the first layer structure of the next sub-feature extraction model, and the second input feature of other layer structures except the first layer structure is the output feature of the previous layer structure and the second output feature of the corresponding layer structure of the next sub-feature extraction model.

[0089] In an optional embodiment of the present application, the text unit extraction module is specifically used to: Segmenting the target text to obtain at least one second text unit; The obtained number of second text units is compared with a preset number, and if the number of second text units is not less than the preset number, the preset number of second text units are used as first text units according to the order in which the second text units appear in the target text; If the number of second text units is less than the preset number, third text units of the difference between the preset number and the second text units are obtained, and each second text unit and each third text unit are taken together as the first text unit.

[0090] In an optional embodiment of the present application, the first text unit feature extraction module is specifically used to: For each first text unit, input the first text unit into a preset word vector conversion model to obtain a first text vector matrix corresponding to the first text unit; The first text vector matrix is ​​convolved using a preset convolution kernel to obtain a first text unit feature of the first text unit.

[0091] In an optional embodiment of the present application, the text category determination module is specifically used to: Based on each second text unit feature, obtaining text features corresponding to the target text; The text features are calculated through a preset activation function to obtain the text type probability distribution of the target text, and the category label of the target text is output based on the text type probability distribution.

[0092] In an optional embodiment of the present application, the text category determination module is further used to: Obtain the maximum probability index value in the text type probability, and match the maximum probability index value with each category label stored in the preset database; The category label matched from the preset database is used as the category label of the target text.

[0093] Reference below Figure 6 , which shows an electronic device suitable for implementing the embodiments of the present application (for example, Figure 1 The electronic device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable devices, etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0094] The electronic device includes: a memory and a processor, the memory is used to store a program for executing the method described in each of the above method embodiments; the processor is configured to execute the program stored in the memory. The processor here can be referred to as the processing device 601 described below, and the memory can include at least one of the read-only memory (ROM) 602, the random access memory (RAM) 603, and the storage device 608 described below, as shown below: like Figure 6As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0095] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0096] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present application are executed.

[0097] It should be noted that the computer-readable storage medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0098] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0099] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0100] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: Obtain a target text, extract a preset number of first text units from the target text; extract first text unit features corresponding to each first text unit respectively; for each first text unit, based on the association relationship between other text units and the first text unit and the first text unit features of the first text unit, extract the second text unit features corresponding to the first text unit through a first feature extraction model; the other text units are the first text units other than the first text unit in each first text unit; based on the features of each second text unit, determine the category label of the target text; the category label is used to indicate whether the target text contains sensitive information.

[0101] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] The modules or units involved in the embodiments of the present application may be implemented by software or hardware. The name of a module or unit does not limit the unit itself in some cases. For example, the first constraint acquisition module may also be described as a "module for acquiring the first constraint".

[0104] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0105] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0107] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A text processing method, characterized in that: include: Acquire a target text, and extract a preset number of first text units from the target text; Extracting first text unit features corresponding to each first text unit respectively; For each first text unit, based on the association relationship between other text units and the first text unit and the first text unit feature of the first text unit, extract the second text unit feature corresponding to the first text unit through a first feature extraction model; the other text units are other first text units in each first text unit except the first text unit; Based on each second text unit feature, a category label of the target text is determined; the category label is used to indicate whether the target text contains sensitive information.

2. The method according to claim 1, characterized in that The first feature extraction model includes the preset number of sub-feature extraction models cascaded in sequence; The extracting the second text unit feature corresponding to the first text unit by using the first feature extraction model includes: For each first text unit, a second text unit feature corresponding to the first text unit is extracted by using a corresponding sub-feature extraction model.

3. The method according to claim 2, characterized in that For each sub-feature extraction model, the sub-feature extraction model includes a plurality of layer structures cascaded in sequence; each layer structure includes a first cycle unit and a second cycle unit; The extracting the second text unit feature corresponding to the first text unit by using the corresponding sub-feature extraction model includes: The first text unit is input into a first layer structure of the sub-feature extraction model until an output of a last layer structure of the sub-feature extraction model is obtained, and the output of the last layer structure is used as the second text unit feature.

4. The method according to claim 3, characterized in that The output of each layer structure is obtained in the following way: For each layer structure of the sub-feature extraction model, obtaining a first input feature corresponding to a first cycle unit of the layer structure, and obtaining a second input feature corresponding to a second cycle unit of the layer structure; Performing feature extraction on the first input feature through a first cyclic unit of the layer structure to obtain a first output feature; Performing feature extraction on the second input feature through a second cyclic unit of the layer structure to obtain a second output feature; The first output feature and the second output feature are combined to obtain a third output feature; The input feature of the layer structure and the third output feature are fused to obtain the output of the layer structure.

5. The method according to claim 4, characterized in that For the first sub-feature extraction model, the first input feature of the first layer structure is the corresponding first text unit feature, and the first input feature of the other layer structures except the first layer structure is the output feature of the previous layer structure; for other feature extraction modules except the first sub-feature extraction model, the first input feature of the first layer structure is the corresponding first text unit feature and the first output feature of the first layer structure of the previous sub-feature extraction model, and the first input feature of the other layer structures except the first layer structure is the output feature of the previous layer structure and the first output feature of the layer structure corresponding to the previous sub-feature extraction model; For the last sub-feature extraction model, the second input feature of the first layer structure is the corresponding first text unit feature, and the second input feature of other layer structures except the first layer structure is the output feature of the previous layer structure; for other feature extraction modules except the last sub-feature extraction model, the second input feature of the first layer structure is the corresponding first text unit feature and the second output feature of the first layer structure of the next sub-feature extraction model, and the second input feature of other layer structures except the first layer structure is the output feature of the previous layer structure and the second output feature of the corresponding layer structure of the next sub-feature extraction model.

6. The method according to claim 1, characterized in that The step of extracting a preset number of first text units from the target text includes: Segmenting the target text to obtain at least one second text unit; Compare the obtained number of second text units with the preset number, and if the number of second text units is not less than the preset number, use the preset number of second text units as first text units in the order in which the second text units appear in the target text; If the number of the second text units is less than the preset number, third text units of the difference between the preset number and the second text units are obtained, and each second text unit and each third text unit are taken together as a first text unit.

7. The method according to claim 1, characterized in that The extracting first text unit features corresponding to each first text unit respectively includes: For each first text unit, input the first text unit into a preset word vector conversion model to obtain a first text vector matrix corresponding to the first text unit; The first text vector matrix is ​​convolved using a preset convolution kernel to obtain a first text unit feature of the first text unit.

8. The method according to claim 1, characterized in that The step of determining the category label of the target text based on each second text unit feature includes: Based on each second text unit feature, obtaining text features corresponding to the target text; The text features are calculated by a preset activation function to obtain a text type probability distribution of the target text, and a category label of the target text is output based on the text type probability distribution.

9. The method according to claim 8, characterized in that The step of outputting the category label of the target text based on the text type probability distribution includes: Obtaining a maximum probability index value among the text type probabilities, and matching the maximum probability index value with each category label stored in a preset database; The category label matched from the preset database is used as the category label of the target text.

10. A text processing device, characterized in that: include: A text unit extraction module, used to obtain a target text and extract a preset number of first text units from the target text; A first text unit feature extraction module, used to extract first text unit features corresponding to each first text unit respectively; A second text unit feature extraction module is used to extract, for each first text unit, a second text unit feature corresponding to the first text unit through a first feature extraction model based on the association relationship between other text units and the first text unit and the first text unit feature of the first text unit; the other text units are other first text units in each first text unit except the first text unit; The text category determination module is used to determine the category label of the target text based on each second text unit feature; the category label is used to indicate whether the target text contains sensitive information.

11. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images