Information identification method, device, terminal device and storage medium

Abstract

The present invention discloses an information identification method, device, terminal equipment and storage medium, wherein the information identification method comprises: obtaining information to be identified; inputting the information to be identified into an information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model; and judging whether the information to be identified is junk information according to the prediction result. The present invention solves the problem of information identification accuracy and improves the accuracy of information identification.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to an information identification method, device, terminal equipment and storage medium. Background Art

[0002] With the rapid development of the communication service industry in the information age, spam has caused certain troubles in people's lives. Therefore, information identification has important social value and practical significance.

[0003] Existing information recognition methods usually use Word2Vec to generate word vectors to complete the vector representation of information text and its phonetic text, and then use the corresponding training set to model the relationship between information features and information categories through a CNN model or an RNN model, and finally use this to determine whether a piece of information is spam.

[0004] However, when vectorizing information based on Word2Vec, the information considered is relatively single and cannot solve the problem of polysemous words; the feature extraction capabilities of the CNN model and the RNN model are not strong enough, resulting in insufficient modeling of the relationship between information features and information categories; and the RNN model has poor parallel computing capabilities for processing data. Summary of the invention

[0005] The main purpose of the present invention is to provide an information identification method, apparatus, terminal device and storage medium, aiming to improve the accuracy of information identification.

[0006] To achieve the above object, the present invention provides an information identification method, which comprises the following steps:

[0007] Obtaining information to be identified;

[0008] Inputting the information to be identified into an information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model;

[0009] Determine whether the information to be identified is spam according to the prediction result.

[0010] Optionally, before the step of inputting the information to be identified into the information identification model for prediction and obtaining the prediction result, the step further includes:

[0011] Creating the information recognition model specifically includes:

[0012] Acquire first sample information for training, where the first sample information includes first junk information and first normal information;

[0013] Based on the first BERT model, Transformer encoder, pre-created sentence coherence model, prediction layer, and interaction layer, an initial information recognition model is constructed;

[0014] The first sample information is input into the initial information recognition model, and the initial information recognition model is iteratively trained in combination with a first loss function to obtain a created information recognition model.

[0015] Optionally, the step of inputting the information to be identified into an information identification model for prediction to obtain a prediction result includes:

[0016] The information to be identified is input into the information identification model for the following processing:

[0017] The information text and the pinyin text are represented by text vectorization through the first BERT model in the information recognition model to obtain corresponding information matrix and pinyin matrix;

[0018] Inputting the information matrix into a pre-created sentence coherence model for judgment to obtain a first coherence vector corresponding to the information text;

[0019] Input the information matrix and the pinyin matrix into the first Transformer encoder and the second Transformer encoder respectively for feature extraction to obtain corresponding information feature matrix and pinyin feature matrix;

[0020] Processing the information feature matrix and the phonetic feature matrix through the interaction layer to obtain corresponding information vectors and phonetic vectors;

[0021] The information vector, the phonetic vector and the first fluency vector are calculated through the prediction layer to obtain a prediction result.

[0022] Optionally, the step of processing the information feature matrix and the phonetic feature matrix through the interaction layer to obtain corresponding information vectors and phonetic vectors includes:

[0023] The information feature matrix and the phonetic feature matrix are interacted with each other through the interaction layer to obtain an interactive attention matrix;

[0024] Based on the interactive attention matrix, the maximum pooling operation is performed on the information feature matrix and the pinyin feature matrix respectively to obtain the corresponding information vector and pinyin vector.

[0025] Optionally, the prediction layer includes a concatenation layer, a first fully connected layer, and a first Sigmoid function, and the step of calculating the information vector, the pinyin vector, and the first coherence vector through the prediction layer to obtain a prediction result includes:

[0026] The information vector, the phonetic vector and the fluency vector are concatenated by the concatenation layer to obtain a concatenated prediction vector;

[0027] The prediction vector is calculated through the first fully connected layer and the first Sigmoid function to obtain the probability that the information to be identified is spam information as a prediction result.

[0028] Optionally, the step of inputting the information to be identified into an information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model, further comprises:

[0029] Creating the sentence coherence model specifically includes:

[0030] Acquire second sample information for training, where the second sample information includes second junk information and second normal information;

[0031] Based on the second BERT model, the third Transformer encoder, the pooling layer, the second fully connected layer and the second Sigmoid function, an initial sentence coherence model is constructed;

[0032] The second sample information is input into the initial sentence coherence model, and the initial sentence coherence model is iteratively trained in combination with a second loss function to obtain a created sentence coherence model.

[0033] Optionally, the step of inputting the second sample information into the initial sentence coherence model, and iteratively training the initial sentence coherence model in combination with a second loss function to obtain the created sentence coherence model comprises:

[0034] The second sample information is input into the initial sentence coherence model for the following processing:

[0035] Vectorize the second sample information through a second BERT model to obtain a second sample matrix corresponding to the second sample information;

[0036] Performing feature extraction on the second sample matrix by the third Transformer encoder to obtain a second sample feature matrix corresponding to the second sample matrix;

[0037] Processing the second sample feature matrix in a preset direction through the pooling layer to obtain a second sample vector corresponding to the second sample feature matrix;

[0038] Based on the second fully connected layer and the second Sigmoid function, the second sample vector is calculated to obtain fluency and a second fluency vector;

[0039] The fluency is calculated based on a standard cross entropy loss function to obtain a total loss value;

[0040] Based on the gradient descent algorithm and the total loss value, a repair parameter is obtained;

[0041] training the initial sentence coherence model in combination with the patching parameters; and returning to the step of: inputting the second sample information into the initial sentence coherence model to obtain coherence and a second coherence vector;

[0042] The training is terminated according to the preset number of cycles until the cycle is completed, and the created sentence fluency model is obtained.

[0043] The present application also provides an information identification device, which includes:

[0044] An information acquisition module is used to acquire information to be identified;

[0045] An information recognition module, used for inputting the information to be recognized into an information recognition model for prediction to obtain a prediction result, wherein the information recognition model is trained based on a preset matrix encoder and a pre-created sentence coherence model;

[0046] Information judgment module: used to judge whether the information to be identified is junk information according to the prediction result.

[0047] An embodiment of the present application also proposes a terminal device, which includes a memory, a processor, and an information identification program stored in the memory and executable on the processor, wherein the information identification program implements the steps of the information identification method described above when executed by the processor.

[0048] An embodiment of the present application further provides a computer-readable storage medium, on which an information identification program is stored. When the information identification program is executed by a processor, the steps of the information identification method described above are implemented.

[0049] An information identification method, apparatus, terminal device and storage medium proposed in an embodiment of the present invention obtain information to be identified; input the information to be identified into an information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence fluency model; and judge whether the information to be identified is spam according to the prediction result. By identifying information with a trained information identification model, the problem of information identification accuracy can be solved and the accuracy of information identification can be improved. Based on the solution of the present invention, starting from the pain points that have always existed in the difficult industry regarding the spam information identification function, a sample information data set of spam information and normal information in the difficult industry is constructed, and the effectiveness of the information identification method proposed by the present invention is verified on the data set. Finally, the accuracy of information recognition by the method of the present invention is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a schematic diagram of the functional modules of the terminal device to which the information identification device of the present invention belongs;

[0051] Figure 2 is a flow chart of a first exemplary embodiment of an information identification method of the present invention;

[0052] Figure 3 It is a schematic diagram of the overall data flow of the information identification model in the embodiment of the information identification method of the present invention;

[0053] Figure 4 A schematic diagram of a modeling process of an information identification model involved in a second exemplary embodiment of the information identification method of the present invention;

[0054] Figure 5 A schematic diagram of an application flow of an information identification model involved in a third exemplary embodiment of the information identification method of the present invention;

[0055] Figure 6 A schematic diagram of a modeling process of a sentence coherence model involved in a fourth exemplary embodiment of the information recognition method of the present invention;

[0056] Figure 7 It is a schematic diagram of the overall data flow of the sentence coherence model in the embodiment of the information identification method of the present invention;

[0057] Figure 8 A schematic diagram of a training process of a sentence coherence model involved in a fifth exemplary embodiment of the information recognition method of the present invention;

[0058] Fig. 9 for Figure 8 A detailed flowchart of step S800 in FIG.

[0059] Fig.10A flowchart of a sixth exemplary embodiment of the information identification method of the present invention.

[0060] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0061] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0062] The main solution of the embodiment of the present invention is: input the information to be identified into the information identification model for the following processing: the information text and the pinyin text are represented by text vectorization through the first BERT model in the information identification model to obtain the corresponding information matrix and pinyin matrix; the information matrix is ​​input into the pre-created sentence coherence model for judgment to obtain the first coherence vector corresponding to the information text; the information matrix and the pinyin matrix are respectively input into the first Transformer encoder and the second Transformer encoder for feature extraction to obtain the corresponding information feature matrix and pinyin feature matrix; the information feature matrix and the pinyin feature matrix are processed through the interaction layer to obtain the corresponding information vector and pinyin vector; the information vector, the pinyin vector and the first coherence vector are calculated through the prediction layer to obtain the prediction result. Based on the scheme of the present invention, starting from the pain point problem that has always existed in the industry regarding the spam information identification function, a sample information data set of spam information and normal information in the real world is constructed, and the effectiveness of the information identification method of the present invention is verified on the data set. Finally, the accuracy of information identification by the method of the present invention is significantly improved.

[0063] Technical terms involved in the embodiments of the present invention:

[0064] Transformer model;

[0065] BERT model, Bidirectional Encoder Representation from Transformers;

[0066] Fully connected neural network: MLP, Multi-layer Perceptron;

[0067] Max pooling;

[0068] Sigmoid function, Logistic function.

[0069] Transformer: It is the first conversion model that relies entirely on self-attention without using sequence-aligned RNN or convolution to calculate input-output representations.

[0070] Self-attention is sometimes called intra-attention. It is attention at different positions in a single sentence, and a representation of the sequence is obtained. It can be well applied to many tasks, including reading comprehension, summarization, textual implications, and sentence representation independent of tasks. End-to-end networks are generally based on recurrent attention mechanisms rather than sequence alignment cycles, and there is evidence that they perform well on simple language question answering and language modeling tasks. Before the birth of Transformer, most sequence acquisition models (Encoder-Decoder) were based on CNN and RNN. Before this article, an attention and self-attention mechanism have been introduced, and Transformer is based on the attention mechanism. The attention mechanism is better than CNN and RNN. Attention can solve the long-distance dependency problem of RNN and its variants, that is, the attention mechanism can have better memory and can remember information at a longer distance. In addition, the most important thing is that attention supports parallel computing. The Transformer model is completely based on the attention mechanism, completely abandoning the structure of CNN and RNN.

[0071] BERT model: The full name is Bidirectional Encoder Representation from Transformers, which is a pre-trained language representation model. It emphasizes that it no longer uses the traditional unidirectional language model or the shallow splicing of two unidirectional language models for pre-training as in the past, but uses a new masked language model (MLM) to generate deep bidirectional language representation. When the BERT paper was published, it mentioned that new state-of-the-art results were obtained in 11 NLP (Natural Language Processing) tasks.

[0072] Fully connected neural network: MLP, Multi-layer Perceptron: It is a forward-structured artificial neural network ANN that maps a set of input vectors to a set of output vectors. MLP can be regarded as a directed graph consisting of multiple node layers, and each layer is fully connected to the next layer. In addition to the input node, each node is a neuron with a nonlinear activation function. The supervised learning method of the BP back-propagation algorithm is used to train MLP. MLP is a generalization of the perceptron, which overcomes the weakness of the perceptron that it cannot recognize linearly inseparable data. Compared with the single-layer perceptron, the MLP multi-layer perceptron can have multiple hidden layers in the middle in addition to the input and output layers. The simplest MLP contains only one hidden layer, that is, a three-layer structure. The layers of the multi-layer perceptron are fully connected. The bottom layer of the multi-layer perceptron is the input layer, the middle layer is the hidden layer, and the last layer is the output layer.

[0073] Max pooling: Commonly used pooling methods include max-pooling and mean-pooling. According to relevant theories, feature extraction errors mainly come from two aspects:

[0074] (1) The variance of the estimated value increases due to the limited neighborhood size;

[0075] (2) The error in convolutional layer parameters causes the estimated mean to shift.

[0076] Generally speaking, mean-pooling can reduce the first error and retain more background information of the image, while max-pooling can reduce the second error and retain more texture information. Similar to mean-pooling, in a local sense, it obeys the criterion of max-pooling.

[0077] The size of the max-pooling convolution kernel is generally 2×2. Very large inputs may require 4x4. However, choosing a larger shape will significantly reduce the size of the signal and may cause excessive loss of information. Generally, non-overlapping pooling windows perform best.

[0078] Sigmoid function: also called Logistic function, used for hidden layer neuron output, with a value range of (0,1). It can map a real number to the interval of (0,1) and can be used for binary classification. It works better when the feature difference is complex or the difference is not particularly large. Advantages of Sigmoid as an activation function: smooth and easy to derive; Disadvantages: The activation function has a large amount of calculation, and when backpropagating to find the error gradient, the derivation involves division; during backpropagation, the gradient disappears easily, making it impossible to complete the training of deep networks.

[0079] Existing information recognition methods vectorize information based on Word2Vec. The information considered is relatively single and cannot solve the problem of polysemous words. The feature extraction capabilities of CNN models and RNN models are not strong enough, resulting in insufficient modeling of the relationship between information features and information categories. In addition, the RNN model has poor parallel computing capabilities for processing data.

[0080] Therefore, the embodiment of the present invention starts from the actual problem of low information recognition accuracy, combines the computing power of the Transformer encoder and the modeling ability of the pre-created sentence coherence model, and designs an information recognition model based on the Transformer encoder and the sentence coherence model to solve the problem of information recognition accuracy and improve the accuracy of information recognition.

[0081] Specifically, refer to Figure 1 , Figure 1 The figure is a functional module diagram of the terminal device to which the information identification device of the present invention belongs. The information identification device can be a device independent of the terminal device that can perform information identification and network model training, and can be carried on the terminal device in the form of hardware or software. The terminal device can be a smart mobile terminal with data processing function such as a mobile phone or a tablet computer, or a fixed terminal device or server with data processing function.

[0082] In this embodiment, the terminal device to which the information identification apparatus belongs includes at least an output module 110 , a processor 120 , a memory 130 and a communication module 140 .

[0083] The memory 130 stores an operating system and an information recognition program. The information recognition device can obtain the information to be recognized, input the information to be recognized into the information recognition model for prediction, and obtain the information to be recognized.

[0084] The prediction result, whether the information to be identified is junk information determined according to the prediction result, and the sample information including junk information and normal information for training are stored in the memory 130; the output module 110 can be a display screen, etc. The communication module 140 can include a WIFI module, a mobile communication module, a Bluetooth module, etc., and communicates with an external device or server through the communication module 140.

[0085] When the information recognition program in the memory 130 is executed by the processor, the following steps are implemented:

[0086] Obtaining information to be identified;

[0087] Inputting the information to be identified into an information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model;

[0088] Determine whether the information to be identified is spam according to the prediction result.

[0089] Furthermore, when the information recognition program in the memory 130 is executed by the processor, the following steps are also implemented:

[0090] Creating the information recognition model specifically includes:

[0091] Acquire first sample information for training, where the first sample information includes first junk information and first normal information;

[0092] Based on the first BERT model, Transformer encoder, pre-created sentence coherence model, prediction layer, and interaction layer, an initial information recognition model is constructed;

[0093] The first sample information is input into the initial information recognition model, and the initial information recognition model is iteratively trained in combination with a first loss function to obtain a created information recognition model.

[0094] Furthermore, when the information recognition program in the memory 130 is executed by the processor, the following steps are also implemented:

[0095] The information to be identified is input into the information identification model for the following processing:

[0096] The information text and the pinyin text are represented by text vectorization through the first BERT model in the information recognition model to obtain corresponding information matrix and pinyin matrix;

[0097] Inputting the information matrix into a pre-created sentence coherence model for judgment to obtain a first coherence vector corresponding to the information text;

[0098] Input the information matrix and the pinyin matrix into the first Transformer encoder and the second Transformer encoder respectively for feature extraction to obtain corresponding information feature matrix and pinyin feature matrix;

[0099] Processing the information feature matrix and the phonetic feature matrix through the interaction layer to obtain corresponding information vectors and phonetic vectors;

[0100] The information vector, the phonetic vector and the first fluency vector are calculated through the prediction layer to obtain a prediction result.

[0101] Furthermore, when the information recognition program in the memory 130 is executed by the processor, the following steps are also implemented:

[0102] The information feature matrix and the phonetic feature matrix are interacted with each other through the interaction layer to obtain an interactive attention matrix;

[0103] Based on the interactive attention matrix, the maximum pooling operation is performed on the information feature matrix and the pinyin feature matrix respectively to obtain the corresponding information vector and pinyin vector.

[0104] Furthermore, when the information recognition program in the memory 130 is executed by the processor, the following steps are also implemented:

[0105] The information vector, the phonetic vector and the fluency vector are concatenated by the concatenation layer to obtain a concatenated prediction vector;

[0106] The prediction vector is calculated through the first fully connected layer and the first Sigmoid function to obtain the probability that the information to be identified is spam information as a prediction result.

[0107] Furthermore, when the information recognition program in the memory 130 is executed by the processor, the following steps are also implemented:

[0108] Creating the sentence coherence model specifically includes:

[0109] Acquire second sample information for training, where the second sample information includes second junk information and second normal information;

[0110] Based on the second BERT model, the third Transformer encoder, the pooling layer, the second fully connected layer and the second Sigmoid function, an initial sentence coherence model is constructed;

[0111] The second sample information is input into the initial sentence coherence model, and the initial sentence coherence model is iteratively trained in combination with a second loss function to obtain a created sentence coherence model.

[0112] Furthermore, when the information recognition program in the memory 130 is executed by the processor, the following steps are also implemented:

[0113] The second sample information is input into the initial sentence coherence model for the following processing:

[0114] Vectorize the second sample information through a second BERT model to obtain a second sample matrix corresponding to the second sample information;

[0115] Performing feature extraction on the second sample matrix by the third Transformer encoder to obtain a second sample feature matrix corresponding to the second sample matrix;

[0116] Processing the second sample feature matrix in a preset direction through the pooling layer to obtain a second sample vector corresponding to the second sample feature matrix;

[0117] Based on the second fully connected layer and the second Sigmoid function, the second sample vector is calculated to obtain fluency and a second fluency vector;

[0118] The fluency is calculated based on a standard cross entropy loss function to obtain a total loss value;

[0119] Based on the gradient descent algorithm and the total loss value, a repair parameter is obtained;

[0120] training the initial sentence coherence model in combination with the patching parameters; and returning to the step of: inputting the second sample information into the initial sentence coherence model to obtain coherence and a second coherence vector;

[0121] The training is terminated according to the preset number of cycles until the cycle is completed, and the created sentence fluency model is obtained.

[0122] This embodiment uses the above scheme to obtain information to be identified; input the information to be identified into the information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model; and judge whether the information to be identified is spam according to the prediction result. The trained information identification model can solve the problem of information identification accuracy and improve the accuracy of information identification. Based on the scheme of the present invention, starting from the problem of rampant spam in the real world, a sample information data set of spam and normal information in the real world is constructed, and the effectiveness of the information identification method proposed by the present invention is verified on the data set. Finally, the accuracy of spam identified by the method of the present invention is significantly improved.

[0123] Based on the above terminal device architecture but not limited to the above architecture, an embodiment of the method of the present invention is proposed.

[0124] Reference Figure 2 , Figure 2 1 is a flow chart of a first exemplary embodiment of an information identification method of the present invention. The information identification method comprises:

[0125] Step S200, obtaining information to be identified;

[0126] Step S210, inputting the information to be identified into an information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model;

[0127] Step S220: determining whether the information to be identified is spam according to the prediction result.

[0128] The information identification method in this embodiment can be applied to applications such as SMS monitoring systems. The execution subject can be an information identification device or an information identification terminal device or server. This embodiment takes an information identification device as an example. The information identification device can be integrated into terminal devices such as smart phones and tablet computers with data processing functions.

[0129] The solution of this embodiment mainly realizes the recognition of information, especially junk information, and improves the recognition accuracy of junk information.

[0130] Specifically, natural language processing refers to the technology of using the natural language used by humans to communicate with machines for interactive communication. Deep learning is to learn the inherent laws and representation levels of sample data, so that machines can have analytical learning capabilities like humans, and can recognize data such as text, images and sounds. Text classification uses a computer to automatically classify and mark a text set (or other entities or objects) according to a certain classification system or standard. It finds the relationship model between text features and text categories based on a set of labeled training texts, and then uses this learned relationship model to classify new texts. Spam identification is to determine whether a text message is spam, which is a two-classification problem. In order to pass the blocking of the text message detection system, spam will always modify harmful keywords. Therefore, spam has become a social hazard, and how to quickly and effectively identify spam has important social value and practical significance. Starting from the problem of low accuracy in spam identification, this embodiment designs an information identification model, inputs the acquired information to be identified into the information identification model for prediction, obtains the prediction result, and determines whether the information to be identified is spam based on the prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model.

[0131] The preset matrix encoder has powerful parallel computing and feature extraction capabilities, and the preset matrix encoder can be a Transformer encoder or a Word2vec model; the pre-created sentence coherence model models the relationship between information features and information categories more fully, and can solve the problems of sentence coherence such as similar word replacement and word splitting processing of spam information. In this embodiment, the preset matrix encoder is preferably a Transformer encoder, and the pre-created sentence coherence model is preferably a sentence coherence model based on a Transformer encoder.

[0132] More specifically, starting from the problem of low accuracy in spam identification, this embodiment improves the spam identification function used in applications such as SMS monitoring systems, and designs a Transformer-based information identification model. The information identification model is obtained based on the Transformer encoder and a pre-created sentence coherence model, which solves the pain points that have always existed in the industry regarding spam identification functions.

[0133] This embodiment adopts the above scheme, specifically by obtaining the information to be identified; inputting the information to be identified into the information identification model for prediction to obtain the prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model; and judging whether the information to be identified is spam according to the prediction result. By predicting the information based on the Transformer information identification model, the accuracy of information identification can be improved.

[0134] Further, refer to Figure 3 , Figure 3 The overall data flow diagram of the information identification model involved in the embodiment, the framework of the information identification model is specifically as follows Figure 3 As an example, the figure shows a BERT model, two Transformer encoders, a sentence coherence judgment model based on the Transformer encoder, an interaction layer, and a prediction layer, wherein the prediction layer includes a concatenation layer, a fully connected layer, and a Sigmoid function. This embodiment uses an information recognition model to predict information. The overall data flow of the information recognition model is as follows: Figure 3 As shown:

[0135] The SMS text and Pinyin text shown in the figure are input into the information recognition model for the following processing:

[0136] First, the text of the SMS text and the Pinyin text are represented by text vectorization through the BERT model shown in the figure, and the SMS matrix corresponding to the SMS text and the Pinyin matrix corresponding to the Pinyin text are obtained;

[0137] Then, the SMS matrix is ​​input into the pre-created Transformer-based sentence coherence model shown in the figure for judgment, and the coherence vector corresponding to the SMS text is obtained;

[0138] Then, the SMS matrix and the Pinyin matrix are respectively input into the Transformer encoder shown in the figure and another Transformer encoder shown in the figure for feature extraction, so as to obtain the SMS feature matrix corresponding to the SMS matrix and the Pinyin feature matrix corresponding to the Pinyin matrix;

[0139] Then, the SMS feature matrix and the Pinyin feature matrix are interacted through the interactive layer shown in the figure to obtain an interactive attention matrix; based on the interactive attention matrix, the SMS feature matrix and the Pinyin feature matrix are subjected to maximum pooling operations respectively to obtain the SMS vector corresponding to the SMS feature matrix and the Pinyin vector corresponding to the Pinyin feature matrix;

[0140] Then, the short message vector, the pinyin vector and the fluency vector are concatenated through the concatenation layer in the prediction layer shown in the figure to obtain a concatenated prediction vector;

[0141] Finally, the prediction vector is calculated through the fully connected layer and Sigmoid function shown in the figure to obtain the output, that is, the probability that the information to be identified is a spam message, which is used as the prediction result.

[0142] Reference Figure 4 , Figure 4 This is a schematic diagram of a modeling process of the second embodiment of the information identification method of the present invention, based on the above Figure 2 In the embodiment shown, the step S210, inputting the information to be identified into the information identification model for prediction, and obtaining the prediction result, further includes:

[0143] Creating the information recognition model specifically includes:

[0144] Step S400, obtaining first sample information for training, where the first sample information includes first junk information and first normal information;

[0145] Step S410, constructing an initial information recognition model based on the first BERT model, the Transformer encoder, the pre-created sentence coherence model, the prediction layer, and the interaction layer;

[0146] Step S420: input the first sample information into the initial information recognition model, and iteratively train the initial information recognition model in combination with a first loss function to obtain a created information recognition model.

[0147] In this embodiment, steps S400 to S420 are implemented before step S200. In other embodiments, steps S400 to S420 may also be implemented between step S200 and step S210.

[0148] Compared with the above Figure 2 The embodiment shown, this embodiment also includes a scheme for training an information recognition model.

[0149] Specifically, firstly, this embodiment collects a number of first junk information and first normal information in advance to form first sample information;

[0150] Then, the first sample information is subjected to text preprocessing, and interference symbols in the first sample information are removed based on a preset symbol library to obtain the first sample information text after the interference symbols are removed;

[0151] Convert the first sample information text into pinyin to obtain a corresponding first sample pinyin text;

[0152] Then, based on the first BERT model, the Transformer encoder, the pre-created sentence coherence model, the prediction layer, and the interaction layer, wherein the prediction layer includes the concatenation layer, the first fully connected layer, and the first Sigmoid function, an initial information recognition model is constructed;

[0153] Then, the first sample information text and the first sample pinyin text are input into the initial information recognition model for the following processing:

[0154] The first sample information text and the first sample pinyin text are represented by text vectorization through the first BERT model in the information recognition model to obtain the first sample information matrix corresponding to the first sample information text and the first sample pinyin matrix corresponding to the first sample pinyin text; the first sample information matrix is ​​input into the pre-created Transformer-based sentence coherence model for judgment to obtain the first sample coherence vector corresponding to the first sample information text; the first sample information matrix and the first sample pinyin matrix are respectively input into the first Transformer encoder and the second Transformer encoder for feature extraction to obtain the first sample information feature matrix corresponding to the first sample information matrix and the first sample pinyin feature matrix corresponding to the first sample pinyin matrix; the information feature matrix and the pinyin feature matrix are processed through the interaction layer to obtain the corresponding information vector and pinyin vector; the first sample information vector, the first sample pinyin vector and the first sample coherence vector are spliced ​​through the splicing layer in the prediction layer to obtain the spliced ​​first sample prediction vector; the prediction vector is calculated through the fully connected layer and the first Sigmoid function to obtain the first sample probability that the first sample information is junk information.

[0155] Then, since determining whether a piece of text is spam is a binary classification problem, a standard cross entropy loss function is used to train the information recognition model. When the information recognition model processes the first sample information text and the first sample pinyin text, each step needs to use the cross entropy loss function to calculate the total loss value of the first sample information recognition model of the information recognition model. The calculation formula is:

[0156]

[0157] Wherein, L represents the total loss value of the information recognition model, N represents the number of the first sample information, and y represents the parameter in the information recognition model;

[0158] Then, based on the gradient descent algorithm and the total loss value of the information recognition model, the patching parameters of the information recognition model are obtained;

[0159] Then, the initial information recognition model is trained in combination with the patching parameters of the information recognition model; and the process returns to the step of: inputting the first sample information text and the first sample pinyin text into the initial information recognition model for processing;

[0160] Finally, the number of loops is set according to actual needs, and the loop is repeated until the loop is completed, and the training is terminated to obtain the created information recognition model.

[0161] After that, the information to be identified can be predicted through the created information identification model to obtain the prediction result, and then it can be determined whether the information to be identified is spam based on the prediction result.

[0162] This embodiment creates the information recognition model through the above scheme, specifically including: obtaining the first sample information for training, the first sample information including the first junk information and the first normal information; constructing an initial information recognition model based on the first BERT model, Transformer encoder, prediction layer, interaction layer, and pre-created sentence coherence model; inputting the first sample information into the initial information recognition model, and iteratively training the initial information recognition model in combination with the first loss function to obtain the created information recognition model. Using the BERT model to implement text vectorization representation can solve the problem of polysemy; using the pre-created Transformer-based information recognition model to predict the input information to be recognized can solve the problem of information recognition accuracy and improve the accuracy of information recognition.

[0163] Reference Figure 5 , Figure 5 This is a schematic diagram of an application flow of the third embodiment of the information identification method of the present invention. Figure 2 In the embodiment shown, step S210, the Transformer encoder includes a first Transformer encoder and a second Transformer encoder, the information to be identified includes information text and pinyin text, and the step of inputting the information to be identified into the information recognition model for prediction to obtain the prediction result includes:

[0164] The information to be identified is input into the information identification model for the following processing:

[0165] Step S500, performing text vectorization representation on the information text and the pinyin text through the first BERT model in the information recognition model to obtain corresponding information matrix and pinyin matrix;

[0166] Specifically, the first BERT model is used to vectorize the input pinyin text and information text to obtain the pinyin matrix corresponding to the pinyin text and the information matrix corresponding to the information text. Since the generated vector is 768-dimensional, the pinyin matrix is ​​expressed as:

[0167] P n×768 =[p j ](j=1,2,…,n)

[0168] In the formula, n represents the total number of characters in the pinyin text, p j Represents each character vector in the pinyin text.

[0169] The information matrix is ​​expressed as:

[0170] D m×768 =[d k ](k=1,2,…,m)

[0171] In the formula, m represents the total number of characters in the information text, d k Represents each character vector in the message text.

[0172] Step S510, inputting the information matrix into a pre-created sentence coherence model for judgment to obtain a first coherence vector corresponding to the information text;

[0173] Specifically, the information matrix is ​​input into a pre-created sentence coherence model for judgment, and a first coherence vector corresponding to the information text is obtained. The first coherence vector is expressed as:

[0174] (output, t)

[0175] Where output represents the fluency of the information text, and t represents the first fluency vector of the information text.

[0176] Step S520, inputting the information matrix and the pinyin matrix into the first Transformer encoder and the second Transformer encoder respectively for feature extraction to obtain corresponding information feature matrix and pinyin feature matrix;

[0177] Specifically, the information matrix and the pinyin matrix are respectively input into the first Transformer encoder and the second Transformer encoder for feature extraction to obtain an information feature matrix corresponding to the information matrix and a pinyin feature matrix corresponding to the pinyin matrix. The pinyin feature matrix is ​​expressed as:

[0178] P n×768 =Trans1(P n×768 )

[0179] The information feature matrix is ​​expressed as:

[0180] D m×768 =Trans2(D m×768 )

[0181] Step S530, processing the information feature matrix and the phonetic feature matrix through the interaction layer to obtain corresponding information vectors and phonetic vectors;

[0182] Specifically, as an implementation scheme, the information feature matrix and the pinyin feature matrix are interacted through the interaction layer to obtain an interactive attention matrix; the interactive attention matrix and the pinyin feature matrix are subjected to a maximum pooling operation, wherein the pinyin feature matrix performs a maximum pooling operation in the row direction to obtain a pinyin vector; and the interactive attention matrix and the information feature matrix are subjected to a maximum pooling operation at the same time, wherein the information feature matrix performs a maximum pooling operation in the column direction to obtain an information vector. Thus, the pinyin vector can be obtained by performing a maximum pooling operation in the row direction, and the information vector can be obtained by performing a maximum pooling operation in the column direction, thereby realizing the interaction between the information feature matrix and the pinyin feature matrix.

[0183] As another implementation scheme, the information feature matrix and the pinyin feature matrix are interacted through the interaction layer to obtain an interactive attention matrix; the interactive attention matrix and the pinyin feature matrix are subjected to a maximum pooling operation, wherein the pinyin feature matrix performs a maximum pooling operation in the direction of columns to obtain a pinyin vector; and the interactive attention matrix and the information feature matrix are subjected to a maximum pooling operation at the same time, wherein the information feature matrix performs a maximum pooling operation in the direction of rows to obtain an information vector. Thus, the pinyin vector can be obtained by performing a maximum pooling operation in the direction of columns, and the information vector can be obtained by performing a maximum pooling operation in the direction of rows, thereby realizing the interaction between the information feature matrix and the pinyin feature matrix.

[0184] Therefore, the method of the present invention can not only obtain a pinyin vector by performing a maximum pooling operation in the row direction, and obtain an information vector by performing a maximum pooling operation in the column direction; it can also obtain a pinyin vector by performing a maximum pooling operation in the column direction, and obtain an information vector by performing a maximum pooling operation in the row direction. This embodiment takes the example of obtaining a pinyin vector by performing a maximum pooling operation in the row direction, and obtaining an information vector by performing a maximum pooling operation in the column direction.

[0185] Step S540, calculating the information vector, the pinyin vector and the first fluency vector through the prediction layer to obtain a prediction result.

[0186] Specifically, the prediction layer includes a concatenation layer, a first fully connected layer, and a first Sigmoid function.

[0187] As an implementation scheme, the information vector, the phonetic vector and the first coherence vector are concatenated into a prediction vector through a concatenation layer, and the prediction vector is calculated through a first fully connected layer and a first Sigmoid function to obtain the probability that the information to be identified is spam information, and obtain a prediction result. Thus, the information vector, the phonetic vector and the first coherence vector can be concatenated into a prediction vector for calculation to obtain a prediction result.

[0188] As another implementation scheme, the information vector, the phonetic vector and the first coherence vector are calculated respectively through the first fully connected layer and the first Sigmoid function to obtain the probability that the information to be identified is junk information and obtain the prediction result. Thus, the prediction result can be obtained by calculating the information vector, the phonetic vector and the first coherence vector respectively.

[0189] Therefore, the method of the present invention can not only calculate the prediction result by splicing the information vector, the pinyin vector and the first coherence vector into a prediction vector; it can also calculate the prediction result by separately calculating the information vector, the pinyin vector and the first coherence vector through the prediction layer. This embodiment takes the prediction result obtained by splicing the information vector, the pinyin vector and the first coherence vector into a prediction vector as an example.

[0190] Furthermore, in step S530, the step of processing the information feature matrix and the phonetic feature matrix through the interaction layer to obtain corresponding information vectors and phonetic vectors includes:

[0191] Step S531, interacting the information feature matrix and the pinyin feature matrix through the interaction layer to obtain an interactive attention matrix;

[0192] Step S532, based on the interactive attention matrix, perform maximum pooling operations on the information feature matrix and the pinyin feature matrix respectively to obtain corresponding information vectors and pinyin vectors.

[0193] Specifically, first, the information feature matrix and the pinyin feature matrix are interacted through the interaction layer to obtain the interactive attention matrix:

[0194] A n×m =PD T

[0195] In the formula, P represents the pinyin feature matrix, and D represents the information feature matrix.

[0196] Then, the interaction attention matrix and the pinyin feature matrix are subjected to the maximum pooling operation, and the pinyin vector corresponding to the pinyin feature matrix is ​​obtained as follows:

[0197]

[0198] In the formula, MP row The function represents the maximum pooling operation performed in the row direction. represents row-wise multiplication, and P represents the pinyin feature matrix.

[0199] At the same time, the interactive attention matrix and the information feature matrix are calculated to obtain the information vector corresponding to the information feature matrix:

[0200]

[0201] In the formula, MP col The function represents the maximum pooling operation performed in the column direction. represents row multiplication, and D represents the information feature matrix.

[0202] Therefore, by interacting the information feature matrix and the pinyin feature matrix through the interaction layer and performing the maximum pooling operation, the ability of the information recognition model to model the relationship between information features and information categories can be enhanced.

[0203] Furthermore, in step S540, the step of calculating the information vector, the phonetic vector and the first fluency vector through the prediction layer to obtain a prediction result includes:

[0204] Step S541, concatenating the information vector, the phonetic vector and the first coherence vector through the concatenation layer to obtain a concatenated prediction vector;

[0205] Step S542: Calculate the prediction vector through the first fully connected layer and the first Sigmoid function to obtain the probability that the information to be identified is spam information as a prediction result.

[0206] Specifically, the prediction layer includes a concatenation layer, a first fully connected layer, and a first Sigmoid function. The three vectors enter the prediction layer, and the probability that the current information is spam is calculated and used as the prediction result. The probability that the current information is spam is expressed as:

[0207] y′=S(FC(Concat(p,d,t)))

[0208] In the formula, y′ represents the predicted value of the probability that the current information is spam, p represents the pinyin vector, d represents the information vector, t represents the first coherence vector, the Concat function represents the concatenation of the pinyin vector, the information vector, and the first coherence vector into a predicted vector, the FC function represents the predicted vector passing through the first fully connected layer, and the S function represents the first Sigmoid function.

[0209] Therefore, the information vector, the pinyin vector and the first coherence vector are concatenated through the concatenation layer to obtain a concatenated prediction vector; the prediction vector is calculated through the first fully connected layer and the first Sigmoid function to obtain the probability that the information to be identified is spam information, and as a prediction result, the ability of the information recognition model to model the relationship between information features and information categories can be enhanced.

[0210] This embodiment adopts the above scheme, specifically, uses the first BERT model in the information recognition model to perform text vectorization representation on the information text and the pinyin text to obtain corresponding information matrices and pinyin matrices; inputs the information matrix into a pre-created sentence coherence model for judgment to obtain a first coherence vector corresponding to the information text; inputs the information matrix and the pinyin matrix into the first Transformer encoder and the second Transformer encoder respectively for feature extraction to obtain corresponding information feature matrices and pinyin feature matrices; interacts the information feature matrix and the pinyin feature matrix through the interaction layer to obtain an interaction attention matrix; based on the interaction attention matrix, performs maximum pooling operations on the information feature matrix and the pinyin feature matrix respectively to obtain corresponding information vectors and pinyin vectors; splices the information vector, the pinyin vector and the first coherence vector through the splicing layer to obtain a spliced ​​prediction vector; calculates the prediction vector through the first fully connected layer and the first Sigmoid function to obtain the probability that the information to be identified is spam as a prediction result. The BERT model is used to realize text vectorization representation, solving the problem of polysemy. The information text and pinyin text are interacted through the interaction layer, and the pinyin vector, information vector, and first fluency vector are concatenated into a prediction vector. The relationship between information features and information categories is more fully modeled, which improves the information recognition rate and computing efficiency, and ultimately solves the problem of information recognition accuracy and improves the accuracy of information recognition.

[0211] Reference Figure 6 , Figure 6 A schematic diagram of a modeling process of the fourth embodiment of the information identification method of the present invention is based on the above Figure 2 In the embodiment shown, step S210, the information to be identified is input into the information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model, and the step of:

[0212] Creating the sentence coherence model specifically includes:

[0213] Step S666: Acquire second sample information for training, where the second sample information includes second junk information and second normal information;

[0214] Step S600, constructing an initial sentence coherence model based on the second BERT model, the third Transformer encoder, the pooling layer, the second fully connected layer and the second Sigmoid function;

[0215] Step S610: input the second sample information into the initial sentence coherence model, and iteratively train the initial sentence coherence model in combination with a second loss function to obtain a created sentence coherence model.

[0216] In this embodiment, steps S666 to S610 are implemented before step S200. In other embodiments, steps S666 to S610 may also be implemented before steps S200 and S210, and may also be implemented before step S410, and may also be implemented between steps S400 and S410.

[0217] The second sample information in this embodiment may be the same sample information as the first sample information in the above step S420, and the second loss function may be the same loss function as the first loss function in the above step S420.

[0218] Specifically, firstly, this embodiment collects a number of second junk information and second normal information in advance to form second sample information;

[0219] Then, the second sample information is subjected to text preprocessing, and interference symbols in the second sample information are removed based on a preset symbol library to obtain a text of the second sample information after the interference symbols are removed;

[0220] Then, based on the second BERT model, the third Transformer encoder, the pooling layer, the second fully connected layer and the second Sigmoid function, an initial sentence coherence model is constructed;

[0221] Inputting the second sample information into the initial sentence coherence model to obtain coherence and a second coherence vector;

[0222] Then, the fluency is calculated based on the standard cross entropy loss function to obtain the total loss value of the sentence fluency model;

[0223] Then, based on the gradient descent algorithm and the total loss value of the sentence coherence model, the patching parameters of the sentence coherence model are obtained;

[0224] Then, the initial sentence coherence model is trained in combination with the repair parameters of the sentence coherence model; and the process returns to the step of: inputting the second sample information into the initial sentence coherence model to obtain coherence and a second coherence vector;

[0225] Finally, the training is terminated by repeating the preset number of cycles until the cycle is completed, and the created sentence fluency model is obtained.

[0226] This embodiment adopts the above scheme, specifically by obtaining second sample information for training, wherein the second sample information includes second junk information and second normal information; constructing an initial sentence coherence model based on a second BERT model, a third Transformer encoder, a pooling layer, a second fully connected layer and a second Sigmoid function; inputting the second sample information into the initial sentence coherence model, and iteratively training the initial sentence coherence model in combination with a second loss function to obtain a created sentence coherence model; obtaining information to be identified; inputting the information to be identified into an information recognition model for prediction to obtain a prediction result, wherein the information recognition model is trained based on a preset matrix encoder and a pre-created sentence coherence model; judging whether the information to be identified is junk information according to the prediction result; realizing text vectorization representation through the BERT model, which can solve the problem of polysemy; by creating a Transformer-based sentence coherence model, the relationship between information features and information categories can be fully modeled, and problems such as similar word replacement and word splitting processing can be solved.

[0227] Reference Figure 7 , Figure 7 The overall data flow diagram of the sentence coherence model involved in this embodiment is as follows: Figure 7 As an example, the figure shows the BERT model, Transformer encoder, pooling layer, fully connected layer and Sigmoid function, and constructs an initial sentence coherence model. This embodiment uses the sentence coherence model to enhance the information recognition model, thereby solving problems in sentence coherence such as similar word replacement and word splitting processing, and improving the accuracy of information recognition. The overall data flow of the sentence coherence model is as follows Figure 7 As shown:

[0228] The input text is input into the initial sentence coherence model for the following processing:

[0229] First, the input second sample text is vectorized by the illustrated BERT model to obtain a second sample matrix corresponding to the second sample text;

[0230] Then, the second sample matrix is ​​subjected to feature extraction by the Transformer encoder shown in the figure to obtain a second sample feature matrix corresponding to the second sample matrix;

[0231] Then, a maximum pooling operation is performed on the second sample feature matrix in a preset direction through the pooling layer shown in the figure to obtain a second sample vector corresponding to the second sample feature matrix;

[0232] Finally, based on the fully connected layer and Sigmoid function shown in the figure, the second sample vector is calculated to obtain the fluency and the second fluency vector.

[0233] Reference Figure 8 , Figure 8 A training flow diagram of the fifth embodiment of the information recognition method of the present invention is shown based on the above Figure 6 In the embodiment shown, step S610, inputting the second sample information into the initial sentence coherence model, and iteratively training the initial sentence coherence model in combination with the second loss function, the step of obtaining the created sentence coherence model comprises:

[0234] Step S800, inputting the second sample information into the initial sentence coherence model to obtain coherence and a second coherence vector;

[0235] Step S810, calculating the fluency based on a standard cross entropy loss function to obtain a total loss value;

[0236] Step S820, obtaining a repair parameter based on a gradient descent algorithm and the total loss value;

[0237] Step S830, training the initial sentence coherence model in combination with the patching parameters; and returning to the step of inputting the second sample information into the initial sentence coherence model to obtain coherence and a second coherence vector;

[0238] Step S840, looping for a preset number of times until the loop is completed, terminating the training, and obtaining the created sentence coherence model.

[0239] Specifically, firstly, this embodiment collects a certain amount of junk information and normal information in advance to form the second sample information;

[0240] Then, based on the second BERT model, the third Transformer encoder, the pooling layer, the second fully connected layer and the second Sigmoid function, an initial sentence coherence model is constructed;

[0241] Then, the second sample information is input into the initial sentence coherence model for the following processing:

[0242] The second sample information is vectorized by the second BERT model to obtain a second sample matrix corresponding to the second sample information; the second sample matrix is ​​feature extracted by the third Transformer encoder to obtain a second sample feature matrix corresponding to the second sample matrix; the second sample feature matrix is ​​processed in a preset direction by the pooling layer to obtain a second sample vector corresponding to the second sample feature matrix; the second sample vector is calculated based on the second fully connected layer and the second Sigmoid function to obtain fluency and second fluency vectors;

[0243] Then, because judging whether a text is fluent is a binary classification problem, the standard cross entropy loss function is used to train the sentence coherence model. When the sentence coherence model processes the second sample information, the cross entropy loss function is used in each step to calculate the total loss value of the second sample of the sentence coherence model. The calculation formula is:

[0244]

[0245] In the formula, L represents the total loss value, N represents the number of the second sample information, and y represents the parameters in the model;

[0246] Then, based on the gradient descent algorithm and the total loss value, a repair parameter of the sentence coherence model is obtained;

[0247] Then, the initial sentence coherence model is trained in combination with the repair parameters of the sentence coherence model; and the process returns to the step of: inputting the second sample information into the initial sentence coherence model for processing;

[0248] Finally, the number of loops is set according to actual needs, and the loop is repeated until the loop is completed, and the training is terminated to obtain the created sentence fluency model.

[0249] After that, the initial information recognition model can be constructed based on the first BERT model, Transformer encoder, pre-created sentence coherence model, prediction layer, and interaction layer.

[0250] This embodiment adopts the above scheme, specifically by constructing an initial sentence coherence model based on a second BERT model, a third Transformer encoder, a pooling layer, a second fully connected layer and a second Sigmoid function; inputting the second sample information into the initial sentence coherence model to obtain coherence and a second coherence vector; calculating the coherence based on a standard cross entropy loss function to obtain a total loss value; obtaining a patching parameter of the sentence coherence model based on a gradient descent algorithm and the total loss value; training the initial sentence coherence model in combination with the patching parameter of the sentence coherence model; and returning to the step of: inputting the second sample information into the initial sentence coherence model to obtain coherence and a second coherence vector; looping in this way according to a preset number of loops until the loop is completed, terminating the training, and obtaining the created sentence coherence model; inputting the information matrix into the pre-created sentence coherence model for judgment, and obtaining a first coherence vector corresponding to the information text. The trained Transformer-based sentence coherence model enhances the predictive ability of the information recognition model for identifying information, solves sentence coherence problems such as similar character replacement and split character processing that are difficult to deal with with existing technologies, and more fully models the relationship between information features and information categories, solves problems such as similar character replacement and split character processing, and improves information recognition rate and computational efficiency.

[0251] Reference Fig. 9 , Fig. 9 For the above Figure 8 The detailed flow chart of step S800 in the embodiment shown in FIG. Figure 8 In the illustrated embodiment, step S800, the step of inputting the second sample information into the initial sentence coherence model to obtain coherence and a second coherence vector includes:

[0252] The second sample information is input into the initial sentence coherence model for the following processing:

[0253] Step S900, vectorizing the second sample information through a second BERT model to obtain a second sample matrix corresponding to the second sample information;

[0254] Specifically, the second sample information input is represented by text vectorization through the second BERT model to obtain a second sample matrix corresponding to the second sample information. Since the generated vector is 768-dimensional, the second sample matrix is ​​represented as:

[0255] T l×768 =[t i ](i=1,2,…,l)

[0256] Where l is the total number of characters in the second sample information, t ifor each character vector.

[0257] Step S910, performing feature extraction on the second sample matrix through the third Transformer encoder to obtain a second sample feature matrix corresponding to the second sample matrix;

[0258] Specifically, the third Transformer encoder performs feature extraction on the second sample matrix to obtain a second sample feature matrix corresponding to the second sample matrix. The second sample feature matrix is ​​expressed as:

[0259] T l×768 =Trans(T l×768 )

[0260] Step S920, processing the second sample feature matrix in a preset direction through the pooling layer to obtain a second sample vector corresponding to the second sample feature matrix;

[0261] Specifically, as an implementation scheme, a maximum pooling operation is performed on the second sample feature matrix in the row direction through a pooling layer to obtain a second sample vector corresponding to the second sample feature matrix. The second sample vector is expressed as:

[0262] t=MP row (T l×768 )

[0263] In the formula, MP row The function represents the maximum pooling operation performed in the row direction.

[0264] As another implementation scheme, a maximum pooling operation is performed on the second sample feature matrix in a column direction through a pooling layer to obtain a second sample vector corresponding to the second sample feature matrix.

[0265] Therefore, the method of the present invention can not only obtain the second sample vector by processing the second sample feature matrix in the row direction, but also obtain the second sample vector by processing the second sample feature matrix in the column direction. This embodiment takes the example of obtaining the second sample vector by processing the second sample feature matrix in the row direction.

[0266] Step S930: Based on the second fully connected layer and the second Sigmoid function, the second sample vector is calculated to obtain fluency and a second fluency vector.

[0267] Specifically, the second sample vector is input into the fully connected neural network and then the prediction result is output through Sigmoid function inference. The prediction result is expressed as:

[0268] (output,t);

[0269] Among them, output represents the sentence fluency of the second sample text, and t represents the second fluency vector corresponding to the second sample text after maximum pooling.

[0270] This embodiment uses the above scheme to specifically vectorize the second sample information through the second BERT model to obtain the second sample matrix corresponding to the second sample information; extract features from the second sample matrix through the third Transformer encoder to obtain the second sample feature matrix corresponding to the second sample matrix; process the second sample feature matrix in a preset direction through the pooling layer to obtain the second sample vector corresponding to the second sample feature matrix; calculate the second sample vector based on the second fully connected layer and the second Sigmoid function to obtain the fluency and the second fluency vector. Using the BERT model to achieve text vectorization representation can solve the problem of polysemy; by creating a Transformer-based sentence fluency model, the relationship between information features and information categories can be fully modeled, solving problems such as similar word replacement and word splitting processing, thereby improving the accuracy of information recognition.

[0271] Reference Fig.10 , Fig.10 A flowchart of a sixth embodiment of the information identification method of the present invention is shown based on the above Figure 2 In the embodiment shown, after the step S200 of obtaining the information to be identified, the following steps are further included:

[0272] Step S100, removing interference symbols in the information to be identified based on a preset symbol library to obtain information text after removing the interference symbols;

[0273] Step S110, converting the information text into pinyin to obtain a corresponding pinyin text;

[0274] The step S210, inputting the information to be identified into the information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model, comprises:

[0275] Step S120, inputting the information text and the pinyin text into an information recognition model for prediction to obtain a prediction result, wherein the information recognition model is trained based on a preset matrix encoder and a pre-created sentence coherence model.

[0276] Specifically, the information to be identified is subjected to text preprocessing, and interference symbols can be removed based on a pre-created symbol library, for example: "*", "#", "_", "need+need+loan", "need_need_loan", etc.; pinyin, homophonic characters or traditional Chinese characters can also be processed for replacement, such as "need dai loan", "need loan" or "need loan" and other common pinyin, homophonic characters or traditional Chinese characters replacement; after processing, a relatively clean information text is obtained, and then the Chinese in the information text is converted into pinyin to generate the corresponding pinyin text, and then the information text and the pinyin text are input into the information recognition model for prediction to obtain the prediction result, and the prediction result is used to determine whether the information to be identified is spam.

[0277] This embodiment uses the above scheme to obtain information to be identified; based on a preset symbol library, remove interference symbols in the information to be identified to obtain information text after removing interference symbols; convert the information text into pinyin to obtain corresponding pinyin text; input the information to be identified into an information recognition model for prediction to obtain a prediction result, wherein the information recognition model is trained based on a preset matrix encoder and a pre-created sentence fluency model; and judge whether the information to be identified is junk information based on the prediction result. By preprocessing the text of the information to be identified, it is more efficient than word-level text processing, and solves the problems of replacing pinyin, homophonic characters or traditional Chinese characters, and segmenting text with interference symbols, thereby improving the accuracy of information recognition.

[0278] Moreover, compared with the prior art, this embodiment proposes a new method for identifying spam information. Starting from the pain points that have always existed in the industry regarding the spam information identification function, a sample information data set of spam information and normal information in the industry is constructed, and the effectiveness of the information identification method proposed by the present invention is verified on the data set. Finally, the accuracy of information identification by the method of the present invention is significantly improved.

[0279] In addition, an embodiment of the present application further provides an information identification device, the information identification device comprising:

[0280] An information acquisition module is used to acquire information to be identified;

[0281] An information recognition module, used for inputting the information to be recognized into an information recognition model for prediction to obtain a prediction result, wherein the information recognition model is trained based on a preset matrix encoder and a pre-created sentence coherence model;

[0282] Information judgment module: used to judge whether the information to be identified is junk information according to the prediction result.

[0283] For the principle and implementation process of realizing information identification in this embodiment, please refer to the above embodiments and will not be repeated here.

[0284] In addition, an embodiment of the present application also proposes a terminal device, which includes a memory, a processor, and an information identification program stored in the memory and executable on the processor, and when the information identification program is executed by the processor, the steps of the information identification method described above are implemented.

[0285] Since all the technical solutions of all the aforementioned embodiments are adopted when the information recognition program is executed by the processor, it has at least all the beneficial effects brought by all the technical solutions of all the aforementioned embodiments, which will not be described one by one here.

[0286] In addition, an embodiment of the present application further proposes a computer-readable storage medium, on which an information identification program is stored. When the information identification program is executed by a processor, the steps of the information identification method described above are implemented.

[0287] Since all the technical solutions of all the aforementioned embodiments are adopted when the information recognition program is executed by the processor, it has at least all the beneficial effects brought by all the technical solutions of all the aforementioned embodiments, which will not be described one by one here.

[0288] Compared with the prior art, the information identification method, apparatus, terminal device and storage medium proposed in the embodiments of the present application obtain information to be identified; input the information to be identified into the information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model; and judge whether the information to be identified is spam according to the prediction result. The trained information identification model can solve the problem of information identification accuracy and improve the accuracy of information identification. Based on the solution of the present invention, starting from the problem of rampant spam in the real world, a sample information data set of real-world spam and normal information is constructed, and the effectiveness of the information identification method proposed by the present invention is verified on the data set. Finally, the accuracy of spam identified by the method of the present invention is significantly improved.

[0289] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or method. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or method including the element.

[0290] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0291] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present invention.

[0292] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An information identification method, characterized in that: The information identification method comprises the following steps: Obtaining information to be identified; Inputting the information to be identified into an information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model; Determining whether the information to be identified is spam according to the prediction result; Before the step of inputting the information to be identified into the information identification model for prediction and obtaining the prediction result, the step further includes: Creating the information recognition model specifically includes: Acquire first sample information for training, where the first sample information includes first junk information and first normal information; Based on the first BERT model, Transformer encoder, pre-created sentence coherence model, prediction layer, and interaction layer, an initial information recognition model is constructed; Inputting the first sample information into the initial information recognition model, and iteratively training the initial information recognition model in combination with a first loss function to obtain a created information recognition model; The Transformer encoder includes a first Transformer encoder and a second Transformer encoder, the information to be identified includes information text and phonetic text, and the step of inputting the information to be identified into the information recognition model for prediction to obtain the prediction result includes: The information to be identified is input into the information identification model for the following processing: The information text and the pinyin text are represented by text vectorization through the first BERT model in the information recognition model to obtain corresponding information matrix and pinyin matrix; Inputting the information matrix into a pre-created sentence coherence model for judgment to obtain a first coherence vector corresponding to the information text; Input the information matrix and the pinyin matrix into the first Transformer encoder and the second Transformer encoder respectively for feature extraction to obtain corresponding information feature matrix and pinyin feature matrix; Processing the information feature matrix and the phonetic feature matrix through the interaction layer to obtain corresponding information vectors and phonetic vectors; The information vector, the phonetic vector and the first fluency vector are calculated through the prediction layer to obtain a prediction result.

2. The information identification method according to claim 1, characterized in that: The step of processing the information feature matrix and the phonetic feature matrix through the interaction layer to obtain corresponding information vectors and phonetic vectors comprises: The information feature matrix and the phonetic feature matrix are interacted with each other through the interaction layer to obtain an interactive attention matrix; Based on the interactive attention matrix, the maximum pooling operation is performed on the information feature matrix and the pinyin feature matrix respectively to obtain the corresponding information vector and pinyin vector.

3. The information identification method according to claim 1, characterized in that: The prediction layer includes a concatenation layer, a first fully connected layer, and a first Sigmoid function. The step of calculating the information vector, the pinyin vector, and the first fluency vector through the prediction layer to obtain a prediction result includes: The information vector, the phonetic vector and the fluency vector are concatenated by the concatenation layer to obtain a concatenated prediction vector; The prediction vector is calculated through the first fully connected layer and the first Sigmoid function to obtain the probability that the information to be identified is spam information as a prediction result.

4. The information identification method according to claim 1, characterized in that: The step of inputting the information to be identified into the information identification model for prediction to obtain a prediction result, wherein the information identification model is trained based on a preset matrix encoder and a pre-created sentence coherence model, further comprises: Creating the sentence coherence model specifically includes: Acquire second sample information for training, where the second sample information includes second junk information and second normal information; Based on the second BERT model, the third Transformer encoder, the pooling layer, the second fully connected layer and the second Sigmoid function, an initial sentence coherence model is constructed; The second sample information is input into the initial sentence coherence model, and the initial sentence coherence model is iteratively trained in combination with a second loss function to obtain a created sentence coherence model.

5. The information identification method according to claim 4, characterized in that: The step of inputting the second sample information into the initial sentence coherence model, and iteratively training the initial sentence coherence model in combination with the second loss function to obtain the created sentence coherence model comprises: The second sample information is input into the initial sentence coherence model for the following processing: Vectorize the second sample information through a second BERT model to obtain a second sample matrix corresponding to the second sample information; Performing feature extraction on the second sample matrix by the third Transformer encoder to obtain a second sample feature matrix corresponding to the second sample matrix; Processing the second sample feature matrix in a preset direction through the pooling layer to obtain a second sample vector corresponding to the second sample feature matrix; Based on the second fully connected layer and the second Sigmoid function, the second sample vector is calculated to obtain fluency and a second fluency vector; The fluency is calculated based on a standard cross entropy loss function to obtain a total loss value; Based on the gradient descent algorithm and the total loss value, a repair parameter is obtained; training the initial sentence coherence model in combination with the patching parameters; and returning to the step of: inputting the second sample information into the initial sentence coherence model to obtain coherence and a second coherence vector; The training is terminated according to the preset number of cycles until the cycle is completed, and the created sentence fluency model is obtained.

6. An information recognition device, characterized in that: The information identification device comprises: An information acquisition module is used to acquire information to be identified; An information recognition module is used to input the information to be recognized into an information recognition model for prediction to obtain a prediction result, wherein the information recognition model is trained based on a preset matrix encoder and a pre-created sentence coherence model; and creating the information recognition model, specifically comprising: obtaining first sample information for training, wherein the first sample information includes first junk information and first normal information; constructing an initial information recognition model based on a first BERT model, a Transformer encoder, a pre-created sentence coherence model, a prediction layer, and an interaction layer; inputting the first sample information into the initial information recognition model, and iteratively training the initial information recognition model in combination with a first loss function to obtain the created information recognition model; Information judgment module: used to judge whether the information to be identified is junk information according to the prediction result; The information identification device is also used to input the information to be identified into the information identification model for the following processing, wherein the Transformer encoder includes a first Transformer encoder and a second Transformer encoder, and the information to be identified includes information text and pinyin text; the information text and the pinyin text are represented by text vectorization through the first BERT model in the information identification model to obtain corresponding information matrix and pinyin matrix; the information matrix is ​​input into a pre-created sentence coherence model for judgment to obtain a first coherence vector corresponding to the information text; The information matrix and the pinyin matrix are respectively input into the first Transformer encoder and the second Transformer encoder for feature extraction to obtain corresponding information feature matrix and pinyin feature matrix; the information feature matrix and the pinyin feature matrix are processed by the interaction layer to obtain corresponding information vector and pinyin vector; the information vector, the pinyin vector and the first coherence vector are calculated by the prediction layer to obtain a prediction result.

7. A terminal device, characterized in that: The terminal device includes a memory, a processor, and an information identification program stored in the memory and executable on the processor. When the information identification program is executed by the processor, the steps of the information identification method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an information identification program, and when the information identification program is executed by the processor, the steps of the information identification method according to any one of claims 1 to 5 are implemented.

Images