Sensitive text recognition method based on modal transformation

Abstract

The invention provides a sensitive text recognition method based on modal transformation. The sensitive text recognition method comprises the following steps: S1, preparing a data set of a variant text; s2, a step of converting the data set text modal data set of the variation text in the step S1 into an image modal data set; and S3, realizing a step of converting the text modal data set of the data set of the variation text in the step S1 into an audio modal data set. The method adopts a mode conversion technology, focuses on recognition of Chinese sensitive texts, and aims at optimizing the Chinese text auditing effect and improving the sensitive text recognition accuracy. According to the scheme, the input text information is converted into the image mode and the audio mode, so that visual and auditory features of Chinese are deeply mined. Special classifiers are respectively trained on a text mode, an image mode and an audio mode, and multi-angle analysis is carried out on an input text from multiple dimensions. And finally, by combining the classification results of the three modes, whether the input text contains sensitive content or not can be more accurately judged.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a sensitive text recognition method based on modal conversion. Background Art

[0002] Chinese is different from other languages. Even if Chinese characters are changed by replacing similar characters or similar-sounding characters, it can still clearly convey the intention and others can understand it relatively easily. However, for machines or software programs, such changes are often difficult to identify. For example, on the Internet, insults, contraband words, and certain political words are often sent out by the sender by replacing characters to mutate the original text due to the existence of sensitive word libraries. These words may cause serious content security problems for many websites or software. In the face of this challenge, although there have been many studies, such as similar word libraries, similar-sounding word libraries, Chinese error correction technology, pinyin-character text similarity calculation and model training, most methods are still limited to a single modality, that is, only focusing on the text modality where the text string is located. These methods may involve the splitting of Chinese characters' radicals and components, the extraction of Chinese character shape encoding features, the extraction of Chinese character pinyin encoding features, and the use of knowledge bases of similar-sounding and similar-shaped words, but they are usually limited to extracting features such as glyphs and pinyin from text modalities, and these features are often not rich and comprehensive enough.

[0003] In fact, the particularity of Chinese is that when it is displayed on the screen, the human eye receives this information in image mode (some similar characters will be automatically ignored and corrected at this time). At the same time, for some variant texts (that is, the sender uses homophonic characters or similar characters for some reason to replace them), the human brain will automatically try to recognize the text through the concatenation of pronunciations. Therefore, humans actually receive and recognize the information contained in the text through three modes. However, in most text review methods and systems, people only use one mode of Chinese, namely text mode. This single mode of use leads to obvious limitations of these methods and systems in practical applications. Specifically, they often cannot effectively identify and process a large number of variant texts, such as chat records on websites or systems, variant insulting texts in barrage comments, and texts involving sensitive content that may exist. The existence of these problems makes the existing text review methods and systems seem powerless in practical applications and cannot meet the increasingly complex and diverse content security and content compliance requirements.

[0004] In the prior art (application number CN202111231729.6), a method, device, equipment and storage medium for sensitive word detection based on deep learning are disclosed, and the following technical features are announced: extracting the glyph combination features, glyph encoding features and pinyin encoding features corresponding to the text to be detected through the sensitive word detection model; determining multiple initial sensitive words corresponding to the text to be detected according to the preset sensitive word library, glyph combination features, glyph encoding features and pinyin encoding features; extracting the semantic features of each initial sensitive word; extracting the semantic features of the text to be detected; determining the target sensitive word corresponding to the text to be detected according to the semantic features and the semantic features of each initial sensitive word. The following technical problems exist in the prior art: (1) The glyph combination features, glyph encoding features and pinyin encoding features in the technology are all obtained using the sensitive word detection model in a single text mode, and the features are not comprehensive or accurate enough. After all, the glyph is most easily perceived visually, while the pronunciation is most easily perceived from the audio. Extracting glyph and pinyin features from pure text is insufficient. (2) This technology also relies on a sensitive word library, in which it searches for sensitive words that match the glyph combination features, glyph encoding features, and pinyin encoding features. After all, the sensitive word library can only contain a limited number of words.

[0005] In summary, in order to solve the above technical problems, this application proposes a sensitive text recognition method based on modal conversion. Summary of the invention

[0006] The purpose of the present invention is to provide a sensitive text recognition method based on modal conversion in view of the deficiencies in the prior art, and the sensitive text recognition method based on modal conversion can well solve the above problems.

[0007] In order to achieve the above requirements, the technical solution adopted by the present invention is: to provide a sensitive text recognition method based on modal conversion, and the sensitive text recognition method based on modal conversion includes the following steps:

[0008] S1: Steps for preparing a dataset of variant texts;

[0009] S2: Implementing the step of converting the text modality dataset of the mutated text in step S1 into an image modality dataset;

[0010] S3: Implementing the step of converting the text modality dataset of the mutated text in step S1 into an audio modality dataset;

[0011] S4: Use the text modality dataset, image modality dataset, and audio modality dataset to train the corresponding text classifier, image classifier, and audio classifier respectively;

[0012] S5: combining the modal feature vectors and classification result probability values ​​output by the trained three types of classifiers, training a dynamic weight fusion network to dynamically adjust the classifier weights according to the input;

[0013] S6: When receiving new input text, convert it into image mode and audio mode, and classify it using text, image, and audio classifiers respectively, and combine the results of the three classifiers to determine whether the text is sensitive text.

[0014] Preferably, the step of S1 preparing a dataset of variant texts specifically includes:

[0015] Collect real variant-sensitive texts on the Internet and annotate their corresponding prototype texts.

[0016] Preferably, the step of S1 preparing a dataset of variant texts specifically includes:

[0017] Collect a batch of data sets containing normal texts and sensitive texts, rewrite and mutate these texts by introducing homophonic characters and near-phonetic characters, and construct a data set corresponding to normal texts and mutated texts.

[0018] Preferably, the step of implementing text-to-image mode conversion in S2 specifically includes:

[0019] The input text content is converted into Songti font through a word processing algorithm and automatically rendered into image form, that is, an image modality of the input text is generated.

[0020] Preferably, the step of implementing text-to-audio mode conversion in S3 specifically includes:

[0021] Use some of the ordinary texts in the dataset to generate corresponding Chinese audio, then construct the corresponding variant texts and corresponding audios of the ordinary texts to form a training set, and train a TTS model suitable for the variant texts in this task.

[0022] The advantages of this sensitive text recognition method based on modal conversion are as follows:

[0023] This solution has the ability to more accurately identify sensitive texts that are sent using pinyin, homophones, and similar characters. For texts with sensitive information, traditional single-modal text classification technologies often find it difficult to achieve completely accurate recognition. In contrast, this solution makes more full use of the characteristics of Chinese in visual and audio modalities, and can use broader and more comprehensive Chinese semantic features to more accurately identify these sensitive texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The same reference numerals are used in these drawings to represent the same or similar parts. The exemplary embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 The flowchart of a method for sensitive text recognition based on modal conversion according to an embodiment of the present application is schematically shown.

[0026] Figure 2 The figure schematically shows a flow chart of preparing a variant text dataset according to one embodiment of the present application.

[0027] Figure 3 The figure schematically shows a processing flow diagram of a text and image modality conversion module according to one embodiment of the present application.

[0028] Figure 4 A schematic diagram of a model for training text and audio modality conversion according to an embodiment of the present application is schematically shown.

[0029] Figure 5 The figure schematically shows a processing flow diagram of a text and audio mode conversion module according to an embodiment of the present application.

[0030] Figure 6 The figure schematically shows a schematic diagram of preparing training sets for image modality and audio modality according to one embodiment of the present application.

[0031] Figure 7 The figure schematically shows the training of text, image, and audio modality-sensitive text classifiers and the fusion of results according to one embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the objectives, technical solutions and advantages of the present application more clear, the present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0033] In the following description, references to "one embodiment", "an embodiment", "an example", "an example", etc. indicate that the embodiment or example described in this way may include specific features, structures, characteristics, properties, elements or limitations, but not every embodiment or example necessarily includes the specific features, structures, characteristics, properties, elements or limitations. In addition, repeated use of the phrase "according to one embodiment of the present application" may refer to the same embodiment, but does not necessarily refer to the same embodiment.

[0034] For the sake of simplicity, certain technical features well known to those skilled in the art are omitted in the following description.

[0035] According to an embodiment of the present application, a sensitive text recognition method based on modal conversion is provided, such as Figure 1-7 As shown, the following steps are included:

[0036] 1. Prepare a dataset of variant texts, such as Figure 2 As shown in the figure, there are mainly two methods. One is to collect real data, collect real variant sensitive texts on the Internet and annotate their corresponding prototype texts; the other is to create a high-quality synthetic dataset. First, collect a batch of datasets containing normal texts and sensitive texts, and then rewrite and mutate these texts by introducing homophonic characters, near-phonetic characters, etc., so as to construct a huge dataset corresponding to normal texts and variant texts.

[0037] 2. Implement the text-to-image modality conversion module, such as Figure 3 As shown, the core function of this module is to convert the input text content into Songti font through a word processing algorithm and automatically render it into an image form, that is, to generate an image modality of the input text.

[0038] 3. Implement the text-to-audio modality conversion module, such as Figure 4 , Figure 5 As shown, some ordinary texts in the dataset are used to generate corresponding Chinese audios, and then the corresponding variant texts and corresponding audios of the ordinary texts are constructed to form a training set. The TTS model suitable for the variant texts in this task is trained, which can convert the input text (whether it is ordinary text or variant text) into suitable Chinese audio, that is, generate the audio modality of the input text.

[0039] In order to make the Chinese audio of the variant text more reasonable after modal conversion, a dedicated TTS model is trained as a text-audio modal conversion model. The specific steps can be: first use the existing TTS model to generate the Chinese audio corresponding to the ordinary text, and correspond it one-to-one with the variant text corresponding to the ordinary text to form a data set of variant text and corresponding Chinese audio, which is mixed with the original ordinary text and corresponding Chinese audio data set as the training set of the text-audio modal conversion model; then use the text in the training set (including ordinary text and variant text) as the training input, and the corresponding Chinese audio as the training output target to perform model training, and finally obtain a text-audio modal conversion TTS model suitable for ordinary text and variant text. The trained text-audio modal conversion model can convert the input text into the corresponding audio.

[0040] 4. Such as Figure 6 As shown, all text modality datasets prepared in the first step are converted into image modality datasets and audio modality datasets respectively using a text-to-image modality conversion module and a text-to-audio modality conversion module.

[0041] 5. Use the text modality dataset, image modality dataset and audio modality dataset to train the corresponding text classifier, image classifier and audio classifier respectively, and combine the feature vectors and classification result probability values ​​of each modality output by the trained classifier to train a dynamic weight fusion network to dynamically adjust the weights of each classifier according to the input, so as to obtain the best classification result.

[0042] like Figure 7 As shown in the figure, in order to improve the performance of the entire system and strengthen the semantic features of text in different modalities, a text classifier based on the BERT architecture, an image classifier based on the VIT architecture, and an audio classifier based on Wav2Vec can be used. These classifiers can work in parallel in practical applications, thereby improving processing efficiency. By combining the modal feature vectors and classification probability values ​​output by these classifiers, a dynamic weight fusion network is trained, which can dynamically adjust the weights according to the input data to obtain the optimal classification results. The fusion performance of the dynamic weight fusion network was evaluated on the validation set, and it was found that the recognition accuracy after fusion was higher than that of any single classifier.

[0043] 6. When a new input text is received, it is converted into image mode and audio mode, and classified using text, image, and audio classifiers respectively. Finally, the results of the three classifiers are combined to determine whether the text is sensitive text.

[0044] According to one embodiment of the present application, the sensitive text identification method based on modality conversion can select a combination of any two modalities (for example, using only text and picture modalities, or only text and audio modalities, or only picture and audio modalities) for sensitive text classification, or only use a single converted modality (such as using only picture modality, or only audio modality) for classification.

[0045] According to one embodiment of the present application, the sensitive text recognition method based on modal conversion explicitly converts the plain text modality into two new modalities, image modality and audio modality, through a modal conversion module. The method proposed in this scheme can more comprehensively understand and analyze the text content by comprehensively considering the characteristics of Chinese text in these three modalities. Compared with other methods that rely on a single modality, it can more accurately identify and process variant sensitive texts, thereby improving the accuracy of sensitive text recognition in content security audits. For example, when processing text involving sensitive content, the method not only analyzes the plain text, but also combines the audio voice information and image visual information corresponding to the text to more accurately judge the true intention and sensitivity of the text. In this way, the method proposed in this scheme effectively makes up for the shortcomings of existing sensitive text recognition methods and systems, and provides a more reliable and efficient solution for practical applications.

[0046] The above-mentioned embodiments only represent several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the claims.

Claims

1. A sensitive text recognition method based on modal conversion, characterized in that: The steps include: S1: Steps for preparing a dataset of variant texts; S2: Implementing the step of converting the text modality dataset of the mutated text in step S1 into an image modality dataset; S3: Implementing the step of converting the text modality dataset of the mutated text in step S1 into an audio modality dataset; S4: Use the text modality dataset, image modality dataset, and audio modality dataset to train the corresponding text classifier, image classifier, and audio classifier respectively; S5: combining the modal feature vectors and classification result probability values ​​output by the trained three types of classifiers, training a dynamic weight fusion network to dynamically adjust the classifier weights according to the input; S6: When receiving new input text, convert it into image mode and audio mode, and classify it using text, image, and audio classifiers respectively, and combine the results of the three classifiers to determine whether the text is sensitive text.

2. The sensitive text recognition method based on modal conversion according to claim 1 is characterized in that: The step of preparing a dataset of variant texts in S1 specifically includes: Collect real variant-sensitive texts on the Internet and annotate their corresponding prototype texts.

3. The method for sensitive text recognition based on modal conversion according to claim 1, characterized in that: The step of preparing a dataset of variant texts in S1 specifically includes: Collect a batch of data sets containing normal texts and sensitive texts, rewrite and mutate these texts by introducing homophonic characters and near-phonetic characters, and construct a data set corresponding to normal texts and mutated texts.

4. The method for sensitive text recognition based on modal conversion according to claim 1, characterized in that: The step of implementing text-to-image mode conversion in S2 specifically includes: The input text content is converted into Songti font through a word processing algorithm and automatically rendered into image form, that is, an image modality of the input text is generated.

5. The method for sensitive text recognition based on modal conversion according to claim 1, characterized in that: The step of implementing text-to-audio mode conversion in S3 specifically includes: Use some of the ordinary texts in the dataset to generate corresponding Chinese audio, then construct the corresponding variant texts and corresponding audios of the ordinary texts to form a training set, and train a TTS model suitable for the variant texts in this task.

Images