Illegal text detection method and device, medium and equipment

Abstract

The invention relates to the field of finance, and particularly discloses an illegal text detection method and device, a medium and equipment, and the method comprises the steps: constructing a database containing a plurality of illegal text fragments in advance; based on each to-be-tested text fragment of a to-be-tested text, matching from the database to obtain a reference illegal text fragment corresponding to each to-be-tested text fragment; and based on each to-be-detected text fragment and a reference illegal text fragment corresponding to the to-be-detected text fragment, determining whether each to-be-detected text fragment is an illegal fragment by using a target text detection model obtained by pre-training, so as to obtain an illegal text detection result. According to the method, the detection efficiency of the illegal text can be improved, and the detection result can be more accurate and reliable.

Description

Technical Field

[0001] The present invention relates to the fields of text processing and financial technology, and in particular to a method, device, medium and equipment for detecting illegal texts. Background Art

[0002] In the financial field, salespersons are usually required to introduce financial management, insurance and other products to users, or answer questions about financial management, insurance claims and other issues raised by users. In scenarios such as insurance salespersons / credit salespersons / financial salespersons’ phone calls, customer meetings, and promotions, the content of the conversations needs to be strictly controlled, and automatic violation detection of salespersons’ conversation content has become an indispensable task. Under the control of automatic violation detection, the probability of violations can be effectively reduced, and the quality and professionalism of business communication can be greatly improved.

[0003] At present, the existing violation detection methods are mainly based on training small models or regular patterns, and using classification to detect violation points / information. However, this method requires a large amount of labeled data. Once the labeled data is missing, the quality inspection effect will drop sharply, and may even fail to reach a usable state. In addition, the scenarios in which the model is applicable have certain limitations, and the ability to adapt to copywriting / text in different scenarios is poor. For example, different text styles require the training of multiple models, resulting in low efficiency in detecting violation text. Summary of the invention

[0004] In view of this, the present invention provides a method, device, medium and equipment for detecting illegal texts, the main purpose of which is to solve the current problems of inaccurate illegal text detection and low detection efficiency.

[0005] To solve the above problems, the present application provides a method for detecting illegal texts, including:

[0006] A database containing several illegal text fragments is pre-built;

[0007] Based on each text segment to be tested of the text to be tested, matching and obtaining reference illegal text segments corresponding to each text segment to be tested from the database;

[0008] Based on each of the text segments to be tested and the reference illegal text segments corresponding to the text segments to be tested, a pre-trained target text detection model is used to determine whether each of the text segments to be tested is an illegal segment, so as to obtain an illegal text detection result.

[0009] Optionally, before obtaining reference illegal text segments corresponding to each of the text segments to be tested from the database through matching, the method further includes:

[0010] The text to be tested is segmented to obtain a plurality of text segments to be tested corresponding to the text to be tested.

[0011] Optionally, the matching of the reference illegal text segments corresponding to the text segments to be tested from the database based on the text segments to be tested specifically includes:

[0012] Performing vector conversion processing on each of the text segments to be tested to obtain a first semantic vector corresponding to each of the text segments to be tested;

[0013] Based on the first semantic vector of each text segment to be tested and the second semantic vector of each illegal text segment in the database, reference illegal text segments corresponding to each text segment to be tested are obtained by matching from each illegal text segment.

[0014] Optionally, before using the pre-trained target text detection model to determine whether each of the to-be-tested text segments is an illegal segment, the method further includes:

[0015] Based on the reference violation text segments corresponding to the text segments to be tested, obtaining reference violation information corresponding to each of the text segments to be tested;

[0016] The method of determining whether each of the text segments to be tested is a violation segment by using a pre-trained target text detection model based on each of the text segments to be tested and the reference violation text segment corresponding to the text segment to be tested specifically includes:

[0017] Based on each of the text segments to be tested and the reference violation information corresponding to the text segments to be tested, construct a target prompt sentence corresponding to each of the text segments to be tested;

[0018] Each of the target prompt sentences is respectively input into the target text detection model, so as to perform violation detection on the text segment to be tested based on the target text detection model and according to the reference violation information in the target prompt sentence, and determine whether the text segment to be tested is a violation segment.

[0019] Optionally, after constructing and obtaining the database, the method further includes:

[0020] Obtain violation information of each violation text segment;

[0021] Based on the violation information of each violation text segment in the violation text, a first correspondence relationship between the violation text and the violation information is created;

[0022] Creating a second correspondence between each illegal text segment and the illegal text;

[0023] The obtaining, based on the reference violation text segments corresponding to the text segments to be tested, reference violation information corresponding to each of the text segments to be tested specifically includes:

[0024] Based on the reference illegal text segment corresponding to the text segment to be tested and the second corresponding relationship, determining a reference illegal text including the reference illegal text segment;

[0025] Based on the reference violation text and the first corresponding relationship, reference violation information corresponding to the reference violation text is determined to obtain reference violation information corresponding to the text segment to be tested.

[0026] Optionally, before determining whether each of the text segments to be tested is a violation segment using the pre-trained target text detection model, the method further includes: determining a target text segment to be tested for violation text detection from each of the text segments to be tested, specifically including:

[0027] Determine the similarity between each test text segment and the reference violation text segment;

[0028] Based on the similarities corresponding to the text segments to be tested and a predetermined similarity threshold, a target text segment to be tested for illegal text detection is determined from the text segments to be tested.

[0029] Optionally, the violation information includes any one or more of the following: violation sentences, violation words, and violation types.

[0030] In order to solve the above problems, the present application provides a device for detecting illegal texts, comprising:

[0031] A building module, used for pre-building a database containing several illegal text fragments;

[0032] A matching module, used for matching and obtaining reference illegal text segments corresponding to each of the text segments to be tested from the database based on each of the text segments to be tested;

[0033] The detection module is used to determine whether each of the text segments to be tested is a violation segment based on each of the text segments to be tested and the reference violation text segment corresponding to the text segment to be tested, respectively, using a pre-trained target text detection model to obtain a violation text detection result.

[0034] In order to solve the above problems, the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned illegal text detection methods are implemented.

[0035] To solve the above problems, the present application provides an electronic device, which at least includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of any of the above-mentioned illegal text detection methods when executing the computer program on the memory.

[0036] The present application discloses a method, device, medium and equipment for detecting illegal texts. By constructing a database containing illegal text fragments, the text fragments to be tested can be quickly matched to obtain corresponding reference illegal text fragments. The reference illegal text fragments can be used to provide detection directions for illegal detection of the text fragments to be tested, that is, to provide illegal prompt information. The target text detection model obtained by pre-training can be used to quickly and accurately determine whether the text fragment to be tested has the same illegal situation as the reference illegal text fragment based on the detection direction / illegal prompt information provided by the illegal text fragments, thereby improving the detection efficiency. And because the target text detection model in the present application only needs to make a "yes" or "no" judgment, and does not need to identify and detect illegal information based on text fragments like the existing detection model to determine the specific illegal details such as the type of violation, the present application involves less training data in the model training stage, and the model in the present application is applicable to a variety of conversation scenarios.

[0037] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0039] Figure 1 This is a flow chart of a method for detecting illegal texts according to an embodiment of the present application;

[0040] Figure 2 This is a structural block diagram of a device for detecting illegal texts according to another embodiment of the present application;

[0041] Figure 3 This is a structural block diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION

[0042] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0043] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0044] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0045] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0046] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0047] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0048] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0049] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0050] The embodiments of the present application provide a method for detecting illegal texts, which can be specifically applied to insurance product introduction scenarios, insurance claims consultation scenarios, financial product introduction scenarios, etc. in the financial field. It can also be applied to other tangible or intangible product introduction, recommendation, and problem consultation scenarios. For example, using the method in the present application, it is possible to identify violations in the conversation text when a salesperson introduces an insurance product, it is possible to identify violations in the conversation text when a financial manager introduces a financial product, and it is possible to identify violations in the conversation text when a manual customer service representative answers user claims consultation and product consultation. The method in the present application can be applied to electronic devices such as terminals and servers. Figure 1 As shown, the method in this embodiment includes the following steps:

[0051] Step S101, pre-building a database containing a number of illegal text fragments;

[0052] In the specific implementation process of this step, the conversation texts between the salesperson and the customer in various conversation scenarios such as insurance sales, financial product introduction, insurance claims consultation, etc. can be collected in advance, and then the text / copywriting containing the predetermined banned words can be used as each illegal text / illegal copywriting. Then, each illegal text / illegal copywriting is segmented to obtain a number of illegal text fragments, and a database can be built based on each illegal text fragment later. In this step, the illegal text fragment can be understood as a text fragment containing a predetermined banned word or banned characters. Among them, banned words or banned characters refer to words or words related to uncivilized language, insulting others, false propaganda, etc. For example, when the text / copywriting contains the word "silly", the copywriting can be used as an illegal copywriting, and the text fragment where the word is located can be determined as an illegal text fragment.

[0053] Step S102, based on each text segment to be tested of the text to be tested, matching and obtaining reference illegal text segments corresponding to each text segment to be tested from the database;

[0054] During the specific implementation of this step, the voice file of the insurance salesperson / financial manager / manual customer service can be obtained, and then the voice file is converted into a format to obtain the text to be tested / copy to be tested. After obtaining the text to be tested / copy to be tested, the text to be tested can be segmented to obtain a number of text segments to be tested corresponding to the text to be tested. In this embodiment, when performing text segmentation, the text to be tested can be specifically segmented into a number of text segments to be tested according to predetermined punctuation marks, paragraphs and other information, which lays the foundation for subsequent violation detection for each text segment to be tested.

[0055] In this step, after obtaining the text segment to be tested, the reference illegal text segment corresponding to the text segment to be tested can be determined based on the similarity between the text segment to be tested and each illegal text segment in the database. Specifically, for example, the text segment to be tested includes segment A and segment B; then, from each illegal text segment in the database, the illegal text segment with the highest similarity to the text segment to be tested A can be determined as the reference illegal text segment corresponding to the text segment to be tested A. Similarly, from each illegal text segment in the database, the illegal text segment with the highest similarity to the text segment to be tested B can be determined as the reference illegal text segment corresponding to the text segment to be tested B.

[0056] Step S103, based on each of the text segments to be tested and the reference illegal text segments corresponding to the text segments to be tested, respectively use the pre-trained target text detection model to determine whether each of the text segments to be tested is an illegal segment, so as to obtain an illegal text detection result.

[0057] In this step, the target text detection model can be a large language model, specifically a large model based on in-context learning. In the specific implementation of this step, reference violation information can be obtained based on the reference violation text fragment, and a prompt sentence corresponding to the text fragment to be tested can be generated based on the reference violation information and the text fragment to be tested using a predetermined prompt prompt template. Subsequently, the prompt sentence of each text fragment to be tested can be input into the target text detection model, so as to quickly and accurately judge the violation of the text fragment to be tested in the prompt sentence based on the target text detection model, and obtain the detection results corresponding to each text fragment to be tested.

[0058] The illegal text detection method in this embodiment, by constructing a database containing illegal text fragments, can quickly match the text fragment to be tested to obtain the corresponding reference illegal text fragment. The reference illegal text fragment can be used to provide detection direction for violation detection of the text fragment to be tested, that is, to provide violation prompt information. Then, the pre-trained target text detection model can be used to quickly and accurately determine whether the text fragment to be tested has the same violation as the reference illegal text fragment based on the detection direction / violation prompt information provided by the illegal text fragment, thereby improving the detection efficiency. And because the target text detection model in this application only needs to make a "yes" or "no" judgment, and does not need to identify and detect violation information based on text fragments like the existing detection model to determine the specific violation details such as the violation type, the training data involved in the model training stage of this application is relatively small, and the model in this application is applicable to a variety of conversation scenarios.

[0059] Based on the above embodiment, another embodiment of the present application provides a method for detecting illegal texts, including the following steps:

[0060] Step S201, pre-building a database containing a number of illegal text fragments;

[0061] Step S202, performing text segmentation processing on the text to be tested to obtain a plurality of text segments to be tested corresponding to the text to be tested;

[0062] Step S203, performing vector conversion processing on each of the text segments to be tested to obtain a first semantic vector corresponding to each of the text segments to be tested; based on the first semantic vector of each of the text segments to be tested and the second semantic vector of each of the illegal text segments in the database, matching and obtaining a reference illegal text segment corresponding to each of the text segments to be tested from each of the illegal text segments;

[0063] In the specific implementation process of this step, the pre-trained semantic vector representation model can be used to convert the text segment to be tested into a first semantic vector. Similarly, the illegal text segment can be converted into a second semantic vector. When the text segment is subsequently matched, the second semantic vector with the highest similarity to the first semantic vector can be determined based on the similarity between each second semantic vector and each first semantic vector, thereby obtaining a reference illegal text segment that matches each text segment to be tested.

[0064] In this step, the conventional semantic vector representation model focuses on the similarity at the semantic level. However, in the field of quality inspection / illegal text detection, relying solely on the similarity at the semantic level will make the generalization ability of quality inspection insufficient. Strictly speaking, what this application requires is the ability of the semantic vector representation model to identify violations under "similar situations". Therefore, at the training level of the semantic vector representation model, when constructing sentence pair samples, the illegal text fragments under the same quality inspection point / text fragment to be tested are considered to belong to the positive class. Because the model's concept of "similarity" needs to be improved here, the initial sample size and training range need to be relatively large. In this embodiment, the semantic vector representation model fine-tuning / training is specifically performed before building the database.

[0065] Step S205, based on the text segment to be tested and the reference violation text segment corresponding to the text segment to be tested, obtaining reference violation information corresponding to the text segment to be tested;

[0066] In this step, the corresponding violation information can be specifically extracted from the reference violation text segment, such as the violation type, violation sentence, violation vocabulary and other violation information to obtain the reference violation information. It is also possible to extract the violation type, violation sentence, violation vocabulary and other violation information from the reference violation copy / reference violation text according to the reference violation text segment where the reference violation text segment is located, so as to obtain the reference violation information. Among them, the violation sentence means a sentence containing a banned word or banned character; the violation vocabulary means a banned word or banned character; the violation type includes: insulting others, uncivilized language, false propaganda and other violation types.

[0067] Step S206, constructing a target prompt sentence corresponding to each of the text segments to be tested based on each of the text segments to be tested and the reference violation information corresponding to the text segments to be tested;

[0068] In this step, after obtaining the reference violation information corresponding to each text segment to be tested, the target prompt sentence corresponding to the text segment to be tested can be constructed based on the reference violation information. Specifically, the prompt prompt sentence can be constructed through the prompt project. For example, the constructed prompt prompt sentence is: 'There is the following text segment [text segment]. Please combine the single sentence and the full text semantic information to analyze whether the text segment contains the following violation information [reference violation information].

[0069] Step S207, inputting each of the target prompt sentences into the target text detection model respectively, so as to perform violation detection on the text segment to be tested based on the target text detection model and according to the reference violation information in the target prompt sentence, and determine whether the text segment to be tested is a violation segment;

[0070] In this step, after obtaining each target prompt sentence, the target prompt sentence can be input into the target text detection model, and the target text detection model can be used to output the corresponding detection result, that is, output a detection result such as "is an illegal text fragment" or "is not an illegal text fragment", so that the violation detection of each text fragment to be tested is more accurate and reliable.

[0071] In the specific implementation process of this embodiment, in order to improve the detection efficiency and reduce the data processing amount of the target text detection model, before using the pre-trained target text detection model to determine whether each of the text segments to be tested is a violation segment, that is, before executing step S205, it is also possible to determine the target text segment to be tested for violation text detection from each of the text segments to be tested. The specific determination process of the target text segment to be tested is as follows: determine the similarity between each text segment to be tested and the reference violation text segment; based on the similarity corresponding to each text segment to be tested and a predetermined similarity threshold, determine the target text segment to be tested for violation text detection from each of the text segments to be tested. In other words, after determining the reference violation text segment corresponding to each text segment to be tested, it is possible to further determine whether to perform subsequent violation text detection on the text segment to be tested based on the similarity between the text segment to be tested and the reference violation text segment.

[0072] Specifically, when the similarity between the text segment to be tested and the reference illegal text segment is less than or equal to a predetermined similarity threshold, it can be directly determined that the text segment to be tested is not an illegal segment, that is, there is no need to use the target text detection model for detection. On the contrary, when the similarity between the text segment to be tested and the reference illegal text segment is greater than the predetermined similarity threshold, it is necessary to further use the pre-trained target text detection model to perform illegal text detection on the text segment to be tested. By adopting the above method, the present application can not only reduce the data processing amount of the target text detection model, but also ensure the accuracy of illegal text detection.

[0073] In the specific implementation process of this embodiment, in order to obtain the reference violation information corresponding to the text segment to be tested, after constructing the database, the violation information of each violation text segment can be further obtained; based on the violation information of each violation text segment in the violation text, a first correspondence between the violation text and the violation information is created; and at the same time, a second correspondence between each violation text segment and the violation text is created. Therefore, when the reference violation information corresponding to the text segment to be tested is obtained based on the text segment to be tested and the reference violation text segment corresponding to the text segment to be tested, that is, when executing step S205, a number of reference violation texts containing the reference violation text segment can be determined by means of an inverted index based on the reference violation text segment corresponding to the text segment to be tested and the second correspondence; then, the reference violation information corresponding to each reference violation text is further determined by means of a forward index based on the reference violation text and the first correspondence, so as to obtain the reference violation information corresponding to the text segment to be tested.

[0074] For example, the database in this application includes several illegal text fragments such as illegal text fragment a, illegal text fragment b, illegal text fragment c, illegal text fragment d, etc. Illegal text fragment a and illegal text fragment b belong to illegal text / illegal copy A, illegal text fragment c and illegal text fragment d belong to illegal text / illegal copy B, thereby establishing a second corresponding relationship between illegal text A and illegal text fragment a and illegal text fragment b, and at the same time establishing a second corresponding relationship between illegal text B and illegal text fragment c and illegal text fragment d. Since each illegal text fragment corresponds to a piece of illegal information, for example, illegal text fragment a corresponds to illegal information 1, illegal text fragment b corresponds to illegal information 2, illegal text fragment c corresponds to illegal information 3, and illegal text fragment d corresponds to illegal information 4, thereby establishing a first corresponding relationship between illegal text A and illegal information 1 and illegal information 2, and at the same time establishing a first corresponding relationship between illegal text B and illegal information 3 and illegal information 4. In this embodiment, after the second corresponding relationship is created, the violation text A can be determined as the reference violation text from each violation text / violation copy based on the reference violation text segment b matched by the text segment B to be tested, and then the violation information 1 and violation information 2 can be determined as the reference violation information based on the reference violation text A and the second corresponding relationship. Subsequently, the target prompt sentence can be generated based on the violation information 1, violation information 2 and the text segment to be tested, and then input into the target text detection model, and the target text detection model is used to determine whether the text segment B to be tested has violation information 1 or violation information 2, thereby improving the detection efficiency of the violation text and ensuring the accuracy of the detection results.

[0075] In the specific implementation process of this embodiment, the target text detection model can be pre-trained, and the specific training process is as follows:

[0076] Step 1: Collect several sample text fragments

[0077] Step 2: determine the sample text where each sample text segment is located, and obtain sample violation information based on the sample text to obtain sample violation information corresponding to each sample violation text segment, and configure a violation label for each sample text segment.

[0078] Step 3: construct a sample prompt sentence based on each sample text segment and sample violation information;

[0079] In this step, a prompt sample prompt statement can be constructed through the prompt project. For example, the constructed prompt sample prompt statement is: 'There is the following text fragment [sample text fragment]. Please combine the single sentence and the full text semantic information to analyze whether the text fragment contains the following violation information [sample violation information].

[0080] Step 3: Input each sample prompt sentence into the initial text detection model, detect the sample violation information in the sample prompt sentence based on the initial text detection model, perform violation text detection on the sample text segment, and obtain the initial detection result corresponding to each sample text segment;

[0081] In this step, the initial text detection model may specifically be an existing large language model, also referred to as a large model.

[0082] Step 4: Based on the initial detection results corresponding to each sample text segment and the violation labels corresponding to each sample text segment, determine whether the predetermined model training stop condition is met. When the predetermined training stop adjustment is not met, adjust the model parameters of the initial text detection model to obtain the current text detection model, and based on the current text detection model, re-perform violation text detection on the sample text segments according to the sample violation information in the sample prompt sentence to obtain the current detection results corresponding to each sample text segment. Until it is determined based on the current detection results that the predetermined training stop condition is met, the current text detection model is determined as the target text detection model.

[0083] In this embodiment, since the large model itself has strong "world knowledge" and basic "reasoning ability", the fine-tuning of the large model can be achieved through the training process described above in this application. Appropriate fine-tuning is intended to make it more adaptable to the detection capabilities of the illegal text detection scenario, especially for the analysis of the matching degree between the given suspected illegal text fragment and the text fragment to be tested. Therefore, a small number of specific samples (about one thousand) are constructed for supervised fine-tuning (supervised fine-tuning, English abbreviation: SFT). Excessive training will greatly reduce the background knowledge and reasoning ability of the large model. It is worth emphasizing that the core of the training of this application is to hope that the large model can learn the ability to identify, rather than the ability to identify specific quality inspection points, so in the process of sample construction, it is necessary to fully cover more and more mixed quality inspection points / sample text fragments. In this embodiment, the fine-tuning of the large model is carried out before the database is built and before the storage. By adopting the above-mentioned method of this application to train and obtain the target text detection model, the foundation is laid for the subsequent rapid and accurate detection of illegal texts based on the target text detection model.

[0084] Another embodiment of the present application provides a device for detecting illegal texts, such as Figure 2 As shown, including:

[0085] A construction module 11 is used to pre-construct a database containing a number of illegal text fragments;

[0086] A matching module 12 is used for matching and obtaining reference illegal text segments corresponding to each of the text segments to be tested from the database based on each of the text segments to be tested;

[0087] The detection module 13 is used to determine whether each of the text segments to be tested is a violation segment based on each of the text segments to be tested and the reference violation text segment corresponding to the text segment to be tested, respectively, using the target text detection model obtained by pre-training to obtain the violation text detection result.

[0088] In the specific implementation process of this embodiment, the illegal text detection device also includes a segmentation module, and the segmentation module is used to: perform text segmentation processing on the text to be tested to obtain a plurality of text segments to be tested corresponding to the text to be tested.

[0089] In the specific implementation process of this embodiment, the matching module is specifically used to: perform vector conversion processing on each of the text segments to be tested to obtain a first semantic vector corresponding to each of the text segments to be tested; based on the first semantic vector of each text segment to be tested and the second semantic vector of each illegal text segment in the database, match from each illegal text segment to obtain a reference illegal text segment corresponding to each text segment to be tested.

[0090] In the specific implementation process of this embodiment, the illegal text detection device further includes an acquisition module, which is used to: acquire reference illegal information corresponding to each of the text segments to be tested based on the reference illegal text segments corresponding to each of the text segments to be tested;

[0091] The detection module is specifically used to: construct a target prompt sentence corresponding to each of the text segments to be tested based on the text segments to be tested and the reference violation information corresponding to the text segments to be tested; input each of the target prompt sentences into the target text detection model respectively, so as to perform violation detection on the text segment to be tested based on the target text detection model and according to the reference violation information in the target prompt sentence, and determine whether the text segment to be tested is a violation segment.

[0092] In the specific implementation process of this embodiment, the construction module is also used to: obtain the violation information of each violation text segment; based on the violation information of each violation text segment in the violation text, create a first correspondence between the violation text and the violation information; create a second correspondence between each violation text segment and the violation text;

[0093] The acquisition module is specifically used to: determine the reference violation text containing the reference violation text segment based on the reference violation text segment corresponding to the text segment to be tested and the second corresponding relationship; determine the reference violation information corresponding to the reference violation text based on the reference violation text and the first corresponding relationship, so as to obtain the reference violation information corresponding to the text segment to be tested.

[0094] In the specific implementation process of this embodiment, the illegal text detection device also includes a determination module, which is used to: determine a target text segment to be tested for illegal text detection from each of the text segments to be tested, specifically used to: determine the similarity between each text segment to be tested and a reference illegal text segment; based on the similarity corresponding to each text segment to be tested and a predetermined similarity threshold, determine the target text segment to be tested for illegal text detection from each of the text segments to be tested.

[0095] In the specific implementation process of this embodiment, the violation information includes any one or more of the following: violation sentences, violation words, and violation types.

[0096] The illegal text detection device in this embodiment, by constructing a database containing illegal text fragments, can quickly match the text fragment to be tested to obtain the corresponding reference illegal text fragment. The reference illegal text fragment can be used to provide a detection direction for the violation detection of the text fragment to be tested, that is, to provide violation prompt information. Then, the pre-trained target text detection model can be used to quickly and accurately determine whether the text fragment to be tested has the same violation as the reference illegal text fragment based on the detection direction / violation prompt information provided by the illegal text fragment, thereby improving the detection efficiency. And because the target text detection model in this application only needs to make a "yes" or "no" judgment, and does not need to identify and detect violation information based on text fragments like the existing detection model to determine the specific violation details such as the violation type, the training data involved in the model training stage of this application is relatively small, and the model in this application is applicable to a variety of conversation scenarios.

[0097] Another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:

[0098] Step 1: Pre-build a database containing several illegal text fragments;

[0099] Step 2: Based on each text segment to be tested of the text to be tested, obtain reference illegal text segments corresponding to each text segment to be tested from the database by matching;

[0100] Step three: based on each of the text segments to be tested and the reference illegal text segments corresponding to the text segments to be tested, use the pre-trained target text detection model to determine whether each of the text segments to be tested is an illegal segment to obtain an illegal text detection result.

[0101] The specific implementation process of the above method steps can be found in the embodiments of any of the above-mentioned illegal text detection methods, and this embodiment will not be repeated here.

[0102] The storage medium in the present application, by constructing a database containing illegal text fragments, can quickly match the text fragment to be tested to obtain the corresponding reference illegal text fragment. The reference illegal text fragment can be used to provide detection direction for violation detection of the text fragment to be tested, that is, to provide violation prompt information. Then, the pre-trained target text detection model can be used to quickly and accurately determine whether the text fragment to be tested has the same violation as the reference illegal text fragment based on the detection direction / violation prompt information provided by the illegal text fragment, thereby improving the detection efficiency. And because the target text detection model in the present application only needs to make a "yes" or "no" judgment, and does not need to identify and detect violation information based on text fragments like the existing detection model to determine the specific violation details such as the violation type, the present application involves less training data in the model training stage, and the model in the present application is applicable to a variety of conversation scenarios.

[0103] Another embodiment of the present application provides an electronic device, such as Figure 3 As shown, at least a memory 1 and a processor 2 are included, the memory 1 stores a computer program, and the processor 2 implements the following method steps when executing the computer program on the memory 1:

[0104] Step 1: Pre-build a database containing several illegal text fragments;

[0105] Step 2: Based on each text segment to be tested of the text to be tested, obtain reference illegal text segments corresponding to each text segment to be tested from the database by matching;

[0106] Step three: based on each of the text segments to be tested and the reference illegal text segments corresponding to the text segments to be tested, use the pre-trained target text detection model to determine whether each of the text segments to be tested is an illegal segment to obtain an illegal text detection result.

[0107] The specific implementation process of the above method steps can be found in the embodiments of any of the above-mentioned illegal text detection methods, and this embodiment will not be repeated here.

[0108] The electronic device in the present application, by constructing a database containing illegal text fragments, can quickly match the text fragment to be tested to obtain the corresponding reference illegal text fragment. The reference illegal text fragment can be used to provide detection direction for violation detection of the text fragment to be tested, that is, to provide violation prompt information. Then, the pre-trained target text detection model can be used to quickly and accurately determine whether the text fragment to be tested has the same violation as the reference illegal text fragment based on the detection direction / violation prompt information provided by the illegal text fragment, thereby improving the detection efficiency. And because the target text detection model in the present application only needs to make a "yes" or "no" judgment, and does not need to identify and detect violation information based on text fragments like the existing detection model to determine the specific violation details such as the violation type, the present application involves less training data in the model training stage, and the model in the present application is applicable to a variety of conversation scenarios.

[0109] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application. The non-Company software tools or components that appear in the embodiments of the present application are merely examples and do not represent actual use.

Claims

1. A method for detecting illegal texts, characterized in that: include: A database containing several illegal text fragments is pre-built; Based on each text segment to be tested of the text to be tested, matching and obtaining reference illegal text segments corresponding to each text segment to be tested from the database; Based on each of the text segments to be tested and the reference illegal text segments corresponding to the text segments to be tested, a pre-trained target text detection model is used to determine whether each of the text segments to be tested is an illegal segment, so as to obtain an illegal text detection result.

2. The method according to claim 1, characterized in that Before obtaining reference illegal text segments corresponding to each of the text segments to be tested from the database through matching, the method further includes: The text to be tested is segmented to obtain a plurality of text segments to be tested corresponding to the text to be tested.

3. The method according to claim 1, characterized in that The matching of the text segments to be tested based on the text to be tested to obtain reference illegal text segments corresponding to the text segments to be tested from the database specifically includes: Performing vector conversion processing on each of the text segments to be tested to obtain a first semantic vector corresponding to each of the text segments to be tested; Based on the first semantic vector of each text segment to be tested and the second semantic vector of each illegal text segment in the database, reference illegal text segments corresponding to each text segment to be tested are obtained by matching from each illegal text segment.

4. The method according to claim 1, characterized in that Before using the pre-trained target text detection model to determine whether each of the to-be-tested text segments is an illegal segment, the method further includes: Based on the reference violation text segments corresponding to the text segments to be tested, obtaining reference violation information corresponding to each of the text segments to be tested; The method of determining whether each of the text segments to be tested is a violation segment by using a pre-trained target text detection model based on each of the text segments to be tested and the reference violation text segment corresponding to the text segment to be tested specifically includes: Based on each of the text segments to be tested and the reference violation information corresponding to the text segments to be tested, construct a target prompt sentence corresponding to each of the text segments to be tested; Each of the target prompt sentences is respectively input into the target text detection model, so as to perform violation detection on the text segment to be tested based on the target text detection model and according to the reference violation information in the target prompt sentence, and determine whether the text segment to be tested is a violation segment.

5. The method according to claim 4, characterized in that After constructing and obtaining the database, the method further comprises: Obtain violation information of each violation text segment; Based on the violation information of each violation text segment in the violation text, a first correspondence relationship between the violation text and the violation information is created; Creating a second correspondence between each illegal text segment and the illegal text; The obtaining, based on the reference violation text segments corresponding to the text segments to be tested, reference violation information corresponding to each of the text segments to be tested specifically includes: Based on the reference illegal text segment corresponding to the text segment to be tested and the second corresponding relationship, determining a reference illegal text including the reference illegal text segment; Based on the reference violation text and the first corresponding relationship, reference violation information corresponding to the reference violation text is determined to obtain reference violation information corresponding to the text segment to be tested.

6. The method according to claim 1, characterized in that Before using the pre-trained target text detection model to determine whether each of the test text segments is a violation segment, the method further includes: determining a target test text segment for violation text detection from each of the test text segments, specifically including: Determining the similarity between each of the text segments to be tested and the reference illegal text segment; Based on the similarities corresponding to the text segments to be tested and a predetermined similarity threshold, a target text segment to be tested for illegal text detection is determined from the text segments to be tested.

7. The method according to claim 4, characterized in that The violation information includes any one or more of the following: violation sentences, violation words, and violation types.

8. A device for detecting illegal texts, characterized in that: include: A building module, used for pre-building a database containing several illegal text fragments; A matching module, used for matching and obtaining reference illegal text segments corresponding to each of the text segments to be tested from the database based on each of the text segments to be tested; The detection module is used to determine whether each of the text segments to be tested is a violation segment based on each of the text segments to be tested and the reference violation text segment corresponding to the text segment to be tested, respectively, using a pre-trained target text detection model to obtain a violation text detection result.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the illegal text detection method described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: The method comprises at least a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the illegal text detection method according to any one of claims 1 to 7 when executing the computer program in the memory.

Images