Text verification method and device, electronic equipment and storage medium
By obtaining the categories of text to be checked and inputting them into multiple pre-trained discriminant models, combined with preset judgment levels and rules, the problem of poor text verification and recognition flexibility in the prior art is solved, and different application scenarios are realized flexibility and robustness of compliance judgment and text discrimination.
Patent Information
- Application Number
- CN202410990167.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-12-27
AI Technical Summary
In the prior art online game products, text verification and recognition flexibility is poor, making compliance judgments for different application scenarios difficult, and it is difficult to achieve distinction using a single discriminant model to adjust the threshold.
By obtaining the categories of the text to be checked and inputting them into multiple pre-trained discriminant models, combining preset judgment levels and rules, we determine whether the text is a compliant text, and using different models to improve the flexibility and robustness of the discrimination.
It realizes text compliance judgment for different application scenarios, improves the flexibility and robustness of text discrimination, and ensures the controllability and distinction of the model.
Smart Images
Figure CN119202244A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data processing technology, and in particular to a text verification method, device, electronic device and storage medium. Background Art
[0002] In online game products, it is often necessary to verify the texts input and published by users to determine whether they contain non-compliant content. In related technologies, a word list is usually used for verification. As long as the text input by the user contains keywords in the word list, non-compliant content exists artificially and needs to be intercepted. However, this method has poor recognition flexibility and can only make compliance judgments for fixed scenarios. Summary of the invention
[0003] The purpose of the present disclosure is to provide a text verification method, device, electronic device and storage medium to adjust text determination rules for different application scenarios and improve the flexibility and robustness of text determination.
[0004] In a first aspect, the present disclosure provides a text verification method, which includes: obtaining a text to be verified, inputting the text to be verified into a pre-trained classification model, and obtaining a text category corresponding to the text to be verified; wherein the text category is a detection category in a plurality of preset detection categories; inputting the text to be verified into a plurality of pre-trained discriminant models respectively, and obtaining an output result corresponding to each discriminant model, the output results being used to indicate whether the text to be verified contains sensitive information corresponding to the text category; determining a target judgment level that matches the text category corresponding to the text to be verified, and a target judgment rule corresponding to the target judgment level from a plurality of preset judgment levels; determining whether the text to be verified is a compliant text based on the target judgment rule and the output result corresponding to each discriminant model.
[0005] In a second aspect, the present disclosure provides a text verification device, which includes: a category identification module, which is used to obtain a text to be verified, input the text to be verified into a pre-trained classification model, and obtain a text category corresponding to the text to be verified; wherein the text category is a detection category in a plurality of preset detection categories; a text discrimination module, which is used to input the text to be verified into a plurality of pre-trained discrimination models respectively, and obtain an output result corresponding to each discrimination model, and the output results are used to indicate whether the text to be verified contains sensitive information corresponding to the text category; a rule determination module, which is used to determine a target judgment level that matches the text category corresponding to the text to be verified from a plurality of preset judgment levels, and a target judgment rule corresponding to the target judgment level; a compliance judgment module, which is used to determine whether the text to be verified is a compliant text based on the target judgment rule and the output result corresponding to each discrimination model.
[0006] In a third aspect, the present disclosure provides an electronic device, which includes a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above-mentioned text verification method.
[0007] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned text verification method.
[0008] The embodiments of the present disclosure bring the following beneficial effects:
[0009] The present disclosure provides a text verification method, device, electronic device and storage medium. First, the text to be verified is obtained, and the text to be verified is input into a pre-trained classification model to obtain a text category corresponding to the text to be verified; wherein the text category is a detection category in a plurality of preset detection categories; then the text to be verified is respectively input into a plurality of pre-trained discriminant models to obtain an output result corresponding to each discriminant model, and the output result is used to indicate whether the text to be verified contains sensitive information corresponding to the text category; then, from a plurality of preset judgment levels, a target judgment level matching the text category corresponding to the text to be verified and a target judgment rule corresponding to the target judgment level are determined; based on the target judgment rule and the output result corresponding to each discriminant model, it is determined whether the text to be verified is a compliant text. This method uses a method of first determining the category to which the text belongs and then determining whether it is compliant, realizes the determination criteria specified for the category, and uses different models for determination to determine the determination criteria, the model is more robust, and the flexibility of text discrimination is improved.
[0010] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.
[0011] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, the following specifically cites preferred implementation modes and describes them in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0013] Figure 1 A flowchart of a text verification method provided by an embodiment of the present disclosure;
[0014] Figure 2 A schematic diagram of a parameter setting panel provided in an embodiment of the present disclosure;
[0015] Figure 3 A schematic diagram of the structure of a text verification device provided by an embodiment of the present disclosure;
[0016] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. The components of the embodiments of the present disclosure described and shown in the drawings here can be arranged and designed in various different configurations.
[0018] Therefore, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the present disclosure claimed for protection, but merely represents selected embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present disclosure.
[0019] In online game products, it is often necessary to verify the texts input and published by users to determine whether they contain non-compliant content. In the related art, a vocabulary is usually used for verification. As long as the text input by the user contains keywords in the vocabulary, non-compliant content exists artificially and needs to be intercepted. However, this method has poor recognition flexibility and can only make compliance judgments for fixed scenarios. The related art also provides a method of using a discriminant model for verification, but the requirements for the model are actually different in different scenarios. Some scenarios have strict requirements, while others require relaxation. In the related art, one discriminant model is usually used, which is relaxed by lowering the threshold and becomes strict by raising the threshold. Since the model threshold is uncontrollable, it is difficult to achieve differentiation, and thus it is difficult to make different compliance judgments for different application scenarios.
[0020] Based on the above problems, an embodiment of the present invention provides a text verification method, device, electronic device and storage medium. This technology can be applied in text compliance verification scenarios, especially in the verification scenario of text content input by users in games.
[0021] In order to facilitate understanding of the embodiment of the present invention, a text verification method provided by the embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following specific steps:
[0022] Step S102, obtaining the text to be verified, inputting the text to be verified into a pre-trained classification model, and obtaining a text category corresponding to the text to be verified; wherein the text category is a detection category among a plurality of preset detection categories.
[0023] In specific implementation, the above-mentioned text to be verified can be any text content input by the user. After obtaining the text to be verified, it is necessary to input the text to be verified into a pre-trained classification model, which can determine which detection category the text to be verified belongs to among multiple preset detection categories, and determine the determined detection category as the text category corresponding to the text to be verified. Among them, which categories are included in the multiple detection categories can be determined according to R&D needs or user settings, and different detection categories correspond to different application scenarios, and each detection category corresponds to different sensitive information.
[0024] The above classification model can be a neural network model or a deep learning model, for example, a BERT (Bidirectional Encoder Representations from Transformers, language representation model) model plus a fully connected layer can be used as the structure of the classification model. The classification model is obtained by training with a preset training set, which contains a large number of training samples, each of which includes a training text and a category label corresponding to the training text, and the category label is used to indicate which detection category the training text belongs to among multiple detection categories.
[0025] Step S104, input the text to be verified into multiple pre-trained discriminant models respectively, and obtain the output results corresponding to each discriminant model, and the output results are used to indicate whether the text to be verified contains sensitive information corresponding to the text category.
[0026] In specific implementation, the total number of models corresponding to the above-mentioned multiple discriminant models can be determined according to research and development needs, and each discriminant model can identify sensitive information corresponding to multiple detection categories; optionally, each discriminant model is good at identifying different detection categories. Therefore, by inputting the text to be verified into multiple discriminant models, the output results corresponding to each discriminant model can be obtained, and the output results corresponding to each discriminant model may be the same or different.
[0027] The above-mentioned discriminant model can be a neural network model or a machine learning model, etc. The specific structure of the model can be determined according to the research and development needs. Among them, the model training methods corresponding to different discriminant models are the same, but the training data sets used during model training are different, so that different discriminant models can be good at identifying sensitive information corresponding to different detection categories. Among them, the model training method will be introduced in detail later.
[0028] Step S106, determining a target determination level that matches the text category corresponding to the text to be verified, and a target determination rule corresponding to the target determination level, from among a plurality of preset determination levels.
[0029] In specific implementation, multiple judgment levels and judgment rules corresponding to each judgment level can be pre-set according to research and development needs. Different judgment levels have different strictness for text verification, and different judgment levels correspond to different judgment rules, which match the strictness of the corresponding judgment level. For example, multiple judgment levels may include but are not limited to: strict, default and free, etc., wherein the strictness of text verification corresponding to strict, default and free is reduced in turn. Specifically, the above judgment rule can be that when the number of output results indicating that the text contains sensitive information in the output results corresponding to multiple discriminant models is less than a preset number threshold, the text is determined to be compliant text; the judgment rule can also be that when the output result corresponding to the specified discriminant model indicates that the text contains sensitive information, the text is determined to be non-compliant text; the judgment rule can also be based on the degree of correlation of the output results corresponding to multiple discriminant models, determine whether the text is compliant text, etc.
[0030] In actual applications, the judgment level corresponding to each detection category can be pre-set according to R&D requirements, or it can be determined according to user settings. That is, users can customize the judgment level corresponding to each detection category to meet the user's personalized needs.
[0031] Step S108, based on the target determination rule and the output results corresponding to each discrimination model, determine whether the text to be verified is a compliant text.
[0032] In a specific implementation, after obtaining the output results corresponding to the multiple discriminant models, it is possible to determine whether the text to be checked is a compliant text according to the target determination rule corresponding to the target determination level that matches the text category corresponding to the text to be checked. Usually, it is determined whether the text to be checked is a compliant text by whether the output results corresponding to the multiple discriminant models meet the target determination rule, wherein the compliant text usually does not contain sensitive information or illegal information.
[0033] A text verification method provided by an embodiment of the present invention uses a method of first determining the category to which the text belongs and then determining whether it is compliant, thereby realizing the specified judgment criteria for the category and using different models for judgment to determine the judgment criteria. The model is more robust and the flexibility of text judgment is improved.
[0034] The following embodiments are used to describe the training method of the discriminant model.
[0035] Specifically, each of the multiple discriminant models is used to identify sensitive information corresponding to multiple detection categories. It can also be understood that each discriminant model can identify whether the input text contains sensitive information corresponding to each detection category in the multiple detection categories.
[0036] In order to make multiple discriminant models better process some vertical domain text data and have a certain generalization ability, it is necessary to set each of the multiple discriminant models to be good at identifying one of the multiple detection categories; usually, different discriminant models are good at identifying different detection categories. Based on this, each of the multiple discriminant models can be trained through the following steps 10-11:
[0037] Step 10, obtaining a training data set; wherein the training data set includes multiple training samples, and the number of samples of the detection categories that the discriminant model is good at identifying contained in the multiple training samples is greater than the number of samples of other detection categories except the detection categories that the discriminant model is good at identifying.
[0038] In specific implementation, when training a discriminant model, it is necessary to determine the target detection category that the discriminant model is good at identifying, so as to obtain a training data set containing a large number of training samples of the target detection category and a small number of training samples corresponding to detection categories other than the target detection category, so that the discriminant model trained based on the training data set can better identify whether the sample to be verified contains sensitive information corresponding to the target detection category.
[0039] In a specific embodiment, assuming that the number of multiple discriminant models includes discriminant model 1, discriminant model 2 and discriminant model 3, when a discriminant model is good at identifying target detection categories, then the number of samples corresponding to the target detection categories contained in the training data set corresponding to the discriminant model is twice the number of samples corresponding to the target detection categories contained in the training data sets corresponding to other discriminant models. As shown in Table 1, the enhanced model corresponding to a detection category is also the discriminant model that is good at identifying the detection category. The number under each discriminant model represents the ratio of the number of training samples corresponding to each detection category in the corresponding training data set.
[0040] Table 1
[0041]
[0042]
[0043] Step 11: train the discriminant model based on the training data set to obtain a trained discriminant model.
[0044] In the specific implementation, each discriminant model uses its own corresponding training data set to perform model training, and a trained discriminant model can be obtained, thereby improving the generalization ability and robustness of multiple discriminant models.
[0045] In an optional embodiment, the idea of N-fold cross-validation can be used to train the judgment model during model training. At the same time, in order to improve the effect of the gun-mounting model, the traditional average data distribution can be adjusted to strengthen the idea of doubling the training data of the corresponding detection category of the model.
[0046] The following embodiments are used to describe the method of determining a judgment rule and the method of determining whether a text is compliant.
[0047] In order to customize security rules, multiple detection categories are provided in advance. Users can customize the judgment level corresponding to each detection category, and different judgment rules are provided for input and output.
[0048] In the specific implementation, before or after obtaining the text to be verified, it is necessary to respond to the judgment level setting instructions for multiple detection categories to determine the judgment level corresponding to each of the multiple detection categories. The above judgment level setting instructions can be triggered according to R&D needs and user settings, and the user can select the judgment level corresponding to each detection category from the preset multiple judgment levels according to needs.
[0049] In a specific embodiment, the discrimination level corresponding to each detection category can be determined in the parameter setting panel. Figure 2 FIG. 1 is a schematic diagram of a parameter setting panel provided by an embodiment of the present invention. Figure 2 It contains 5 detection categories, each of which is followed by a selectable judgment level. Users can select any judgment level as the corresponding judgment level for the detection category according to their needs. When the user selects "off", it means that the judgment level of this detection category is the lowest. It can also be understood that the player does not want to perform sensitive information detection on the detection category selected as off, and thus can ignore the sensitive information corresponding to this detection category. Figure 2 The judgment level corresponding to detection category 1 is highly liberal, the judgment level corresponding to detection category 2 is default, the judgment level corresponding to detection category 3 is strict detection, the judgment level corresponding to detection category 4 is closed, and the judgment level corresponding to detection category 5 is default. Among them, the order of detection levels from high to low is: strict detection, default, highly liberal and closed.
[0050] After obtaining the text to be verified, it is necessary to input the text to be verified into a pre-trained classification model to obtain the text category corresponding to the text to be verified, and then determine the target judgment level corresponding to the text category corresponding to the text to be verified from the judgment levels corresponding to multiple detection categories; wherein different judgment levels are respectively configured with corresponding judgment rules; and determine the target judgment rules corresponding to the target judgment level.
[0051] In the specific implementation, since the text category corresponding to the text to be verified is one of the multiple detection categories, the target judgment level that matches the text category corresponding to the text to be verified can be determined from the judgment levels corresponding to the multiple pre-set detection categories, and then the judgment rule corresponding to the target judgment level can be determined as the target judgment rule.
[0052] In an optional embodiment, the above-mentioned target determination rule includes: when the number of output results corresponding to multiple discriminant models indicating that the text to be verified contains sensitive information corresponding to the text category is less than a preset number threshold, determining that the text to be verified is a compliant text; based on this, the specific process of determining whether the text to be verified is a compliant text based on the target determination rule and the output results corresponding to each discriminant model may include: determining the target number of output results corresponding to multiple discriminant models indicating that the text to be verified contains sensitive information corresponding to the text category; determining whether the target number is less than the preset number threshold, if it is less, determining that the text to be verified is a compliant text; if it is not less than, determining that the text to be verified is an non-compliant text.
[0053] In specific implementation, different judgment levels correspond to different preset number thresholds configured in the judgment rules, and the judgment level is inversely proportional to the preset number threshold. That is, the higher the judgment level, the smaller the preset number threshold set in the judgment rule corresponding to the judgment level.
[0054] Table 2 is a table showing the correspondence between the determination level and the preset quantity threshold when the number of the discriminant models is 3.
[0055] Table 2
[0056] Determination level (from high to low) Preset quantity threshold Strict testing ≥1 default ≥2 High degree of freedom ≥3 closure Regardless of threshold, or ≥4
[0057] Table 2 contains 4 judgment levels, each of which corresponds to its own preset quantity threshold. Among them, the preset quantity threshold set in the judgment rule corresponding to the strict detection judgment level is 1, the preset quantity threshold set in the judgment rule corresponding to the default judgment level is 2, and the preset quantity threshold set in the judgment rule corresponding to the highly free judgment level is 3. The closed judgment level means that the corresponding judgment rule is not considered or the preset quantity threshold is set to a value greater than the total number of discriminant models.
[0058] In an optional embodiment, the specific process of determining the target number of output results indicating that the text to be verified contains sensitive information corresponding to the text category in the output results corresponding to multiple discriminant models may include: obtaining a preset discriminant threshold corresponding to the text category corresponding to the text to be verified from the preset discriminant thresholds corresponding to multiple detection categories; for the output results corresponding to each discriminant model in the multiple discriminant models, determining whether the current output result is greater than or equal to the preset discriminant threshold, and if so, determining that the current output result indicates that the text to be verified contains sensitive information corresponding to the text category; and counting the target number of output results corresponding to multiple discriminant models indicating that the text to be verified contains sensitive information corresponding to the text category.
[0059] In specific implementation, the discriminant model does not directly output the output result of whether the text to be verified contains sensitive information. In fact, the output result is a numerical value. The present invention sets different or the same preset discrimination thresholds for different detection categories. For example, the preset discrimination threshold corresponding to detection category 1 is 0.5, the preset discrimination threshold corresponding to detection category 2 is 0.3, and the preset discrimination threshold corresponding to detection category 3 is 0.6, etc. Among them, the preset discrimination threshold corresponding to each detection category can be set according to research and development requirements, and can also be user-defined.
[0060] For example, the preset discrimination threshold corresponding to detection category 1 is 0.5. If the text category corresponding to the text to be verified is detection category 1, and the value corresponding to the output result of the discrimination model is greater than or equal to 0.5, it is considered that the text to be verified contains sensitive information corresponding to detection category 1.
[0061] In an optional embodiment, since each of the multiple discriminant models is good at identifying one detection category among multiple detection categories, the target judgment rule may also include: when the output result corresponding to the discriminant model that is good at identifying the text category corresponding to the text to be verified indicates that the text to be verified contains sensitive information corresponding to the text category, determining that the text to be verified is non-compliant text.
[0062] In another optional embodiment, in order to give full play to the role of the reinforcement model corresponding to each detection category during prediction, during strict detection, the output result of the reinforcement model must be judged as non-compliant, which can effectively reduce the false recall of text. Among them, the reinforcement model corresponding to a certain detection category is a discriminant model that is good at identifying the detection category.
[0063] The present invention proposes a customizable text compliance detection scheme, which uses a method of first determining the category of the text and then determining whether it is compliant, thereby realizing the specified discrimination criteria for the category; the discrimination criteria are determined by comparing the output results of different models with preset thresholds, and the model is more robust and the threshold discrimination is more obvious. The present disclosure can be applied to any model and is convenient and easy to use. At the same time, since it takes into account both robustness and configurability, it can be better applied in a production environment, so that one service supports different businesses in multiple scenarios.
[0064] Corresponding to the above method embodiment, the embodiment of the present invention also provides a text verification device, such as Figure 3 As shown, the device comprises:
[0065] The category recognition module 30 is used to obtain the text to be verified, input the text to be verified into a pre-trained classification model, and obtain the text category corresponding to the text to be verified; wherein the text category is a detection category among multiple preset detection categories.
[0066] The text discrimination module 31 is used to input the text to be verified into multiple pre-trained discrimination models respectively, and obtain the output results corresponding to each discrimination model. The output results are used to indicate whether the text to be verified contains sensitive information corresponding to the text category.
[0067] The rule determination module 32 is used to determine, from a plurality of preset determination levels, a target determination level that matches the text category corresponding to the text to be verified, and a target determination rule corresponding to the target determination level.
[0068] The compliance determination module 33 is used to determine whether the text to be verified is a compliant text based on the target determination rule and the output result corresponding to each discrimination model.
[0069] The above-mentioned text verification device uses a method of first determining the category to which the text belongs and then determining whether it is compliant, thereby realizing the specified judgment criteria for the category and using different models to determine the judgment criteria. The model is more robust and the flexibility of text judgment is improved.
[0070] Specifically, the above-mentioned device also includes a setting module, which is used to: before obtaining the text to be verified and inputting the text to be verified into a pre-trained classification model to obtain the text category corresponding to the text to be verified, respond to the judgment level setting instructions for multiple detection categories, and determine the judgment level corresponding to each detection category in the multiple detection categories.
[0071] Furthermore, the rule determination module 32 is used to: determine the target determination level corresponding to the text category corresponding to the text to be verified from the determination levels corresponding to multiple detection categories; wherein corresponding determination rules are configured for different determination levels; and determine the target determination rule corresponding to the target determination level.
[0072] Furthermore, the above-mentioned target determination rule includes: when the number of output results indicating that the text to be verified contains sensitive information corresponding to the text category in the output results corresponding to multiple discriminant models is less than a preset number threshold, determining that the text to be verified is a compliant text; based on this, the above-mentioned compliance determination module 33 is used to: determine the target number of output results indicating that the text to be verified contains sensitive information corresponding to the text category in the output results corresponding to multiple discriminant models; determine whether the target number is less than the preset number threshold, if it is less, determine that the text to be verified is a compliant text; if it is not less than, determine that the text to be verified is an non-compliant text.
[0073] In an optional embodiment, different determination levels correspond to different preset quantity thresholds configured in the determination rules, and the determination level is inversely proportional to the preset quantity threshold.
[0074] Furthermore, the compliance determination module 33 is also used to: obtain a preset discrimination threshold corresponding to the text category corresponding to the text to be verified from the preset discrimination thresholds corresponding to multiple detection categories; determine whether the current output result is greater than or equal to the preset discrimination threshold for the output result corresponding to each discriminant model in the multiple discriminant models, and if so, determine that the current output result indicates that the text to be verified contains sensitive information corresponding to the text category; and count the number of targets in the output results corresponding to the multiple discriminant models that indicate that the text to be verified contains sensitive information corresponding to the text category.
[0075] In an optional embodiment, each of the multiple discriminant models is used to identify sensitive information corresponding to multiple detection categories.
[0076] In a specific implementation, each of the above-mentioned multiple discriminant models is good at identifying one detection category among multiple detection categories; based on this, the above-mentioned device also includes a model training module, which is used to: obtain a training data set; wherein the training data set includes multiple training samples, and the number of samples of the detection category that the discriminant model is good at identifying contained in the multiple training samples is greater than the number of samples of other detection categories except the detection category that the discriminant model is good at identifying; train the discriminant model based on the training data set to obtain a trained discriminant model.
[0077] Furthermore, the above-mentioned target determination rule includes: when the output result corresponding to the discriminant model that is good at identifying the text category corresponding to the text to be verified indicates that the text to be verified contains sensitive information corresponding to the text category, the text to be verified is determined to be non-compliant text.
[0078] The text verification device provided in the embodiment of the present disclosure has the same implementation principle and technical effects as those in the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.
[0079] The embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, the electronic device includes a processor and a memory, the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above-mentioned text verification method.
[0080] Specifically, the above-mentioned text verification method includes: obtaining the text to be verified, inputting the text to be verified into a pre-trained classification model, and obtaining a text category corresponding to the text to be verified; wherein the text category is a detection category in a plurality of preset detection categories; inputting the text to be verified into a plurality of pre-trained discrimination models respectively, and obtaining an output result corresponding to each discrimination model, and the output results are respectively used to indicate whether the text to be verified contains sensitive information corresponding to the text category; determining a target judgment level that matches the text category corresponding to the text to be verified, and a target judgment rule corresponding to the target judgment level from a plurality of preset judgment levels; and determining whether the text to be verified is a compliant text based on the target judgment rule and the output result corresponding to each discrimination model.
[0081] The above-mentioned text verification method uses the method of first determining the category to which the text belongs and then determining whether it is compliant, thereby realizing the specified judgment criteria for the category and using different models to determine the judgment criteria. The model is more robust and the flexibility of text judgment is improved.
[0082] In an optional embodiment, before obtaining the text to be verified and inputting the text to be verified into a pre-trained classification model to obtain the text category corresponding to the text to be verified, a judgment level setting instruction for multiple detection categories is responded to to determine the judgment level corresponding to each of the multiple detection categories.
[0083] In an optional embodiment, the above-mentioned step of determining a target judgment level that matches the text category corresponding to the text to be verified from a plurality of preset judgment levels, and a target judgment rule corresponding to the target judgment level, includes: determining the target judgment level corresponding to the text category corresponding to the text to be verified from the judgment levels corresponding to a plurality of detection categories; wherein corresponding judgment rules are respectively configured for different judgment levels; and determining the target judgment rule corresponding to the target judgment level.
[0084] In an optional embodiment, the target determination rule includes: when the number of output results indicating that the text to be verified contains sensitive information corresponding to the text category in the output results corresponding to multiple discriminant models is less than a preset number threshold, determining that the text to be verified is a compliant text; based on this, the step of determining whether the text to be verified is a compliant text based on the target determination rule and the output results corresponding to each discriminant model includes: determining the target number of output results indicating that the text to be verified contains sensitive information corresponding to the text category in the output results corresponding to multiple discriminant models; determining whether the target number is less than the preset number threshold, if it is less, determining that the text to be verified is a compliant text; if it is not less than, determining that the text to be verified is an non-compliant text.
[0085] In an optional embodiment, different determination levels correspond to different preset quantity thresholds configured in the determination rules, and the determination level is inversely proportional to the preset quantity threshold.
[0086] In an optional embodiment, the step of determining a target number of output results indicating that the text to be verified contains sensitive information corresponding to a text category in the output results corresponding to multiple discriminant models includes: obtaining a preset discriminant threshold corresponding to the text category corresponding to the text to be verified from the preset discriminant thresholds corresponding to multiple detection categories; for the output results corresponding to each discriminant model in the multiple discriminant models, determining whether the current output result is greater than or equal to the preset discriminant threshold, and if so, determining that the current output result indicates that the text to be verified contains sensitive information corresponding to the text category; and counting the target number of output results corresponding to multiple discriminant models indicating that the text to be verified contains sensitive information corresponding to the text category.
[0087] In an optional embodiment, each of the multiple discriminant models is used to identify sensitive information corresponding to multiple detection categories.
[0088] In an optional embodiment, each of the above-mentioned multiple discriminant models is good at identifying one detection category among multiple detection categories; each of the above-mentioned multiple discriminant models is trained in the following manner: obtaining a training data set; wherein the training data set includes multiple training samples, and the number of samples of the detection category that the discriminant model is good at identifying contained in the multiple training samples is greater than the number of samples of other detection categories except the detection category that the discriminant model is good at identifying; training the discriminant model based on the training data set to obtain a trained discriminant model.
[0089] In an optional embodiment, the above-mentioned target determination rule includes: when the output result corresponding to the discriminant model that is good at identifying the text category corresponding to the text to be verified indicates that the text to be verified contains sensitive information corresponding to the text category, determining that the text to be verified is non-compliant text.
[0090] Further, Figure 4The electronic device shown further includes a bus 102 and a communication interface 103 , and the processor 101 , the communication interface 103 and the memory 100 are connected via the bus 102 .
[0091] The memory 100 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 102 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0092] The processor 101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 101 or the instruction in the form of software. The above processor 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 100, and the processor 101 reads the information in the memory 100 and completes the steps of the method of the above embodiment in combination with its hardware.
[0093] An embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned text verification method. The specific implementation can be found in the method embodiment, which will not be repeated here.
[0094] Specifically, the above-mentioned text verification method includes: obtaining the text to be verified, inputting the text to be verified into a pre-trained classification model, and obtaining a text category corresponding to the text to be verified; wherein the text category is a detection category in a plurality of preset detection categories; inputting the text to be verified into a plurality of pre-trained discrimination models respectively, and obtaining an output result corresponding to each discrimination model, and the output results are respectively used to indicate whether the text to be verified contains sensitive information corresponding to the text category; determining a target judgment level that matches the text category corresponding to the text to be verified, and a target judgment rule corresponding to the target judgment level from a plurality of preset judgment levels; and determining whether the text to be verified is a compliant text based on the target judgment rule and the output result corresponding to each discrimination model.
[0095] The above-mentioned text verification method uses the method of first determining the category to which the text belongs and then determining whether it is compliant, thereby realizing the specified judgment criteria for the category and using different models to determine the judgment criteria. The model is more robust and the flexibility of text judgment is improved.
[0096] In an optional embodiment, before obtaining the text to be verified and inputting the text to be verified into a pre-trained classification model to obtain the text category corresponding to the text to be verified, a judgment level setting instruction for multiple detection categories is responded to to determine the judgment level corresponding to each of the multiple detection categories.
[0097] In an optional embodiment, the above-mentioned step of determining a target judgment level that matches the text category corresponding to the text to be verified from a plurality of preset judgment levels, and a target judgment rule corresponding to the target judgment level, includes: determining the target judgment level corresponding to the text category corresponding to the text to be verified from the judgment levels corresponding to a plurality of detection categories; wherein corresponding judgment rules are respectively configured for different judgment levels; and determining the target judgment rule corresponding to the target judgment level.
[0098] In an optional embodiment, the target determination rule includes: when the number of output results indicating that the text to be verified contains sensitive information corresponding to the text category in the output results corresponding to multiple discriminant models is less than a preset number threshold, determining that the text to be verified is a compliant text; based on this, the step of determining whether the text to be verified is a compliant text based on the target determination rule and the output results corresponding to each discriminant model includes: determining the target number of output results indicating that the text to be verified contains sensitive information corresponding to the text category in the output results corresponding to multiple discriminant models; determining whether the target number is less than the preset number threshold, if it is less, determining that the text to be verified is a compliant text; if it is not less than, determining that the text to be verified is an non-compliant text.
[0099] In an optional embodiment, different determination levels correspond to different preset quantity thresholds configured in the determination rules, and the determination level is inversely proportional to the preset quantity threshold.
[0100] In an optional embodiment, the step of determining a target number of output results indicating that the text to be verified contains sensitive information corresponding to a text category in the output results corresponding to multiple discriminant models includes: obtaining a preset discriminant threshold corresponding to the text category corresponding to the text to be verified from the preset discriminant thresholds corresponding to multiple detection categories; for the output results corresponding to each discriminant model in the multiple discriminant models, determining whether the current output result is greater than or equal to the preset discriminant threshold, and if so, determining that the current output result indicates that the text to be verified contains sensitive information corresponding to the text category; and counting the target number of output results corresponding to multiple discriminant models indicating that the text to be verified contains sensitive information corresponding to the text category.
[0101] In an optional embodiment, each of the multiple discriminant models is used to identify sensitive information corresponding to multiple detection categories.
[0102] In an optional embodiment, each of the above-mentioned multiple discriminant models is good at identifying one detection category among multiple detection categories; each of the above-mentioned multiple discriminant models is trained in the following manner: obtaining a training data set; wherein the training data set includes multiple training samples, and the number of samples of the detection category that the discriminant model is good at identifying contained in the multiple training samples is greater than the number of samples of other detection categories except the detection category that the discriminant model is good at identifying; training the discriminant model based on the training data set to obtain a trained discriminant model.
[0103] In an optional embodiment, the above-mentioned target determination rule includes: when the output result corresponding to the discriminant model that is good at identifying the text category corresponding to the text to be verified indicates that the text to be verified contains sensitive information corresponding to the text category, determining that the text to be verified is non-compliant text.
[0104] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a terminal device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0105] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.
[0106] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims
1. A text verification method, characterized in that: The method comprises: Obtain a text to be verified, input the text to be verified into a pre-trained classification model, and obtain a text category corresponding to the text to be verified; wherein the text category is a detection category among a plurality of preset detection categories; Inputting the text to be verified into a plurality of pre-trained discriminant models respectively, and obtaining an output result corresponding to each of the discriminant models, wherein the output results are respectively used to indicate whether the text to be verified contains sensitive information corresponding to the text category; Determine, from a plurality of preset determination levels, a target determination level that matches the text category corresponding to the text to be verified, and a target determination rule corresponding to the target determination level; Based on the target determination rule and the output result corresponding to each of the discrimination models, it is determined whether the text to be verified is a compliant text.
2. The method according to claim 1, characterized in that Before the step of obtaining the text to be verified and inputting the text to be verified into a pre-trained classification model to obtain a text category corresponding to the text to be verified, the method further includes: In response to a determination level setting instruction for the plurality of detection categories, a determination level corresponding to each of the plurality of detection categories is determined.
3. The method according to claim 2, characterized in that The step of determining, from a plurality of preset determination levels, a target determination level that matches the text category corresponding to the text to be verified, and a target determination rule corresponding to the target determination level comprises: Determine the target determination level corresponding to the text category corresponding to the text to be verified from the determination levels corresponding to the multiple detection categories; wherein corresponding determination rules are configured for different determination levels; Determine the target determination rule corresponding to the target determination level.
4. The method according to claim 1, characterized in that: The target determination rule includes: when the number of output results corresponding to the multiple discrimination models indicating that the text to be verified contains sensitive information corresponding to the text category is less than a preset number threshold, determining that the text to be verified is a compliant text; The step of determining whether the text to be verified is a compliant text based on the target determination rule and the output result corresponding to each of the discrimination models comprises: Determine a target number of output results indicating that the text to be verified contains sensitive information corresponding to the text category in the output results corresponding to the multiple discriminant models; Determine whether the target quantity is less than the preset quantity threshold; if so, determine that the text to be verified is a compliant text; if not, determine that the text to be verified is an uncompliant text.
5. The method according to claim 4, characterized in that The preset quantity thresholds configured in the determination rules corresponding to different determination levels are different, and the determination level is inversely proportional to the preset quantity threshold.
6. The method according to claim 4, characterized in that The step of determining a target number of output results indicating that the text to be verified contains sensitive information corresponding to the text category in the output results corresponding to the multiple discriminant models comprises: From the preset discrimination thresholds corresponding to the multiple detection categories, obtaining a preset discrimination threshold corresponding to the text category corresponding to the text to be verified; For the output results corresponding to each of the multiple discriminant models, determine whether the current output result is greater than or equal to the preset discrimination threshold, and if so, determine that the current output result indicates that the text to be verified contains sensitive information corresponding to the text category; Count the target number of the output results corresponding to the multiple discriminant models indicating that the text to be verified contains sensitive information corresponding to the text category.
7. The method according to claim 1, characterized in that Each of the multiple discriminant models is used to identify sensitive information corresponding to the multiple detection categories.
8. The method according to claim 7, characterized in that Each of the plurality of discriminative models is good at identifying one of the plurality of detection classes; Each of the multiple discriminant models is trained in the following manner: Acquire a training data set; wherein the training data set includes a plurality of training samples, and the number of samples of the detection category that the discriminant model is good at identifying contained in the plurality of training samples is greater than the number of samples of other detection categories except the detection category that the discriminant model is good at identifying; The discriminant model is trained based on the training data set to obtain a trained discriminant model.
9. The method according to claim 8, characterized in that The target determination rule includes: when the output result corresponding to the discriminant model that is good at identifying the text category corresponding to the text to be verified indicates that the text to be verified contains sensitive information corresponding to the text category, determining that the text to be verified is a non-compliant text.
10. A text verification device, characterized in that: The device comprises: A category recognition module is used to obtain a text to be verified, input the text to be verified into a pre-trained classification model, and obtain a text category corresponding to the text to be verified; wherein the text category is a detection category among a plurality of preset detection categories; A text discrimination module, used to input the text to be verified into a plurality of pre-trained discrimination models respectively, and obtain an output result corresponding to each of the discrimination models, wherein the output results are respectively used to indicate whether the text to be verified contains sensitive information corresponding to the text category; A rule determination module, used to determine a target determination level that matches the text category corresponding to the text to be verified, and a target determination rule corresponding to the target determination level, from a plurality of preset determination levels; The compliance determination module is used to determine whether the text to be verified is a compliant text based on the target determination rule and the output result corresponding to each of the discrimination models.
11. An electronic device, characterized in that: The electronic device includes a processor and a memory, the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the text verification method described in any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the text verification method described in any one of claims 1 to 9.