Text security classification method and device based on multi-model hierarchical collaboration

Abstract

The invention discloses a text security classification method and device based on multi-model hierarchical collaboration, and belongs to the field of artificial intelligence security. The method comprises the steps that a plurality of upstream sub-models are utilized to conduct corresponding dimension judgment on a to-be-classified text in parallel, so that judgment results and confidence degree scores of all the upstream sub-models on all the dimensions of the to-be-classified text are obtained; wherein each uplink sub-model corresponds to one law and regulation index dimension; determining whether to use a downstream final examination model to continue classification based on the confidence score; and inputting the to-be-classified text, the judgment result output by at least part of the upstream sub-model and the confidence score into a downstream final examination model, and performing multi-source attention fusion classification by using the downstream final examination model to obtain a classification result. Therefore, according to the scheme, the reliability and the accuracy can be improved, rough processing of uncertain information is avoided, and the judgment dimension with strong alignment of laws and regulations is realized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, and in particular to a text security classification method and apparatus based on multi-model hierarchical collaboration. Background Technology

[0002] Existing text security classification methods primarily rely on single classification models for review. The inherent limitations of these models lead to inconsistent review results, with a persistent risk of false positives and false negatives, failing to meet high reliability requirements. Furthermore, traditional classification models often output low-confidence judgments when faced with semantically ambiguous text, text with multiple interpretations, or text falling into a security "grey area." Their handling of such "uncertain" information is typically crude: either directly labeling it as "unsafe," causing false positives and impacting user experience, or simply discarding it or labeling it as "unknown," directly passing potential risks downstream and creating security vulnerabilities. Current technologies generally lack an intelligent and reliable mechanism to process and route such high-value uncertain information.

[0003] Furthermore, many security tools offer a broad "safe / unsafe" binary judgment. However, in actual compliance operations, risks need to be categorized in a granular manner. Existing technologies lack a multi-dimensional judgment system that directly aligns with these legal and regulatory requirements, making it difficult to directly use audit results for compliance reporting and risk attribution.

[0004] Therefore, there is an urgent need to provide a new text security classification method based on multi-model hierarchical collaboration. Summary of the Invention

[0005] To address the issues of low reliability of single models in text security classification methods, crude handling of uncertain information, and lack of judgment dimensions strongly aligned with laws and regulations, this invention provides a text security classification method and apparatus based on multi-model hierarchical collaboration.

[0006] On the one hand, a text security classification method based on multi-model hierarchical collaboration is provided, the method comprising:

[0007] Several upstream sub-models are used to make judgments on the corresponding dimensions of the text to be classified in parallel, so as to obtain the judgment results and confidence scores of each upstream sub-model on the text to be classified in each dimension; wherein, each upstream sub-model corresponds to a regulatory indicator dimension.

[0008] Based on the confidence score, determine whether to continue classification using the downstream final review model; otherwise, conduct manual review.

[0009] If so, the text to be classified, at least part of the judgment results and confidence scores output by the upstream sub-models are input into the downstream final review model to perform multi-source attention fusion classification using the downstream final review model to obtain the classification result.

[0010] On the other hand, a text security classification device based on multi-model hierarchical collaboration, based on the steps described in any embodiment of the method in the specification, is provided. The device includes:

[0011] The upstream unit is used to make judgments on the corresponding dimensions of the text to be classified in parallel using several upstream sub-models, so as to obtain the judgment results and confidence scores of each upstream sub-model on the text to be classified in each dimension; wherein, each upstream sub-model corresponds to a regulatory indicator dimension.

[0012] The routing unit is used to determine, based on the confidence score, whether to continue classification using the downstream final review model; otherwise, manual review is required.

[0013] The final review unit is used to input the text to be classified, at least part of the judgment results and confidence scores output by the upstream sub-models into the downstream final review model, so as to use the downstream final review model to perform multi-source attention fusion classification and obtain the classification result.

[0014] On the other hand, a computer device is provided, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for executing the computer program stored in the memory to implement the steps of the method described above.

[0015] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of the method described above.

[0016] On the other hand, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0017] The technical solution provided by this invention can bring at least the following beneficial effects:

[0018] By leveraging the upstream heterogeneous model pool for collaborative analysis, the biases and limitations of individual models are effectively mitigated. The downstream model, acting as the final reviewer, further delves into the in-depth reasoning and logical integration of these initial review opinions. Its decision-making process is far more reliable and intelligent than any single model or simple voting mechanism. The requirements of various regulations are transformed into a judgment indicator system, achieving strong alignment with laws and regulations and resolving compliance challenges. By determining whether to continue using the downstream final review model for classification or conduct manual review based on confidence scores, uncertain information is no longer crudely divided into binary categories. Instead, it is intelligently identified, separated, and routed to the most suitable human expert, avoiding false positives for normal content while ensuring that high-risk and complex cases are not overlooked, achieving a dual improvement in review accuracy and user experience. Therefore, this solution improves reliability and accuracy, avoids the crude handling of uncertain information, and achieves a judgment dimension strongly aligned with laws and regulations. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a text security classification method based on multi-model hierarchical collaboration provided by an embodiment of the present invention;

[0021] Figure 2 This is a structural diagram of a text security classification device based on multi-model hierarchical collaboration provided in an embodiment of the present invention;

[0022] Figure 3 This is a hardware architecture diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0024] The following describes the specific implementation of the above concept.

[0025] Please refer to Figure 1This invention provides a text security classification method based on multi-model hierarchical collaboration, the method comprising:

[0026] Step 100: Use several upstream sub-models to make judgments on the corresponding dimensions of the text to be classified in parallel, so as to obtain the judgment results and confidence scores of each upstream sub-model on the text to be classified in each dimension; wherein, each upstream sub-model corresponds to a regulatory indicator dimension.

[0027] Step 102: Determine whether to continue classification using the downstream final review model based on the confidence score; otherwise, conduct manual review.

[0028] Step 104: If yes, input the text to be classified, at least part of the judgment results and confidence scores of the upstream sub-models into the downstream final review model, so as to use the downstream final review model to perform multi-source attention fusion classification and obtain the classification result.

[0029] In this embodiment of the invention, the upstream heterogeneous model pool effectively mitigates the biases and limitations of individual models through collaborative analysis. The downstream model, acting as the final reviewer, further delves into the in-depth reasoning and logical integration of these initial review opinions. Its decision-making process is far more reliable and intelligent than any single model or simple voting mechanism. The requirements of various regulations are transformed into a judgment indicator system, achieving strong alignment with laws and regulations and resolving compliance challenges. By determining whether to continue using the downstream final review model for classification or conduct manual review based on confidence scores, uncertain information is no longer crudely divided into binary categories. Instead, it is intelligently identified, separated, and routed to the most suitable human expert, avoiding false positives for normal content while ensuring that high-risk and complex cases are not overlooked, thus achieving a dual improvement in review accuracy and user experience. Therefore, this solution improves reliability and accuracy, avoids the crude handling of uncertain information, and achieves a judgment dimension strongly aligned with laws and regulations.

[0030] The following description Figure 1 The execution method of each step is shown.

[0031] For step 100:

[0032] In this embodiment of the invention, the regulatory indicator dimension may include other adverse information such as illegal or irregular activities, bias or discrimination. A judgment indicator system is constructed for each dimension, and upstream sub-models are trained to form a heterogeneous upstream model pool. The text to be classified is simultaneously input into each sub-model in the upstream model pool. Through a carefully designed prompt word engineering, each sub-model is required to independently judge the text to be classified on each dimension according to the constructed judgment indicator system, outputting "safe" or "unsafe," and providing a confidence score (e.g., a floating-point number between 0 and 1) for each judgment result, thus completing parallel multi-dimensional judgment.

[0033] Regarding step 102:

[0034] In some implementations, the step "determining whether to continue classification using the downstream final review model based on the confidence score" includes steps S1-S5:

[0035] S1, concatenate the confidence scores of all upstream sub-models into a confidence matrix.

[0036] Each upstream sub-model performs parallel judgments on the text to be classified based on a pre-defined multi-dimensional indicator system, and outputs its own judgment result with confidence level. These results are concatenated into a DxD confidence matrix C, where D is the number of dimensions and also the number of sub-models, and C... ij This represents the confidence level of the i-th model in judging the j-th dimension.

[0037] S2, obtain the importance vector containing the importance of each dimension. Based on the confidence matrix and the importance vector, sum the judgment variances of all dimensions according to the importance of each dimension to obtain the uncertainty index.

[0038] In some implementations, the uncertainty index is calculated as follows:

[0039] U = Var(C)·β T

[0040] β=[β1,β2,...,β D ]

[0041] In the formula, U is the uncertainty index, C is the confidence matrix, Var(C) is a 1×D row vector, · represents the vector dot product operation, the j-th element is the variance of the j-th column of the confidence matrix, β is the importance vector, and D is the number of upstream sub-models.

[0042] In this embodiment, the importance vector β is a row vector, where each element represents the sensitivity or importance weight of each dimension. For example, the weight of a specific sensitive dimension can be set much higher than that of other undesirable information. Var(C) is a 1xD row vector, where the j-th element is the variance of the j-th column of the confidence matrix C, i.e., the variance of the confidence scores of all models for the i-th dimension. The larger the variance, the greater the divergence of the sub-models on this dimension, and the higher the uncertainty. By calculating the uncertainty index U, the variances of the divergence scores on all dimensions are weighted and summed according to the importance B of the dimensions to obtain a final score characterizing the global ambiguity.

[0043] S3. Obtain the reliability vector containing the prior reliability weights of each upstream sub-model. Based on the confidence matrix and the reliability vector, weight the confidence scores of all upstream sub-models using the prior reliability weights of each upstream sub-model to obtain the confidence matrix.

[0044] In some implementations, the credibility matrix is ​​calculated as follows:

[0045]

[0046] α=[α1,α2,...,α D ]

[0047] In the formula, S is the credibility matrix, α is the reliability vector, and I 1×D The vector consists of 1xD rows, all equal to 1, where D is the number of upstream sub-models. Let represent the Hadamard product, and C be the confidence matrix.

[0048] In this embodiment, the reliability vector α is a Dx1 column vector, where each element represents the prior reliability weight of each upstream sub-model. This weight can be calibrated based on historical performance data, with models having higher reliability receiving higher weights. 1×D The reliability vector 'a' is broadcast as a DxD matrix, where each row represents the reliability coefficient of the corresponding model. The original judgment confidence matrix C is then weighted by the prior reliability of the model it belongs to, using the calculated confidence matrix S, to obtain a more valuable confidence score.

[0049] S4. When the uncertainty index exceeds the pre-set human review threshold, the judgment between the characterization sub-models becomes chaotic, and the text to be classified is directly sent to the human review platform.

[0050] In this embodiment, when the uncertainty index exceeds a pre-set human review threshold, it indicates that the text to be classified is highly controversial, and the judgments between models are highly chaotic. The system directly routes the text to be classified to the human review platform, without calling downstream models, to avoid interference from erroneous information and waste of computing resources. This is the first-level questionable routing mechanism.

[0051] S5. When the uncertainty index is less than or equal to the preset human review threshold, the invalid judgment results in the confidence matrix are filtered using the preset information filtering threshold, and the judgment results and confidence scores used as input to the downstream final review model are integrated.

[0052] In this embodiment, when the uncertainty index is less than or equal to a pre-set human review threshold, the system generates a filter mask based on the credibility matrix S. If any element in the credibility matrix is ​​less than the pre-set information filtering threshold, it means that the corresponding model's judgment result for the corresponding dimension lacks credibility, and this result will be filtered out. This filters invalid judgment results. All judgment results that pass the credibility test are assembled into a structured input. If all results pass the test, it is the full input; if some results are filtered, it is the selective input. This integrated structured input is then sent to the downstream final review model.

[0053] Regarding step 104:

[0054] In some implementations, step 104 may include:

[0055] The downstream final review model uses a text encoder to encode the text to be classified, thus obtaining a context matrix;

[0056] For each upstream sub-model that filters the input, a judgment vector is generated from the judgment result and confidence score output by the sub-model, so as to stack the judgment vectors of at least some upstream sub-models to obtain a judgment feature matrix.

[0057] By using a self-attention layer to enhance the computation of the context matrix, a text representation rich in context information is obtained;

[0058] By utilizing a cross-attention fusion layer, based on text representation and judgment feature matrix, the correlation between each word in the text representation and the judgment vector of each model in the judgment feature matrix is ​​quantified to obtain the fusion matrix;

[0059] Pooling layers are used to aggregate information from the fusion matrix to obtain a global feature vector;

[0060] The global feature vector is input into a multilayer perceptron classifier to obtain the final classification probability distribution.

[0061] In this embodiment, to enable the model to fully understand the syntactic and semantic dependencies within the text, the context matrix H is first... text Input a self-attention layer and compute a context-rich text representation H′. text .

[0062] Specifically,

[0063]

[0064] In the formula, W Q W K and W V These are the learnable weights.

[0065] In some implementations, the fusion matrix is ​​obtained as follows:

[0066]

[0067] In the formula, H fused Let H′ be the fusion matrix. text W′ is the text representation output by the self-attention layer. Q 、W′ K and W′ V H is the weight. judge To determine the characteristic matrix, d k To determine the dimension of a vector.

[0068] In this embodiment, a cross-attention fusion layer is used to deeply fuse textual and judgment information. The cross-attention fusion layer calculates the correlation between each word in the text representation and the judgment result of each sub-model. The final fusion matrix H is obtained. fused Each vector contains both the semantics of the original text and a weighted fusion of the most relevant judgment information, achieving a deep alignment between judgment and evidence. This is achieved using the fusion matrix H. fused By performing global pooling and multilayer perceptron classification, a high-quality classification probability distribution can be obtained.

[0069] In addition, if the downstream final review model is still unable to make a clear judgment of "safe" or "unsafe" after analyzing all the high-quality preliminary review opinions, and its final output conclusion is "questionable", the system will also divert the text to be classified to the manual review platform. This is the second-level questionable routing mechanism.

[0070] In summary, this solution effectively mitigates the biases and limitations of individual models through the consultation of an upstream heterogeneous model pool. The downstream model, acting as the final reviewer, further delves into the in-depth reasoning and logical integration of these initial review opinions. Its decision-making process is far more reliable and intelligent than any single model or simple voting mechanism. It transforms the requirements of various regulations into a judgment indicator system, achieving strong alignment with laws and regulations and resolving compliance challenges. By determining whether to continue using the downstream final review model for classification or conduct manual review based on confidence scores, it avoids the crude binary division of uncertain information, instead intelligently identifying, separating, and routing it to the most suitable human expert. This avoids false positives on normal content while ensuring that high-risk and complex cases are not overlooked, achieving a dual improvement in review accuracy and user experience. Therefore, this solution improves reliability and accuracy, avoids the crude handling of uncertain information, and achieves a judgment dimension strongly aligned with laws and regulations.

[0071] Please refer to Figure 2This invention provides a text security classification device based on multi-model hierarchical collaboration, used to implement the steps of any method embodiment in the specification. The device includes:

[0072] Upstream unit 201 is used to make judgments on the corresponding dimensions of the text to be classified in parallel using several upstream sub-models, so as to obtain the judgment results and confidence scores of each upstream sub-model on the text to be classified in each dimension; wherein, each upstream sub-model corresponds to a regulatory indicator dimension.

[0073] Routing unit 202 is used to determine whether to continue classification using the downstream final review model based on the confidence score; otherwise, manual review is performed.

[0074] The final review unit 203 is used to input the text to be classified, at least part of the judgment results and confidence scores output by the upstream sub-models into the downstream final review model, so as to use the downstream final review model to perform multi-source attention fusion classification and obtain the classification result.

[0075] It should be noted that the above device embodiments and method embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0076] Embodiments of this application also provide a computer device, please refer to... Figure 3 The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the text security classification method based on multi-model hierarchical collaboration provided in the above-described method embodiments.

[0077] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the text security classification method based on multi-model hierarchical collaboration provided in the above-described method embodiments.

[0078] Embodiments of this application also provide a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium and executes the computer program, causing the computer device to perform any of the text security classification methods based on multi-model hierarchical collaboration described in the above embodiments.

[0079] For ease of description, the above devices or apparatuses are described separately according to their functions, divided into various modules or units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0080] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments of this application.

[0081] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0082] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A text security classification method based on multi-model hierarchical collaboration, characterized in that, include: Several upstream sub-models are used to make judgments on the corresponding dimensions of the text to be classified in parallel, so as to obtain the judgment results and confidence scores of each upstream sub-model on the text to be classified in each dimension; wherein, each upstream sub-model corresponds to a regulatory indicator dimension. Based on the confidence score, determine whether to continue classification using the downstream final review model; otherwise, conduct manual review. If so, the text to be classified, at least part of the judgment results and confidence scores output by the upstream sub-models are input into the downstream final review model to perform multi-source attention fusion classification using the downstream final review model to obtain the classification result.

2. The method as described in claim 1, characterized in that, The step of determining whether to continue classification using the downstream final review model based on the confidence score includes: The confidence scores of all upstream sub-models are concatenated into a confidence matrix; Obtain the importance vector containing the importance of each dimension. Based on the confidence matrix and the importance vector, sum the judgment variances of all dimensions according to the importance of each dimension to obtain the uncertainty index. Obtain a reliability vector containing the prior reliability weights of each upstream sub-model. Based on the confidence matrix and the reliability vector, weight the confidence scores of all upstream sub-models using the prior reliability weights of each upstream sub-model to obtain a confidence matrix. When the uncertainty index is greater than the preset human review threshold, the judgment between the characterization sub-models is confused, and the text to be classified is directly sent to the human review platform. When the uncertainty index is less than or equal to a pre-set human review threshold, invalid judgment results in the confidence matrix are filtered using a pre-set information filtering threshold, and the judgment results and confidence scores used as input to the downstream final review model are integrated.

3. The method as described in claim 2, characterized in that, The uncertainty index is calculated as follows: U=Var(C)·β T β＝[β1,β2,...,β D ] In the formula, U is the uncertainty index, C is the confidence matrix, Var(C) is a 1×D row vector, · represents the vector dot product operation, the j-th element is the variance of the j-th column of the confidence matrix, β is the importance vector, and D is the number of upstream sub-models.

4. The method as described in claim 2, characterized in that, The credibility matrix is ​​calculated as follows: α＝[α1,α2,...,α D ] In the formula, S is the credibility matrix, α is the reliability vector, and I 1×D The vector consists of 1xD rows, all equal to 1, where D is the number of upstream sub-models. Let C represent the Hadamard product, and C be the confidence matrix.

5. The method as described in claim 1, characterized in that, The process involves inputting the text to be classified, at least a portion of the judgment results and confidence scores output by the upstream sub-models into the downstream final review model, to perform multi-source attention fusion classification using the downstream final review model, and obtaining classification results, including: The downstream final review model uses a text encoder to encode the text to be classified to obtain a context matrix; For each of the upstream sub-models that filter the input, a judgment vector is generated from the judgment result and confidence score output by the sub-model, so as to stack the judgment vectors of at least some of the upstream sub-models to obtain a judgment feature matrix; By using a self-attention layer to enhance the computation of the context matrix, a text representation rich in context information is obtained. Using a cross-attention fusion layer, based on the text representation and the judgment feature matrix, the correlation between each word in the text representation and the judgment vector of each model in the judgment feature matrix is ​​quantified to obtain the fusion matrix; The fusion matrix is ​​aggregated using a pooling layer to obtain a global feature vector; The global feature vector is input into a multilayer perceptron classifier to obtain the final classification probability distribution.

6. The method as described in claim 5, characterized in that, The fusion matrix is ​​obtained in the following way: In the formula, H fused Let H′ be the fusion matrix. text W′ is the text representation output by the self-attention layer. Q 、W′ K and W′ V H is the weight. judge Let d be the judgment feature matrix. k Let be the dimension of the judgment vector.

7. A text security classification device based on multi-model hierarchical collaboration, used to implement the steps of the method described in any one of claims 1-6, characterized in that, include: The upstream unit is used to make judgments on the corresponding dimensions of the text to be classified in parallel using several upstream sub-models, so as to obtain the judgment results and confidence scores of each upstream sub-model on the text to be classified in each dimension; wherein, each upstream sub-model corresponds to a regulatory indicator dimension. The routing unit is used to determine, based on the confidence score, whether to continue classification using the downstream final review model; otherwise, manual review is required. The final review unit is used to input the text to be classified, at least part of the judgment results and confidence scores output by the upstream sub-models into the downstream final review model, so as to use the downstream final review model to perform multi-source attention fusion classification and obtain the classification result.

8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images