Information hierarchical processing method, device, equipment, storage medium and program product

Abstract

This application provides an information classification processing method, apparatus, device, storage medium, and program product, belonging to the field of data processing technology. The method includes: receiving information to be processed sent by a terminal device; determining the risk level corresponding to the information to be processed; performing risk detection on the information to be processed using a risk detection method corresponding to the risk level, and obtaining a risk detection result; if the risk detection result indicates an input anomaly, sending a security prompt message to an auditing terminal, causing the auditing terminal to output the security prompt message; if the risk detection result indicates a normal result, post-processing the information to be processed using a response generation model to obtain response information; and sending the response information to the terminal device, causing the terminal device to output the response information. This method solves the problem that current response processes cannot cope with multiple types of attacks.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an information classification processing method, apparatus, equipment, storage medium, and program product. Background Technology

[0002] Generative large models, as a core breakthrough in the field of artificial intelligence, have been widely applied in key industries such as finance, healthcare, education, and media.

[0003] Currently, as the capabilities of related technologies improve, the likelihood of attacks they face also increases exponentially. Traditional defense solutions are often designed to target a single type of attack.

[0004] There is an urgent need for a generative large model application method that can cope with multiple attack types. Summary of the Invention

[0005] This application provides information classification processing methods, apparatus, devices, storage media, and program products to address the problem that current response processes cannot cope with various types of attacks.

[0006] In a first aspect, embodiments of this application provide an information classification processing method, comprising: receiving information to be processed sent by a terminal device; determining the risk level corresponding to the information to be processed; performing risk detection on the information to be processed using a risk detection method corresponding to the risk level, and obtaining a risk detection result; if the risk detection result indicates an input abnormality, sending a security prompt message to an auditing terminal to enable the auditing terminal to output a security prompt message; if the risk detection result indicates a normal result, using a response generation model to post-process the information to be processed to obtain a response message; and sending the response message to the terminal device to enable the terminal device to output a response message.

[0007] In one possible implementation, determining the risk level of the information to be processed includes: determining the information entropy and similar attack frequency of the information to be processed; weighting and summing the information entropy and similar attack frequency to obtain a risk value; and finding the correspondence between a preset risk value range and a risk level based on the risk value to obtain the risk level of the information to be processed.

[0008] In one possible implementation, determining the information entropy and similar attack frequency of the information to be processed includes: obtaining the occurrence probability of characters in the information to be processed; and determining the information entropy corresponding to the information to be processed based on the occurrence probability.

[0009] In one possible implementation, a risk detection method corresponding to the risk level is used to detect the risk of the information to be processed and obtain the risk detection result. This includes: if the risk level is the first risk level, then a preset logical rule is used to detect the risk of the information to be processed and obtain the risk detection result; if the risk level is the second risk level, then a preset detection model is used to detect the risk of the information to be processed and obtain the risk detection result, wherein the second risk level is higher than the first risk level; if the risk level is the third risk level, then the format of the information to be processed is converted to obtain the reverse generated text; based on the reverse generated text and the information to be processed, the risk detection result is determined, wherein the third risk level is higher than the second risk level.

[0010] In one possible implementation, the information to be processed is format-converted to obtain reverse-generated text, including: converting the information to be processed into an image format to obtain an image to be converted; and converting the image to be converted into text to obtain reverse-generated text.

[0011] In one possible implementation, the risk detection result is determined based on the reverse-generated text and the information to be processed, including: determining the similarity between the information to be processed and the reverse-generated text; if the similarity is less than the similarity threshold, the input anomaly is determined as the risk detection result, otherwise the input normal is determined as the risk detection result.

[0012] In one possible implementation, before using the response generation model to post-process the information to be processed and obtain the response information if the risk detection result is normal, the method further includes: training a teacher model using training data, wherein the training data includes adversarial data; extracting training samples from the training data; inputting the training samples into the teacher model and the student model to obtain a first output value from the teacher model and a second output value from the student model; calculating a loss value based on the first output value and the second output value; if the loss value is greater than a preset loss value threshold, optimizing the student model using the loss value, and repeating the steps of extracting training samples to calculate the loss value until the loss value is less than the loss value threshold, and then determining the student model as the response generation model.

[0013] Secondly, embodiments of this application provide an information classification processing apparatus, comprising: an information receiving module for receiving information to be processed sent by a terminal device; a level determination module for determining the risk level corresponding to the information to be processed; a risk detection module for performing risk detection on the information to be processed using a risk detection method corresponding to the risk level, and obtaining a risk detection result; a prompt output module for sending a security prompt message to an auditing terminal if the risk detection result indicates an input abnormality, so that the auditing terminal outputs a security prompt message; and a response generation module for post-processing the information to be processed using a response generation model if the risk detection result indicates a normality, obtaining response information; and sending the response information to the terminal device, so that the terminal device outputs response information.

[0014] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0016] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0017] The information classification processing method, apparatus, device, storage medium, and program product provided in this application receive information to be processed sent by a terminal device, determine the risk level corresponding to the information to be processed, perform risk detection using a detection method corresponding to the risk level, and accurately obtain the risk detection result. If the risk detection result indicates an input abnormality, a prompt is sent to the review terminal for manual review. If the risk detection result indicates a normal result, a response generation model is used to generate response information, thereby achieving classified risk processing and ensuring security while ensuring response efficiency. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] Figure 1 A schematic diagram illustrating the scenario of the information classification processing method provided in this application;

[0020] Figure 2A flowchart illustrating the information hierarchical processing method provided in this application embodiment;

[0021] Figure 3 This is a schematic diagram of the structure of the information classification processing device provided in the embodiments of this application;

[0022] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0023] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0025] Generative large models, as a core breakthrough technology in the field of artificial intelligence, have deeply empowered key industries such as finance, healthcare, education, and media.

[0026] However, as the capabilities of these models continue to iterate and upgrade, the risk of attacks on them also increases exponentially. Traditional defense solutions are mostly designed for single attack types and are difficult to adapt to complex and ever-changing attack scenarios. Therefore, there is an urgent need for a security application solution for generative large models that can effectively cope with multiple attack types.

[0027] To address the above technical problems, the inventors propose the following technical concept: First, the input content is classified into risk levels, and different anomaly detection methods are used for input content with different risk levels to obtain detection results. If the detection results are normal, a response generation model capable of resisting abnormal input is then used to output response information.

[0028] This application is applied to scenarios involving the graded processing of information. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0029] Figure 1 A schematic diagram illustrating a scenario for the information hierarchical processing method provided in this application. For example... Figure 1 In this scenario, the components include: terminal device 101, server 102, and audit terminal 103.

[0030] In the specific implementation process, the terminal device 101 and the review terminal 103 may include computers, servers, tablets, mobile phones, PDAs (personal digital assistants), and laptops, etc., which can input data.

[0031] Server 102 can be implemented using a single server or a cluster of multiple servers with more powerful processing capabilities and higher security. Where possible, it can also be replaced by a computer or laptop with strong computing power.

[0032] The connection between server 102 and terminal device 101 can be either wired or wireless.

[0033] Terminal device 101 is used to send information to be processed to the server. The server is used to detect the risk level of the information to be processed and perform risk detection on the information to be processed according to the detection method corresponding to the risk level. If the risk detection result is that the input is abnormal, the server sends a security prompt message to the audit terminal 103 so that the audit terminal outputs a security prompt message. Otherwise, the information to be processed is input into the response generation model to obtain the response information, and the response information is sent to terminal device 101.

[0034] It is understood that the scenarios illustrated in the embodiments of this application do not constitute a specific limitation on the information classification processing method. In other feasible embodiments of this application, the above scenarios may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and are not limited here. Figure 1 The scenario shown can be implemented by hardware, software, or a combination of both.

[0035] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0036] Figure 2 This is a flowchart illustrating the information hierarchical processing method provided in an embodiment of this application. The execution entity of this embodiment may be... Figure 1The server 102 in this embodiment can also be a computer and / or a mobile phone, etc., and this embodiment does not impose any special restrictions on it. Figure 2 As shown, the method includes:

[0037] S201: Receive pending information sent by the terminal device.

[0038] In this step, pending information sent by the terminal device can be received through methods such as receiving messages and data packets. Pending information may include business inquiries.

[0039] S202: Determine the risk level corresponding to the information to be processed.

[0040] In this step, the risk level of the information to be processed can be determined by extracting risk characterization indicators (information entropy, frequency of historical similar attacks).

[0041] S203: Use the risk detection method corresponding to the risk level to perform risk detection on the information to be processed, and obtain the risk detection results.

[0042] In this step, based on the risk level determined in S202, the detection method with the corresponding intensity is dynamically scheduled.

[0043] Risk detection methods can include fixed rule detection, model detection, and multi-model joint detection.

[0044] S204: If the risk detection result indicates an input anomaly, a security alert message is sent to the audit terminal so that the audit terminal outputs the security alert message.

[0045] In this step, a security alert containing basic characteristics of the abnormal information can be sent to the audit terminal through a preset communication interface.

[0046] Among them, the basic characteristics of abnormal information include the type of abnormality and the risk level.

[0047] S205: If the risk detection result is normal, the response generation model is used to post-process the information to be processed to obtain response information. The response information is then sent to the terminal device so that the terminal device can output response information.

[0048] In this step, the response generation model can be a lightweight model trained with an intrinsically robust hardening layer (DKD).

[0049] As can be seen from the description of the above embodiments, the embodiments of this disclosure determine the risk level corresponding to the information to be processed by receiving the information to be processed sent by the terminal device, and perform risk detection using the detection method corresponding to the risk level to accurately obtain the risk detection result. If the risk detection result is that the input is abnormal, a prompt is sent to the review terminal for manual review. If the risk detection result is normal, a response generation model is used to generate response information, thereby realizing the graded processing of risks and ensuring security while ensuring response efficiency.

[0050] In one possible implementation, step S202 above, determining the risk level corresponding to the information to be processed, includes:

[0051] S2021: Determine the information entropy and similar attack frequency of the information to be processed.

[0052] This step involves converting the information to be processed into a standard format (e.g., unifying text encoding to UTF-8, unifying image resolution and format), then obtaining the frequency of occurrence of each character (text character, encoded character corresponding to image pixels, etc.) in the information, dividing the frequency of each character by the total number of characters to obtain the probability of occurrence for each character, and inputting the probability of occurrence for each character into the information entropy calculation formula to obtain the information entropy. The frequency of similar attacks can be obtained by searching a historical attack database, filtering out input records similar to the current information to be processed, and statistically analyzing the percentage of times attacks have occurred in these similar input records.

[0053] S2022: The risk value is obtained by weighted summation of information entropy and similar attack frequency.

[0054] In this step, the weights of information entropy and similar attack frequency can be preset by the staff.

[0055] S2023: Based on the risk value, find the correspondence between the preset risk value range and the risk level to obtain the risk level corresponding to the information to be processed.

[0056] In this step, a matching risk value range can be determined based on the risk value, and then the correspondence between the preset risk value range and the risk level can be found based on the risk value range to obtain the risk level corresponding to the information to be processed.

[0057] The correspondence between risk value ranges and risk levels can be pre-set by staff based on experimental data and stored in formats such as tables, key-value pairs, and files.

[0058] As can be seen from the description of the above embodiments, the embodiments of this disclosure achieve a comprehensive representation of the intrinsic characteristics of information and the correlation with historical attacks by using two risk indicators, namely top-down information entropy and similar attack frequency, and weighted fusion. Then, the abstract risk value is transformed into a concrete level label by using level mapping, which ultimately provides a scientific and reliable decision-making basis for the precise scheduling of subsequent differentiated defense detection resources, and ensures the balance between the defense efficiency and accuracy of the input detection layer from the source.

[0059] In one possible implementation, step S2021 above, determining the information entropy and similar attack frequency of the information to be processed, includes:

[0060] S20211: Obtain the probability of occurrence of characters in the information to be processed.

[0061] In this step, the occurrence count of each character (text character, encoded character corresponding to image pixel, etc.) in the information to be processed can be obtained, and then divided by the total number of characters to obtain the occurrence probability of each character.

[0062] S20212: Determine the information entropy corresponding to the information to be processed based on the probability of occurrence.

[0063] In this step, the probability of occurrence can be input into the information entropy calculation formula to obtain the corresponding information entropy.

[0064] The formula for calculating information entropy is as follows:

[0065]

[0066] In the formula, H represents information entropy. This represents the probability of character i appearing, where n is the total number of characters.

[0067] As can be seen from the description of the above embodiments, the embodiments of this disclosure provide reliable basic data for information entropy calculation by accurately counting the probability of character occurrence, and transform the disorder of the information to be processed into a quantifiable information entropy index, successfully achieving the preliminary characterization of the inherent abnormality of information, and effectively supporting the accurate classification of subsequent risk levels.

[0068] In one possible implementation, step S203 above involves using a risk detection method corresponding to the risk level to perform risk detection on the information to be processed, obtaining the risk detection result, including:

[0069] S2031: If the risk level is the first risk level, then the information to be processed is subjected to risk detection using preset logical rules to obtain the risk detection result.

[0070] In this step, the first risk level is low risk, prioritizing detection efficiency and controlling computational overhead. Pre-defined logical rules can be built based on common low-risk attack characteristics found in business scenarios (such as common sensitive words in text or basic format violations in images). By matching the information to be processed against the rule base, it quickly determines whether clear low-risk attack characteristics exist. The underlying principle is that low-risk attacks typically possess fixed characteristic patterns, requiring no complex model calculations for identification, thus enabling rapid screening of low-risk information.

[0071] S2032: If the risk level is the second risk level, a pre-set detection model is used to perform risk detection on the information to be processed, and the risk detection result is obtained, wherein the second risk level is higher than the first risk level.

[0072] In this step, the second risk level is medium risk, where the main challenge is to find a balance between efficiency and accuracy. The pre-built detection model is a lightweight malicious detection model (such as a lightweight BERT model for text detection or a lightweight frequency domain analysis model for image detection). The model learns the feature differences between historical normal and malicious information, extracts deep features of the information to be processed, and compares them with the threshold obtained from model training to achieve the identification of attack features of moderate complexity.

[0073] S2033: If the risk level is the third risk level, the information to be processed is format-converted to obtain the reverse-generated text. Based on the reverse-generated text and the information to be processed, the risk detection result is determined, where the third risk level is higher than the second risk level.

[0074] In this step, the third risk level is high-risk, requiring deeper detection to address advanced attacks (such as injection vulnerabilities and jailbreak attacks). At this point, the information to be processed undergoes at least two format conversions to obtain reverse-generated text. By comparing the original information to be processed with the reverse-generated text, the risk detection result can be obtained, thus determining whether the information to be processed is abnormal.

[0075] As can be seen from the description of the above embodiments, the embodiments of this disclosure achieve differentiated and accurate detection of information of different risk levels through a hierarchical defense mechanism of low-risk rule matching, medium-risk lightweight model analysis, and high-risk format conversion verification. It not only quickly screens low- and medium-risk attacks and effectively controls system computational overhead and latency through lightweight means, but also breaks through the limitations of traditional detection by multimodal cross-validation to accurately identify advanced malicious attacks such as prompt injection. Ultimately, it achieves the defense goal of "balancing efficiency and accuracy and covering all risks", which significantly improves the attack resistance capability and scenario adaptability of the entire input detection layer.

[0076] In one possible implementation, step S2033 above performs format conversion on the information to be processed to obtain the reverse-generated text, including:

[0077] S20331: Convert the information to be processed into an image format to obtain the image to be converted.

[0078] In this step, a preset text-to-image model can be invoked to convert the semantic content of the information to be processed (mainly text information) into a corresponding visual image.

[0079] S20332: Convert the image to be converted into text to obtain the reverse generated text.

[0080] In this step, an image-to-text model (such as the BLIP model) can be called to extract visual features and interpret semantics of the image to be converted, and the visual information in the image can be converted back into text to obtain the reverse-generated text.

[0081] As can be seen from the description of the above embodiments, the present disclosure embodiments, through the conversion process of text to image and then back to text, not only fully preserve the core semantics of normal information, but also amplify the semantic deviation of malicious attack instructions. The generated reverse-generated text provides a basis for subsequent semantic similarity calculation and provides a foundation for the accurate detection of advanced malicious attacks.

[0082] In one possible implementation, step S2033 above determines the risk detection result based on the reverse-generated text and the information to be processed, including:

[0083] S20333: Determine the similarity between the information to be processed and the reverse-generated text.

[0084] In this step, a semantic similarity calculation model is used to determine the similarity between the information to be processed and the reverse-generated text, or the cosine similarity between the information to be processed and the reverse-generated text is calculated.

[0085] S20334: If the similarity is less than the similarity threshold, the abnormal input will be identified as a risk detection result; otherwise, the normal input will be identified as a risk detection result.

[0086] In this step, the similarity threshold can be preset by staff based on experimental data or empirical parameters. If the similarity is below the threshold, it indicates that the core semantics of the information to be processed have been severely lost in the cross-modal conversion, and it is likely malicious information (its malicious semantics cannot be maintained consistently during the conversion); if the similarity is above the threshold, it indicates that the core semantics are stably preserved, and it is normal information.

[0087] Among them, similarity threshold

[0088] As can be seen from the description of the above embodiments, the embodiments of this disclosure utilize the characteristic of poor consistency after abnormal input conversion, and determine the risk detection result by obtaining the similarity between the information to be processed and the reverse-generated text.

[0089] In one possible implementation, before step S205 above, where the risk detection result is normal, and the response generation model is used to post-process the information to be processed to obtain the response information, the following steps are also included:

[0090] S220: Train the teacher model using training data, which includes adversarial data.

[0091] In this step, adversarial data can be data with added anomalous and adversarial samples, used to enable the teacher model to learn the characteristics of adversarial attacks during training, forming the ability to ignore malicious inputs (i.e., actively suppressing the attention distribution of perturbation regions), and increasing the robustness of the teacher model.

[0092] S221: Extract training samples from the training data.

[0093] In this step, samples can be randomly drawn from the training data to obtain training samples. These training samples can include normal samples and various abnormal samples.

[0094] S222: Input the training samples into the teacher model and the student model to obtain the first output value of the teacher model and the second output value of the student model.

[0095] In this step, the same training sample is simultaneously input into the teacher model and the student model to be optimized, and the output results of the two are obtained. The first output value is a robust reference standard, and the second output value is the current performance of the student model. The difference between the two serves as the basis for subsequent loss calculation and model optimization.

[0096] S223: Calculate the loss value based on the first output value and the second output value.

[0097] This step may include calculating the standard cross-entropy loss, attention alignment loss, and adversarial confidence smoothing loss, and then weighting and summing these three loss values ​​to obtain a comprehensive loss value.

[0098] The standard cross-entropy loss is calculated using the following formula:

[0099]

[0100] In the formula, Let represent the standard cross-entropy loss, where i represents the category of the i-th piece of information to be processed, and C represents the total number of categories of information to be processed. This represents the first output value of the training sample. This represents the second output value of the student model for the input sample x in the i-th class.

[0101] The formula for calculating attention alignment loss is as follows:

[0102]

[0103] In the formula, This indicates attention alignment loss. This represents the attention matrix of the l-th layer of the teacher model. This represents the attention matrix of the l-th layer of the student model, where L is the total number of layers in the model. This represents the Frobenius norm.

[0104] The formula for calculating the adversarial confidence smoothing loss is as follows:

[0105]

[0106] In the formula, This represents the adversarial confidence smoothing loss, JS is the Jensen-Shannon divergence, and u is a uniform distribution. This represents the output probability distribution of the student model for the input sample x.

[0107] S224: If the loss value is greater than the preset loss value threshold, the student model is optimized using the loss value, and the step of extracting training samples to calculate the loss value is executed again until the loss value is less than the loss value threshold, and the student model is determined as the response generation model.

[0108] In this step, the loss threshold can be preset by the staff based on experiments or experience. The step of extracting training samples can be step S221 as described above, and the step of calculating the loss value can be step S223 as described above.

[0109] As can be seen from the description of the above embodiments, the embodiments of this disclosure transfer the strong robustness of the teacher model trained with adversarial data to the lightweight student model through the defensive knowledge distillation teacher-student model training process, and finally obtain a response generation model that has both high anti-interference capability and lightweight characteristics. This ensures the ability to resist malicious input in the subsequent response generation process and meets the efficiency requirements in actual deployment, providing reliable model support for the normal service link of the entire defense system.

[0110] In one possible implementation, in step S203 above, if the risk level is the third risk level, then multi-model joint reasoning is used to obtain the risk value, and the risk detection result is determined based on the risk value.

[0111] The risk values ​​obtained through multi-model joint inference include:

[0112] If the information to be processed is text, then determine the semantic embedding of the information to be processed and calculate the degree of deviation between the semantic embedding and the embedding space of the information to be processed.

[0113] The BERT embedding of the information x to be processed is The clean text embedding space is M. The method for calculating the degree of deviation is as follows:

[0114]

[0115] in, The degree of deviation is indicated by h, which represents the BERT embedding vector of the information to be processed. It is the projection operator of the embedded space M (which can be implemented by PCA or autoencoder).

[0116] τ is the dynamic threshold, calculated using the following formula:

[0117]

[0118] If the score exceeds the dynamic threshold, the risk detection result is determined to be an abnormal input, triggering an interception.

[0119] If the information to be processed is an image, frequency domain analysis is used. High-frequency components are extracted using Discrete Cosine Transform (DCT), and anomalous energy distributions are detected (anti-perturbations typically exhibit recognizable patterns in the frequency domain). The input image x has the following DCT coefficients:

[0120]

[0121] Where, when u=0, α represents the normalization coefficient, used to normalize the discrete cosine transform result; i and j represent the spatial coordinates of image pixels, corresponding to the row and column positions of the input image in the spatial domain; and H and W represent the height and width of the image, respectively. ,otherwise u and v represent frequency domain coordinates, corresponding to the indices of different frequency components after discrete cosine transform, characterizing the frequency features of the image (e.g., the larger the u and v values, the higher the corresponding high-frequency components).

[0122] The anti-perturbation function is designed as follows.

[0123]

[0124] Where I[·] is an indicator function, returning 1 when the internal condition is true, and 0 otherwise. ∑(u,v∈H) is the summation of frequency domain coordinates u and v over the high-frequency region H. This perturbation function outputs a binary result: 1 indicates an anomaly, and 0 indicates normal operation. If the output is 1, interception is triggered (sent to the audit terminal).

[0125] Figure 3 This is a schematic diagram of the information hierarchical processing device provided in an embodiment of this application. Figure 3As shown, the information classification processing device 300 includes: an information receiving module 301, a classification determination module 302, a risk detection module 303, a prompt output module 304, and a response generation module 305.

[0126] The information receiving module 301 is used to receive unprocessed information sent by the terminal device.

[0127] The risk level determination module 302 is used to determine the risk level corresponding to the information to be processed.

[0128] The risk detection module 303 is used to perform risk detection on the information to be processed using the risk detection method corresponding to the risk level, and obtain the risk detection result.

[0129] The prompt output module 304 is used to send a security prompt message to the audit terminal if the risk detection result is an input abnormality, so that the audit terminal outputs the security prompt message.

[0130] The response generation module 305 is used to post-process the information to be processed using the response generation model if the risk detection result is normal, and obtain response information. The response information is then sent to the terminal device so that the terminal device can output response information.

[0131] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0132] In one possible implementation, the level determination module 302 is used to determine the information entropy and similar attack frequency of the information to be processed; to obtain a risk value by weighted summation of the information entropy and similar attack frequency; and to find the correspondence between the preset risk value range and the risk level based on the risk value to obtain the risk level corresponding to the information to be processed.

[0133] In one possible implementation, the level determination module 302 is used to obtain the occurrence probability of characters in the information to be processed; and to determine the information entropy corresponding to the information to be processed based on the occurrence probability.

[0134] In one possible implementation, the risk detection module 303 is used to perform risk detection on the information to be processed using preset logical rules if the risk level is the first risk level, and obtain a risk detection result; if the risk level is the second risk level, it uses a preset detection model to perform risk detection on the information to be processed, and obtain a risk detection result, wherein the second risk level is higher than the first risk level; if the risk level is the third risk level, it performs format conversion on the information to be processed to obtain reverse generated text; and determines the risk detection result based on the reverse generated text and the information to be processed, wherein the third risk level is higher than the second risk level.

[0135] In one possible implementation, the risk detection module 303 is used to convert the information to be processed into an image format to obtain an image to be converted; and to convert the image to be converted into text to obtain reverse-generated text.

[0136] In one possible implementation, the risk detection module 303 is used to determine the similarity between the information to be processed and the reverse-generated text; if the similarity is less than the similarity threshold, the input is identified as abnormal and the risk detection result is determined; otherwise, the input is identified as normal and the risk detection result is determined.

[0137] In one possible implementation, the information classification processing device 300 further includes a model training module 306. The model training module 306 is used to train a teacher model using training data, including adversarial data; extract training samples from the training data; input the training samples into the teacher model and the student model to obtain a first output value from the teacher model and a second output value from the student model; calculate a loss value based on the first and second output values; if the loss value is greater than a preset loss value threshold, optimize the student model using the loss value, and repeat the steps of extracting training samples and calculating the loss value until the loss value is less than the loss value threshold, at which point the student model is determined as the response generation model.

[0138] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0139] To implement the above embodiments, this application also provides an electronic device.

[0140] refer to Figure 4 The diagram illustrates a structural schematic of an electronic device 400 suitable for implementing embodiments of this application. The electronic device 400 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0141] like Figure 4As shown, the electronic device 400 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 401 and a memory 402 communicatively connected to the processor. The processor can perform various appropriate actions and processes based on programs stored in the memory 402, computer-executed instructions, or programs loaded from storage device 408 into random access memory (RAM) 403, implementing the information hierarchical processing method in any of the above embodiments. The memory may be a read-only memory (ROM). The RAM 403 also stores various programs and data required for the operation of the electronic device 400. The processing device 401, memory 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0142] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0143] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 409, or installed from storage device 408, or installed from memory 402. When the computer program is executed by processing device 401, it performs the functions defined in the methods of embodiments of this application.

[0144] It should be noted that the computer-readable storage medium described above in this application can be a computer-readable signal medium, a computer storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0145] The aforementioned computer-readable storage medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0146] The computer-readable storage medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method of the above embodiments.

[0147] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0148] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0149] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the units do not necessarily limit the module itself.

[0150] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0151] This application also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the technical solution of the information hierarchical processing method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the information hierarchical processing method, and can be found in the implementation principle and beneficial effects of the information hierarchical processing method, which will not be repeated here.

[0152] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0153] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the technical solution of the information hierarchical processing method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the information hierarchical processing method, and can be found in the implementation principle and beneficial effects of the information hierarchical processing method, which will not be repeated here.

[0154] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

[0155] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0156] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. An information hierarchical processing method, characterized in that, include: Receive pending information sent by terminal devices; Determine the risk level corresponding to the information to be processed; The information to be processed is subjected to risk detection using the risk detection method corresponding to the risk level, and the risk detection result is obtained. If the risk detection result indicates an input anomaly, a security alert message is sent to the auditing terminal so that the auditing terminal outputs the security alert message. If the risk detection result is normal, the response generation model is used to post-process the information to be processed to obtain response information. The response information is sent to the terminal device so that the terminal device outputs the response information.

2. The method according to claim 1, characterized in that, Determining the risk level corresponding to the information to be processed includes: Determine the information entropy and similar attack frequency of the information to be processed; The risk value is obtained by weighted summation of the information entropy and the frequency of similar attacks. Based on the risk value, the correspondence between the preset risk value range and the risk level is found to obtain the risk level corresponding to the information to be processed.

3. The method according to claim 2, characterized in that, Determining the information entropy and similar attack frequency of the information to be processed includes: Obtain the probability of occurrence of characters in the information to be processed; The information entropy corresponding to the information to be processed is determined based on the occurrence probability.

4. The method according to claim 1, characterized in that, The step of performing risk detection on the information to be processed using the risk detection method corresponding to the risk level, and obtaining the risk detection result, includes: If the risk level is the first risk level, then the information to be processed is subjected to risk detection using preset logic rules to obtain the risk detection result; If the risk level is the second risk level, then a preset detection model is used to perform risk detection on the information to be processed to obtain the risk detection result, wherein the second risk level is higher than the first risk level; If the risk level is the third risk level, the format of the information to be processed is converted to obtain reverse generated text; based on the reverse generated text and the information to be processed, the risk detection result is determined, wherein the third risk level is higher than the second risk level.

5. The method according to claim 4, characterized in that, The process of converting the format of the information to be processed to obtain the reverse-generated text includes: The information to be processed is converted into an image format to obtain the image to be converted; The image to be converted is converted into text to obtain the reverse-generated text.

6. The method according to claim 4, characterized in that, The step of determining the risk detection result based on the reverse-generated text and the information to be processed includes: Determine the similarity between the information to be processed and the reverse-generated text; If the similarity is less than the similarity threshold, then the input is determined to be abnormal as the risk detection result; otherwise, the input is determined to be normal as the risk detection result.

7. The method according to any one of claims 1 to 6, characterized in that, Before the step of using a response generation model to post-process the information to be processed to obtain response information if the risk detection result is normal, the method further includes: The teacher model is trained using training data, including adversarial data. Extract training samples from the training data; The training samples are input into the teacher model and the student model to obtain the first output value of the teacher model and the second output value of the student model. Calculate the loss value based on the first output value and the second output value; If the loss value is greater than the preset loss value threshold, the student model is optimized using the loss value, and the step of extracting training samples to calculate the loss value is performed again until the loss value is less than the loss value threshold, and the student model is determined as the response generation model.

8. An information hierarchical processing device, characterized in that, include: The information receiving module is used to receive unprocessed information sent by the terminal device; The risk level determination module is used to determine the risk level corresponding to the information to be processed; The risk detection module is used to perform risk detection on the information to be processed using the risk detection method corresponding to the risk level, and obtain the risk detection result; The prompt output module is used to send a security prompt message to the audit terminal if the risk detection result is an input abnormality, so that the audit terminal outputs the security prompt message; The response generation module is used to post-process the information to be processed using the response generation model to obtain response information if the risk detection result is normal. The response information is sent to the terminal device so that the terminal device outputs the response information.

9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 7.

Images