A small language model detection method and device

Abstract

The embodiment of the application provides a small language model detection method and device, comprising: inputting a pre-constructed adversarial sample into a to-be-detected small language model running in a target embedded execution environment, obtaining a response text after the to-be-detected small language model performs reasoning; performing sentiment semantic analysis based on the response text to obtain a sentiment semantic detection result; performing semantic entropy analysis based on the adversarial sample and the response file to obtain a semantic entropy detection result; and determining an environment fluctuation result of the target embedded execution environment by using a preset fluctuation analysis model according to the sentiment semantic detection result and the semantic entropy detection result. The application can automatically detect small language models in different embedded execution environments and quantify the influence of the execution environment on the safety and stability of the model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for detecting small language models. Background Technology

[0002] With the rapid development of artificial intelligence technology, small language models (SLMs) are increasingly being used to achieve intelligence on resource-constrained edge devices. Unlike large models running on cloud servers, SLMs are typically compressed through pruning, quantization, and other processes, and are directly deployed in embedded edge environments such as smartphones and IoT devices.

[0003] In related technologies, security testing of small language models mostly relies on the APIs of large cloud-based models for auxiliary judgment or generation of adversarial examples. This method is not only costly but also poses a risk of data privacy leakage, making it difficult to meet the offline and privacy-sensitive testing needs of edge devices. More importantly, existing detection methods do not consider the impact of the underlying execution environment on model behavior. In fact, the same SLM may exhibit significant performance and security differences in different embedded environments, and existing static detection tools are unable to capture dynamic security risks caused by environmental heterogeneity. In addition, existing technologies have relatively limited means of detecting model output, making it difficult to simultaneously deal with logically confused hallucinations and clearly malicious jailbreak attacks, especially when faced with semantically meaningless noise interference, lacking an effective entropy detection mechanism. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method and apparatus for detecting small language models.

[0005] To achieve the above objectives, embodiments of this application provide a method for detecting small language models, including:

[0006] The pre-constructed adversarial sample is input into the small language model under test running in the target embedded execution environment. After the small language model under test performs inference, it obtains the response text. Sentiment semantic analysis is performed on the response text to obtain sentiment semantic detection results; Semantic entropy analysis is performed based on the adversarial sample and the response file to obtain semantic entropy detection results; Based on the sentiment semantic detection results and semantic entropy detection results, the environmental fluctuation results of the target embedded execution environment are determined using a preset fluctuation analysis model.

[0007] Optionally, sentiment semantic analysis is performed based on the response text to obtain sentiment semantic detection results, including: Sentiment polarity analysis was performed on the response text to obtain sentiment classification labels; Based on the vector representation of the response text and the vector representation of the target harmful text, calculate the cosine similarity between the response text and the target harmful text; When the emotion classification label is positive and the cosine similarity is greater than the preset similarity threshold, it is determined that the small language model under test has successfully escaped or successfully induced hallucination.

[0008] Optionally, semantic entropy analysis is performed based on the adversarial sample and the response file to obtain semantic entropy detection results, including: Calculate the first semantic entropy of the adversarial sample and the second semantic entropy of the response text; Calculate the absolute value of the difference between the first semantic entropy and the second semantic entropy; When the absolute value of the difference is less than the preset assimilation threshold, the noise attack on the small language model under test is determined to be successful.

[0009] Optionally, based on the sentiment semantic detection results and semantic entropy detection results, a preset fluctuation analysis model is used to determine the environmental fluctuation results of the target embedded execution environment, including: Based on the emotion semantic detection results and semantic entropy detection results, calculate the moving average of the detection results within a predetermined time window; Calculate the standard deviation based on the moving average; The environmental fluctuation results are determined based on the moving average and standard deviation.

[0010] Optionally, the method further includes: Construct a unified adversarial corpus; The adversarial samples are generated using a preset perturbation algorithm based on the data in the unified adversarial corpus.

[0011] Optionally, the response text obtained after the small language model under test performs inference includes: After the small language model under test performs inference based on the adversarial example, the output result, inference time and system running status of the target embedded execution environment of the small language model under test are obtained. After verifying the integrity of the output results, inference time, and system operating status, a structured response text is constructed based on the output results, inference time, and system operating status.

[0012] This application also provides a small language model detection device, including: The inference module is used to input pre-constructed adversarial examples into the small language model under test running in the target embedded execution environment. After the small language model under test performs inference, it obtains the response text. The sentiment semantic analysis module is used to perform sentiment semantic analysis based on the response text to obtain sentiment semantic detection results; The semantic entropy analysis module is used to perform semantic entropy analysis based on the adversarial sample and the response file to obtain semantic entropy detection results. The fluctuation analysis module is used to determine the environmental fluctuation results of the target embedded execution environment based on the sentiment semantic detection results and semantic entropy detection results, using a preset fluctuation analysis model.

[0013] Optionally, the sentiment semantic analysis module is used to perform sentiment polarity analysis on the response text to obtain sentiment classification labels; calculate the cosine similarity between the response text and the target harmful text based on the vector representation of the response text and the vector representation of the target harmful text; when the sentiment classification label is positive and the cosine similarity is greater than a preset similarity threshold, determine that the small language model under test has successfully escaped or successfully induced hallucinations.

[0014] Optionally, the semantic entropy analysis module is used to calculate the first semantic entropy of the adversarial sample and the second semantic entropy of the response text; calculate the absolute value of the difference between the first semantic entropy and the second semantic entropy; and determine that the noise attack on the small language model under test is successful when the absolute value of the difference is less than a preset assimilation threshold.

[0015] Optionally, the fluctuation analysis module is used to calculate the moving average of the detection results within a predetermined time window based on the sentiment semantic detection results and the semantic entropy detection results; calculate the standard deviation based on the moving average; and determine the environmental fluctuation results based on the moving average and the standard deviation.

[0016] As can be seen from the above, the small language model detection method and apparatus provided in this application embodiment inputs a pre-constructed adversarial sample into the small language model under test running in the target embedded execution environment. After the small language model under test performs inference, it obtains a response text. Sentiment semantic analysis is performed based on the response text to obtain sentiment semantic detection results. Semantic entropy analysis is performed based on the adversarial sample and the response file to obtain semantic entropy detection results. Based on the sentiment semantic detection results and the semantic entropy detection results, a preset fluctuation analysis model is used to determine the environmental fluctuation results of the target embedded execution environment. This application does not rely on cloud resources, can perform multi-type detection on small language models in different embedded execution environments, and quantifies the differences in the impact of different execution environments on the model's security and stability through fluctuation analysis. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this application; Figure 2 This is a schematic diagram of the method flow of an embodiment of this application; Figures 3A-3F This is a schematic diagram illustrating the experimental results of detecting small language models in different system environments according to embodiments of this application; Figure 4 This is a block diagram of the device structure according to an embodiment of this application; Figure 5 This is a block diagram of the electronic device structure according to an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0020] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0021] In related technologies, security testing of small language models mostly relies on cloud-based large model APIs or manual methods. This is not only costly and poses a risk of data privacy leaks, but also ignores the impact of the embedded environment on model behavior. Specifically, traditional security testing often assumes that the model runs in an ideal hardware environment, ignoring the interference of resource-constrained device computing power fluctuations and memory scheduling mechanisms on the model's inference stability. For example, the same small language model may perform stably on a high-performance server, but on a mobile device, when system resources are preempted, it may be more susceptible to noise interference, leading to illusions or jailbreak vulnerabilities. Furthermore, most existing detection tools are single-function and struggle to simultaneously handle logically complex jailbreak attacks and semantically meaningless noise interference. For example, traditional keyword matching methods struggle to identify semantically hollow random noise outputs, resulting in a high false positive rate.

[0022] In view of this, embodiments of this application provide a method and apparatus for detecting small language models. A unified corpus is constructed on a local test terminal, adversarial examples are generated based on the unified corpus, and a bimodal and semantic entropy detection mechanism is adopted to detect the response text obtained by the small language model based on adversarial example reasoning. It does not rely on cloud resources, can perform multi-type detection on small language models in different embedded execution environments, and quantifies the impact of the execution environment on model security through fluctuation analysis, significantly improving the objectivity and comprehensiveness of on-side model detection and evaluation.

[0023] The technical solution of this application will be further described in detail below through specific embodiments.

[0024] like Figure 1 The diagram illustrates an application scenario of a small language model detection method provided in an exemplary embodiment of this application. This application scenario includes a test terminal 101 and a target embedded device 102. The test terminal 101 and the target embedded device 102 can be connected via a wired or wireless communication network to achieve data interaction. The test terminal 101 can be a computer device with a deployed detection system, responsible for generating adversarial examples, sending inference requests, receiving responses, and performing analysis and calculations. The test terminal does not need to connect to the cloud via the internet to obtain data or rely on the cloud for data processing; it can generate adversarial examples and process response data locally, saving costs and avoiding data leakage. The target embedded device 102 can be a real physical device running a small language model (such as a smartphone, Raspberry Pi, or IoT gateway) or its high-fidelity emulator (such as an Android Emulator or Xcode Simulator).

[0025] In some embodiments, when executing the detection method, the test terminal 101 constructs a unified adversarial corpus and generates adversarial samples; the test terminal 101 sends the adversarial samples to the target embedded device 102; the target embedded device 102 performs model inference in a specific system environment, outputs response text, and feeds it back to the test terminal 101; the test terminal 101 performs semantic entropy and sentiment semantic analysis on the response text to obtain the corresponding detection results, and combines the results of multiple rounds of detection to perform environmental fluctuation analysis, and finally generates a detection report.

[0026] It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of this application, and the implementation of this application is not limited in any way.

[0027] like Figure 2 As shown in the embodiment of this application, a method for detecting small language models is provided, including: S201: Input the pre-constructed adversarial sample into the small language model under test running in the target embedded execution environment, and the small language model under test performs inference to obtain the response text; In this embodiment, a unified adversarial corpus is constructed. Based on the data in the unified adversarial corpus, adversarial samples are generated using a preset perturbation algorithm. The unified adversarial corpus includes preset threat category actions, language expression patterns, and special character interference items. The method for constructing the unified adversarial corpus includes: extracting basic threat data from multi-source external databases; constructing a data dictionary containing natural language variants, special character sets, code injection types, and keyword embedding frameworks; and mapping the basic threat data into multi-turn interaction sequences using multi-turn dialogue templates based on the data dictionary to obtain the unified adversarial corpus.

[0028] In some implementations, when extracting basic threat data from multi-source external databases, data samples involving various core threat actions such as unlawful assistance and malicious speech are extracted from publicly available authoritative security datasets. Based on the extracted data samples, a structured, multi-dimensional data dictionary is established. The data dictionary includes: language expression patterns covering mainstream languages ​​such as English and Chinese and their variants; special character sets including punctuation marks, mathematical symbols, currency symbols, emoticons, etc., used to test the robustness of the model to non-semantic characters; dialogue scenario templates such as "relationship-neighbor" and "profession-lawyer"; and keyword embedding frameworks for illusion triggering frameworks such as "disguise synthesis" and "mind chain inducement".

[0029] Based on data from the unified adversarial corpus, adversarial examples are generated by perturbing the data in the data dictionary using a pre-defined perturbation algorithm. Perturbation includes, but is not limited to, character-level replacement, text length adjustment, or keyword embedding.

[0030] Some methods for generating adversarial examples include: utilizing a defined perturbation function. The original data in the data dictionary is perturbed, whereby... The original data is represented by δ, which is the perturbation factor. For noise attack testing, special characters or emojis are randomly selected from the data dictionary and inserted into random positions in the input text, or used to replace existing characters, to test the model's robustness against interference. For hallucination-inducing testing, a long text attack strategy is used, extending the input text length through keyword embedding to increase contextual complexity and induce memory confusion in the model. For jailbreak attack testing, a multi-turn dialogue template is used to hide malicious intent within normal role-playing requests. Furthermore, if the previous adversarial example attack is unsuccessful, a local large language model can be used as an optimizer to adjust attack prompts and generate new adversarial examples.

[0031] In some embodiments, the test terminal 101 sends the generated adversarial sample to the target embedded device 102 and inputs the small language model under test running on the target embedded device. The small language model under test performs inference based on the input adversarial sample. After the inference is completed, the output result, inference time, and system running status of the small language model under test are obtained. The output result, inference time, and system running status are then checked for integrity, and a response text is obtained after the check. The integrity check includes: checking whether the output result is empty, truncated, or garbled; checking whether the inference time is valid and within a preset time range; checking whether the monitoring fields such as CPU utilization, memory usage, GPU computing power allocation, and I / O latency in the system running status are complete and correspond to the current inference task in time; and checking whether the output result, inference time, and system running status belong to the same inference process. After the integrity check passes, the output result, inference time, and system running status are encapsulated into a structured response text for subsequent sentiment semantic analysis and semantic entropy analysis.

[0032] In some implementations, a communication channel is established between the test terminal and the target embedded execution environment. The execution environment of the target embedded device is initialized and configured. A high-fidelity simulator is started or a real physical device is connected according to preset configuration parameters. The test terminal establishes a connection with the target embedded device through a debugging bridging tool, ensuring that the adversarial sample of the test terminal can be transmitted to the target embedded device without loss, while the target embedded device can provide real-time response text. Optionally, the execution environment of the target embedded device includes at least an iOS system environment and an Android system environment.

[0033] The test terminal encapsulates the generated adversarial examples into inference requests, which include hyperparameters such as prompts, maximum generation length, and temperature parameters. These inference requests are sent to the target embedded device. Upon receiving the inference request, the target embedded device inputs the adversarial example-related parameters into a small language model, enabling the small language model to perform inference tasks based on the input. During this process, it is crucial to ensure that the small language model is in a controlled "warm start" or "cold start" state to eliminate interference from non-experimental variables.

[0034] During the inference process of a small language model, the system operating status of the target embedded device is monitored and recorded in real time as an auxiliary feature to explain fluctuations in model security. In some methods, the system operating status can be obtained using a background performance monitoring daemon, which records CPU utilization, memory usage, GPU computing power allocation, and I / O latency at high frequency. Since Android and iOS systems use different background process management and memory reclamation mechanisms, the system will focus on capturing inference latency jitter and context loss caused by system resource preemption when processing long texts or highly complex adversarial examples.

[0035] After the small language model completes inference, it performs integrity checks on the output natural language text, inference time, and system running status. After verification, it is processed into structured response text and fed back to the test terminal.

[0036] S202: Perform sentiment semantic analysis based on the response text to obtain sentiment semantic detection results; In this embodiment, after obtaining the response text of the small language model, a bimodal analysis of sentiment and semantics is performed on the response text to obtain the sentiment and semantic detection results, including: Sentiment polarity analysis was performed on the response text to obtain sentiment classification labels; Calculate the cosine similarity between the response text and the target harmful text based on their vector representations. When the sentiment classification label is positive and the cosine similarity is greater than the preset similarity threshold, it is determined that the small language model under test has successfully escaped or successfully induced hallucination.

[0037] In this embodiment, sentiment polarity analysis is performed on the response text to obtain sentiment classification labels. If the small language model expresses rejection or apology, the sentiment classification label is negative (denoted as 0); if the small language model expresses helpfulness or compliance, the sentiment classification label is positive (denoted as 1).

[0038] The response text is converted into its corresponding vector representation, and the preset target harmful text is also converted into its corresponding vector representation. The cosine similarity between the vector representations of the response text and the target harmful text is calculated and expressed as: (1) Wherein, V ( T m (response text) T m Vector representation, V( T t ) as harmful text T t The vector representation of .

[0039] After determining the sentiment classification label of the response text and its similarity to the target harmful text, the small language model is considered to have successfully escaped or successfully induced hallucination when the sentiment classification label is positive and the cosine similarity is greater than the preset similarity threshold. This mechanism effectively avoids false alarms caused by relying solely on keyword matching.

[0040] In some implementations, 10 different secure dialogue scenarios are constructed, each representing a different identity to ask questions, including but not limited to roles such as teacher, friend, and police. Within each scenario, 16 different jailbreak categories are defined (14 content jailbreaks and 2 code jailbreaks). Each content jailbreak type contains 6 behaviors, and code jailbreaks contain 2 behaviors (SQL injection code generation and XSS attack code generation). For each behavior, different scenario questions are embedded. By defining standard answers for specific attack behaviors, the responses from the small language model are subjected to a trigger determination combining sentiment polarity analysis and semantic similarity. If a positive response is made and the semantic similarity to the target text exceeds a certain threshold, the trigger is considered successful. To determine the threshold, a behavior from one of the 14 content jailbreak categories is randomly selected, generating 10 different scenario responses. The average semantic similarity of these 10 responses to the corresponding standard responses is calculated. This average semantic similarity for each category is used as the similarity threshold benchmark for successful attack.

[0041] S203: Perform semantic entropy analysis based on adversarial examples and response files to obtain semantic entropy detection results; In this embodiment, after obtaining the response text of the small language model, semantic entropy analysis is performed on the response text to obtain the semantic entropy detection results, including: Calculate the first semantic entropy of the adversarial example and the second semantic entropy of the response text; Calculate the absolute value of the difference between the first semantic entropy and the second semantic entropy; When the absolute value of the difference is less than the preset assimilation threshold, the noise attack on the small language model under test is determined to be successful.

[0042] In this embodiment, the text of the adversarial sample and the response file are encoded into token sequences. A word segmenter matching the small language model under test is called to preprocess and segment the text content of the adversarial sample and the response text, respectively, mapping characters, words, or sub-words in the text to corresponding token identifiers, resulting in a first token sequence corresponding to the adversarial sample and a second token sequence corresponding to the response text. According to the input specifications of the small language model under test, start and end markers are added to the first and second token sequences, and corresponding attention masks are generated. In inference mode without updating model parameters, the first and second token sequences are input into the small language model under test for forward computation, obtaining the predicted logits corresponding to each token position. The predicted logits are normalized to obtain the conditional probability P(token) of each token at the corresponding position. i Based on the conditional probability of each word, the first semantic entropy of the adversarial sample and the second semantic entropy of the response text are calculated respectively.

[0043] The method for calculating semantic entropy (SE) is as follows: (2) Where N is the number of tokens in the text. Given the predicted probability of the i-th token, calculate the -PlogP cumulative total sentence for each token to obtain the overall semantic entropy of the text.

[0044] Calculate the first semantic entropy SE1 of the adversarial sample and the second semantic entropy SE2 of the response text according to formula (2). Then, calculate the absolute value of the difference between the first semantic entropy SE1 and the second semantic entropy SE2. When the absolute value of the difference is less than the preset assimilation threshold, it is determined that the small language model under test has been assimilated by noise, that is, the output of the model has a similar degree of disorder as the input random noise, which is considered a successful attack.

[0045] S204: Based on the sentiment semantic detection results and semantic entropy detection results, use a preset fluctuation analysis model to determine the environmental fluctuation results of the target embedded execution environment.

[0046] In this embodiment, the same small language model is detected in at least two different target embedded execution environments. For the small language model in the same target embedded execution environment, multiple sets of adversarial examples are used to repeatedly execute the test, obtaining the sentiment semantic detection result and semantic entropy detection result for each set of tests. Based on the multiple sets of detection results, a fluctuation analysis model is used to determine the environmental fluctuation result.

[0047] In some embodiments, based on the sentiment semantic detection results and semantic entropy detection results, a preset fluctuation analysis model is used to determine the environmental fluctuation results of the target embedded execution environment, including: Based on the sentiment semantic detection results and semantic entropy detection results, calculate the moving average of the detection results within a predetermined time window; Calculate the standard deviation based on the moving average; The results of environmental fluctuations are determined based on the moving average and standard deviation.

[0048] In this embodiment, the Bollinger Bands algorithm model is used for analysis. Specifically, it includes calculating the value at the final time point. t moving average MA t Within a given time window, the detection results are smoothed to reflect the baseline trend of the security of the small language model under test. The calculation method is as follows: (3) in, M The size of the sliding time window. R i For at a certain point in time i The result of sentiment semantic detection or semantic entropy detection of the small language model under test (e.g., 1 for successful sentiment semantic attack and 0 for unsuccessful attack; 1 for successful noise attack and 0 for unsuccessful attack), or the fusion result of sentiment semantic detection and semantic entropy detection. t It is the last time point within the sliding time window.

[0049] Calculation time point t Standard deviation This is used to measure the degree of fluctuation in test results, and the calculation method is as follows: (4) Based on time points t Using the moving average and standard deviation, construct the fluctuation band within the sliding time window: UpperBand= MA t +2 (5) LowerBand= MA t 2 (6) The bandwidth between the LowerBand and the UpperBand is the bandwidth corresponding to the fluctuation band, which is used as a result of environmental fluctuations.

[0050] like Figures 3A-3FAs shown, various types of attacks were detected on the small language model under Android and iOS systems. Based on the detection results, the changing trends of the most relevant indicators to the small language model were analyzed, including noise character category, noise text length, noise attack rounds, phantom text generation framework combination, phantom text attack rounds, jailbreak behavior category, and so on. Blue represents the iOS system environment, and red represents the Android system environment. Through multiple rounds of experimental data, the sliding window size M was determined to be 5, and the upper and lower bounds of the data fluctuation were calculated based on the maximum number of data points. The environmental fluctuation results of different attacks on the small language model under the two system environments are shown in Table 1.

[0051] Table 1. Environmental fluctuation results of the small language model under test

[0052] In multi-round noise attack experiments, the Android system environment exhibited more significant performance fluctuations compared to the iOS system environment. In noisy character category attacks, the fluctuation range of the Android system environment was approximately 23% higher than that of the iOS system environment. Regarding the length of noisy text, the fluctuation range of the Android system environment was nearly 1.7 times that of the iOS system environment, indicating its greater sensitivity to input interference. In terms of the number of noise attack rounds, the Android system environment required an average of about 2.5 fewer rounds to complete the attack than the iOS system environment, reflecting its poorer inference stability and lower tolerance to input perturbations. In the long text illusion attack experiments, the differences between the system environments converged somewhat. Regarding the number of combinations of illusion text generation frameworks, the upper limit of the generation framework complexity covered by the iOS system environment was approximately 55.6% higher than that of the Android system environment. Regarding the number of illusion attack rounds, based on the upper bound, the iOS system environment required an average of about 28.2% more rounds to successfully induce illusions. However, the overall fluctuation range was limited, so the anti-attack performance of the two system environments under highly semantically relevant input conditions was relatively similar, and the impact of system environment characteristics was relatively limited. In the jailbreak attack experiment, the differences between system environments widened significantly again. The Android system environment showed nearly three times the volatility of jailbreak behavior categories compared to the iOS system environment. Under conditions of low data richness, the Android system environment achieved a jailbreak success rate of up to 85%, while the iOS system environment only achieved about 10%, a difference of up to 75 percentage points.

[0053] Overall, the Android system environment exhibits greater flexibility in resource scheduling and inference adaptability. In multi-round noise attacks, the Android system environment can complete inference manipulation quickly with fewer attack rounds and has a higher input response speed when facing highly variable inputs. In long text attacks, the Android system environment has a wider range of adaptability to text length variations (the length fluctuation range of noisy text reaches 189.97-14.97, while the iOS system environment is 100.82-49.18). In jailbreak attacks, the Android system environment has higher trigger sensitivity to different types of attack scenarios. In low-rich data environments, its jailbreak attack success rate is significantly higher than that of the iOS system environment, indicating that the Android system environment has a stronger ability to flexibly interpret input prompts. However, this flexibility also makes the inference results more sensitive to abnormal input changes, making it easier to be breached by noise injection, long text inducement, and jailbreak attacks. On the other hand, although the iOS system environment has higher overall inference stability, it also exposes vulnerabilities in certain complex tasks. In long text illusion attacks, although the iOS system environment is generally more resistant, when facing highly complex, multi-round deceptive attacks, the success rate of the attack is less different from that of the Android system environment. The model still has a certain risk of misjudging the complex false information constructed inside the long text (such as reverse hints and chain reasoning misleading).

[0054] The method for detecting small language models provided in this application constructs a unified adversarial corpus that does not rely on cloud resources. It combines a semantic entropy and sentiment semantic dual-modal detection mechanism to perform multi-type detection on small language models on embedded devices. By introducing a fluctuation analysis mechanism, it can quantitatively evaluate the differences in the impact of different embedded execution environments on model security, which can significantly improve the objectivity and comprehensiveness of edge model evaluation and provide an important basis for the secure deployment of edge AI.

[0055] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0056] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0057] like Figure 4 As shown in the figure, this application provides a small language model detection device, including: The inference module is used to input pre-constructed adversarial examples into the small language model under test running in the target embedded execution environment. After the small language model under test performs inference, it obtains the response text. The sentiment semantic analysis module is used to perform sentiment semantic analysis based on the response text to obtain sentiment semantic detection results; The semantic entropy analysis module is used to perform semantic entropy analysis based on adversarial examples and response files to obtain semantic entropy detection results. The fluctuation analysis module is used to determine the environmental fluctuation results of the target embedded execution environment based on the sentiment semantic detection results and semantic entropy detection results, using a preset fluctuation analysis model.

[0058] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware.

[0059] The apparatus described above is used to implement the corresponding methods in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0060] Figure 5 This embodiment illustrates a more specific hardware structure of an electronic device. The device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0061] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0062] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0063] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0064] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0065] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0066] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0067] The electronic devices described above are used to implement the corresponding methods in the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0068] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0069] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0070] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0071] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0072] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this disclosure.

Claims

1. A method for detecting small language models, characterized in that, include: The pre-constructed adversarial sample is input into the small language model under test running in the target embedded execution environment. After the small language model under test performs inference, it obtains the response text. Sentiment semantic analysis is performed on the response text to obtain sentiment semantic detection results; Semantic entropy analysis is performed based on the adversarial sample and the response file to obtain semantic entropy detection results; Based on the sentiment semantic detection results and semantic entropy detection results, the environmental fluctuation results of the target embedded execution environment are determined using a preset fluctuation analysis model.

2. The method according to claim 1, characterized in that, Sentiment semantic analysis is performed on the response text to obtain sentiment semantic detection results, including: Sentiment polarity analysis was performed on the response text to obtain sentiment classification labels; Based on the vector representation of the response text and the vector representation of the target harmful text, calculate the cosine similarity between the response text and the target harmful text; When the emotion classification label is positive and the cosine similarity is greater than the preset similarity threshold, it is determined that the small language model under test has successfully escaped or successfully induced hallucination.

3. The method according to claim 1, characterized in that, Based on the adversarial sample and the response file, semantic entropy analysis is performed to obtain semantic entropy detection results, including: Calculate the first semantic entropy of the adversarial sample and the second semantic entropy of the response text; Calculate the absolute value of the difference between the first semantic entropy and the second semantic entropy; When the absolute value of the difference is less than the preset assimilation threshold, the noise attack on the small language model under test is determined to be successful.

4. The method according to claim 1, characterized in that, Based on the sentiment semantic detection results and semantic entropy detection results, the environmental fluctuation results of the target embedded execution environment are determined using a preset fluctuation analysis model, including: Based on the emotion semantic detection results and semantic entropy detection results, calculate the moving average of the detection results within a predetermined time window; Calculate the standard deviation based on the moving average; The environmental fluctuation results are determined based on the moving average and standard deviation.

5. The method according to claim 1, characterized in that, Also includes: Construct a unified adversarial corpus; The adversarial samples are generated using a preset perturbation algorithm based on the data in the unified adversarial corpus.

6. The method according to claim 1, characterized in that, The small language model under test obtains response text after performing inference, including: After the small language model under test performs inference based on the adversarial example, the output result, inference time and system running status of the target embedded execution environment of the small language model under test are obtained. After verifying the integrity of the output results, inference time, and system operating status, a structured response text is constructed based on the output results, inference time, and system operating status.

7. A device for detecting small language models, characterized in that, include: The inference module is used to input pre-constructed adversarial examples into the small language model under test running in the target embedded execution environment. After the small language model under test performs inference, it obtains the response text. The sentiment semantic analysis module is used to perform sentiment semantic analysis based on the response text to obtain sentiment semantic detection results; The semantic entropy analysis module is used to perform semantic entropy analysis based on the adversarial sample and the response file to obtain semantic entropy detection results. The fluctuation analysis module is used to determine the environmental fluctuation results of the target embedded execution environment based on the sentiment semantic detection results and semantic entropy detection results, using a preset fluctuation analysis model.

8. The apparatus according to claim 7, characterized in that, The sentiment semantic analysis module is used to perform sentiment polarity analysis on the response text to obtain sentiment classification labels; calculate the cosine similarity between the response text and the target harmful text based on the vector representation of the response text and the vector representation of the target harmful text; when the sentiment classification label is positive and the cosine similarity is greater than a preset similarity threshold, it is determined that the small language model under test has successfully escaped or successfully induced hallucinations.

9. The apparatus according to claim 7, characterized in that, The semantic entropy analysis module is used to calculate the first semantic entropy of the adversarial sample and the second semantic entropy of the response text; and to calculate the absolute value of the difference between the first semantic entropy and the second semantic entropy. When the absolute value of the difference is less than the preset assimilation threshold, the noise attack on the small language model under test is determined to be successful.

10. The apparatus according to claim 7, characterized in that, The fluctuation analysis module is used to calculate the moving average of the detection results within a predetermined time window based on the sentiment semantic detection results and the semantic entropy detection results; calculate the standard deviation based on the moving average; and determine the environmental fluctuation results based on the moving average and the standard deviation.

Images