Test data generation method and device, equipment and medium

Abstract

The invention provides a test data generation method and device, equipment and a medium, and relates to the technical field of artificial intelligence, in particular to the technical fields of machine learning, deep learning, large models and the like. The method comprises the following steps: inputting a target instruction feature into a neural network model, and obtaining an original activation feature output by a middle layer of the neural network model; shifting the original activation feature by using a preset semantic control feature to obtain a target activation feature; obtaining a target result output by a subsequent layer behind the middle layer in the neural network model based on the target activation feature; the to-be-optimized test features and the target instruction features are combined and then input into the neural network model, and controlled activation features output by the middle layer and controlled results output by the follow-up layer are obtained; and based on the target activation feature, the controlled activation feature, the target result and the controlled result, performing back propagation optimization on the test feature to obtain target test data.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to the fields of machine learning, deep learning and large models, and specifically to a test data generation method, a test data generation device, an electronic device, a computer-readable storage medium and a computer program product. Background Technology

[0002] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies mainly include natural language processing, computer vision, speech recognition, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0003] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0004] This disclosure provides a test data generation method, a test data generation apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of this disclosure, a method for generating test data for a neural network model is provided, comprising: inputting target instruction features into a neural network model and obtaining the original activation features output by the intermediate layers of the neural network model; offsetting the original activation features using preset semantic control features to obtain target activation features; obtaining the target result output by subsequent layers in the neural network model based on the target activation features; combining the test features to be optimized with the target instruction features and inputting them into the neural network model, and obtaining the controlled activation features output by the intermediate layers and the controlled result output by the subsequent layers; and performing backpropagation optimization on the test features based on the target activation features, the controlled activation features, the target result, and the controlled result to obtain target test data.

[0006] According to another aspect of this disclosure, a test data generation apparatus for a neural network model is provided, comprising: a first acquisition unit configured to input target instruction features into the neural network model and acquire the original activation features output by the intermediate layers of the neural network model; an offset unit configured to offset the original activation features using preset semantic control features to obtain target activation features; a second acquisition unit configured to acquire the target result output by subsequent layers in the neural network model based on the target activation features; a third acquisition unit configured to input the test features to be optimized and the target instruction features into the neural network model and acquire the controlled activation features output by the intermediate layers and the controlled result output by the subsequent layers; and an optimization unit configured to perform backpropagation optimization on the test features based on the target activation features, the controlled activation features, the target result, and the controlled result to obtain target test data.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the methods described above.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the above-described method.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the above-described method when executed by a processor.

[0010] According to one or more embodiments of this disclosure, by inputting target instruction features into a neural network model and using semantic control features to offset the original activation features output by the intermediate layer, adjusted target activation features and target results can be obtained. Furthermore, by combining the test features to be optimized and the target instruction features and inputting them into the neural network model, and using the target activation features and target results to constrain the controlled activation features and controlled results output by the intermediate layer, test data for the input end is finally obtained. This achieves control over the semantic representation within the model at the input end, facilitating model testing.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0013] Figure 1 A schematic diagram of an exemplary system in which the various methods described herein may be implemented according to embodiments of the present disclosure is shown;

[0014] Figure 2 A flowchart of a test data generation method according to an embodiment of the present disclosure is shown; Figure 3 A flowchart illustrating the operation of obtaining semantic control features according to an embodiment of the present disclosure is shown; Figure 4 A flowchart illustrating backpropagation optimization of test features according to an embodiment of the present disclosure is shown; Figure 5 A schematic diagram illustrating the optimization process of test features according to embodiments of the present disclosure is shown; Figure 6 A structural block diagram of a test data generation apparatus according to an embodiment of the present disclosure is shown; and Figure 7 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0015] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0016] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0017] The terminology used in the description of the various examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0018] In related technologies, existing methods have difficulty obtaining test data for neural network models, and the obtained test data is highly limited, making it inconvenient to test the models.

[0019] To address the aforementioned issues, this disclosure inputs the target instruction features into a neural network model and uses semantic control features to offset the original activation features output by the intermediate layers, thereby obtaining adjusted target activation features and target results. Furthermore, by combining the test features to be optimized and the target instruction features and inputting them into the neural network model, and using the target activation features and target results to constrain the controlled activation features and controlled results output by the intermediate layers, test data for the input is ultimately obtained. This achieves control over the semantic representation within the model at the input end, facilitating model testing.

[0020] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0021] Figure 1 A schematic diagram of an exemplary system 100 in which the various methods and apparatus described herein can be implemented according to embodiments of this disclosure is shown. Reference Figure 1 The system 100 includes one or more client devices 101, 102, 103, 104, 105 and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105 and 106 can be configured to execute one or more applications.

[0022] In embodiments of this disclosure, server 120 may run one or more services or software applications that enable the execution of the methods of this disclosure.

[0023] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as to users of client devices 101, 102, 103, 104, 105 and / or 106 under a Software as a Service (SaaS) model.

[0024] exist Figure 1In the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 can sequentially interact with server 120 using one or more client applications to utilize the services provided by these components. It should be understood that various different system configurations are possible and may differ from system 100. Therefore, Figure 1 This is an example of a system used to implement the various methods described herein, and is not intended to be limiting.

[0025] Users can use client devices 101, 102, 103, 104, 105, and / or 106 for human-computer interaction. The client devices provide interfaces that enable users to interact with them. The client devices can also output information to the user through these interfaces. Although... Figure 1 Only six client devices are described, but those skilled in the art will understand that this disclosure can support any number of client devices.

[0026] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices. These computer devices can run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. Client devices are capable of executing various applications, such as various internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and can use various communication protocols.

[0027] Network 110 can be any type of network well known to those skilled in the art, and can support data communication using any of a variety of available protocols (including but not limited to TCP / IP, SNA, IPX, etc.). By way of example only, one or more networks 110 can be a local area network (LAN), an Ethernet-based network, a token ring network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0028] Server 120 may include one or more general-purpose computers, special-purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0029] The computing unit in server 120 can run one or more operating systems, including any of the aforementioned operating systems and any commercially available server operating system. Server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0030] In some implementations, server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105 and / or 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105 and / or 106.

[0031] In some implementations, server 120 can be a server for a distributed system or a server integrated with blockchain. Server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0032] System 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. Databases 130 may reside in various locations. For example, a database used by server 120 may be local to server 120, or it may be located away from server 120 and may communicate with server 120 via a network-based or dedicated connection. Databases 130 may be of different types. In some embodiments, the database used by server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data from and from the databases in response to commands.

[0033] In some embodiments, one or more of the databases 130 may also be used by an application to store application data. The databases used by the application may be of different types, such as key-value stores, object stores, or regular stores supported by a file system.

[0034] Figure 1 The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0035] According to one aspect of this disclosure, a method for generating test data for neural network models is provided. For example... Figure 2 As shown, method 200 includes: step S201, inputting the target instruction features into a neural network model and obtaining the original activation features output by the intermediate layer of the neural network model; step S202, shifting the original activation features using preset semantic control features to obtain the target activation features; step S203, obtaining the target result output by the subsequent layers after the intermediate layer in the neural network model based on the target activation features; step S204, combining the test features to be optimized with the target instruction features and inputting them into the neural network model, and obtaining the controlled activation features output by the intermediate layer and the controlled result output by the subsequent layer; and step S205, performing backpropagation optimization on the test features based on the target activation features, controlled activation features, target result, and controlled result to obtain the target test data.

[0036] Therefore, by inputting the target instruction features into the neural network model and using semantic control features to offset the original activation features output by the intermediate layer, adjusted target activation features and target results can be obtained. Furthermore, by combining the test features to be optimized and the target instruction features and inputting them into the neural network model, and using the target activation features and target results to constrain the controlled activation features and controlled results output by the intermediate layer, test data for the input end is finally obtained. This achieves control over the semantic representation within the model at the input end, facilitating model testing.

[0037] In the embodiments of this disclosure, "feature" refers to a numerical representation that characterizes data attributes. In neural network models, features can typically exist in the form of vectors, matrices, or tensors to facilitate mathematical operations and backpropagation optimization.

[0038] In some embodiments, the neural network model has the ability to receive instruction features and output corresponding results. Observations have revealed that the activation features output by the intermediate layers of the neural network model contain specific directional components. These directional components provide a semantic control direction in the semantic space, thereby influencing the model's output behavior pattern. Enhancing or removing this directional component from the activation features alters the model's response to the same instruction. Furthermore, this directional component exhibits a degree of generalization, and its effect is often not limited to a single instruction feature.

[0039] Based on this, this disclosure utilizes preset semantic control features in a specific intermediate layer to simulate or cancel the directional component, thereby enabling intervention in the output behavior of the neural network model.

[0040] For example, suppose the pre-defined semantic control features represent the semantic direction of "negative emotion". When the model receives a neutral or negative input instruction, its original activation features may contain this "negative emotion" component, causing the model to output a negative or rejecting result. In this case, by using semantic control features to shift the original activation features in the intermediate layer, the negative influence in the internal representation can be eliminated, thereby guiding the model to output a target result with positive emotion or cooperation.

[0041] Furthermore, this disclosure uses test features as learnable variables and optimizes them through backpropagation, enabling them to simulate the feature shifting effect performed in the intermediate layer at the input of the model. This allows the model to generate outputs that conform to specific behavioral patterns based solely on the input data without internal intervention (i.e., without modifying model parameters or intervening in intermediate layer calculations).

[0042] Based on the above generation mechanism, the method provided in this disclosure can be applied to the study of robustness improvement and security of neural network models, specifically including the following aspects: In security assessment scenarios, this method can be integrated into red team testing processes. By generating target test data, it is possible to discover trigger texts that cause neural network models to change predetermined rejection policies or compliance behaviors, thereby quantifying and evaluating the security boundaries of the model.

[0043] In model diagnostic scenarios, by analyzing the hierarchical and directional role of test features in neural network models and their impact on output distribution, we can help locate sensitive directions or unstable factors within the neural network model, thereby providing data support for optimizing the model architecture.

[0044] In defense research scenarios, the generated test data can be used as adversarial examples to detect or enhance the robustness of neural network models during the training or evaluation phase, thereby improving the model's ability to defend against deceptive inputs.

[0045] In some embodiments, the specific intermediate layer described above may be predetermined among multiple layers of a neural network model through various means. For example, an intermediate layer that can intervene (or has a better interference effect) using semantic control features may be determined based on prior knowledge or through experimentation.

[0046] In one exemplary embodiment, preset semantic control features can be applied to different levels for offsetting, and the magnitude of change in the model output can be observed (e.g., calculating the KL divergence or semantic distance between the original result and the controlled result). The level that causes the largest change in the output or the smallest offset required to successfully achieve the behavior mode switching is selected as the specific intermediate layer.

[0047] In this disclosure, an "intermediate layer" can be a single network layer or a combination of multiple network layers, either consecutive or discontinuous. When multiple layers are involved, the aforementioned feature shift operation can be performed synchronously in each of those multiple layers.

[0048] According to some embodiments, the neural network model can be a large language model, the target instruction features can be obtained by embedding the target instruction text, the target result includes the target probability distribution on the vocabulary and the target text result generated based on the target probability distribution, and the controlled result includes the controlled probability distribution on the vocabulary and the controlled text result generated based on the controlled probability distribution.

[0049] In some embodiments, a Large Language Model (LLM) typically consists of an embedding layer and a multi-layer Transformer structure. The target instruction text is first segmented into a sequence of tokens by a token segmenter, and then input into the embedding layer to be transformed into a high-dimensional vector representation, i.e., the target instruction features mentioned above. The model output can include the model's predicted probability distribution for the next token in the vocabulary, and a sequence of natural language text generated based on this probability distribution. The method disclosed herein utilizes this end-to-end generative characteristic to test the stability and reliability of the model's output by optimizing the test features at the input end.

[0050] According to some embodiments, semantic control features can be used to switch the output response of a neural network model from a first behavior mode to a second behavior mode. Subsequent layers of the neural network model can be configured to output a raw result corresponding to the first behavior mode, based on the original activation features, which is different from the target result.

[0051] Therefore, by establishing a first behavioral pattern based on the original activation features as a benchmark and comparing it with a second behavioral pattern shifted based on semantic control features, a clear differential supervision signal can be provided for the optimization of test features. This differential constraint ensures that the generated test data can accurately guide the model to change behavioral patterns, thereby improving the accuracy and effectiveness of the generated test data.

[0052] In some embodiments, the first behavior pattern can be the denial response pattern of the neural network model under the default security policy, and the second behavior pattern refers to the direct response pattern of the neural network model after bypassing the security policy.

[0053] For example, when the input target instruction features involve a restricted domain predefined by the model, the neural network model in the first behavior mode typically outputs a rejection placeholder text (e.g., "Unable to respond to this request"). However, after using semantic control features to suppress the rejection direction of the intermediate layers, the neural network model switches to the second behavior mode, thereby outputting specific generated content for the restricted domain.

[0054] According to some embodiments, such as Figure 3 As shown, the semantic control features can be obtained through the following operations 300: Step S301, obtain a first sample set that can trigger a first behavior mode and a second sample set that can trigger a second behavior mode; Step S302, input the first sample set and the second sample set into the neural network model respectively, and obtain the first sample activation features output by the intermediate layer based on the first sample set and the second sample activation features output based on the second sample set; and Step S303, determine the semantic control features based on the first sample activation features and the second sample activation features.

[0055] Therefore, by collecting sample sets that trigger different behavioral patterns and extracting the corresponding activation features from the intermediate layer outputs, the key semantic directions that determine the differences in output behavior within the neural network model can be captured from a statistical perspective. This set-based feature extraction method utilizes the common features of a large number of samples, effectively eliminating the random noise interference specific to individual samples, thereby improving the accuracy and generalization ability of semantic control features and ensuring the effectiveness of subsequent feature shift operations.

[0056] In some embodiments, the first sample set refers to a set of instruction texts known to trigger a neural network model to generate a first behavioral pattern, and the second sample set refers to a set of instruction texts known to trigger a neural network model to generate a second behavioral pattern. The first sample activation features and the second sample activation features refer to the activation tensors or their statistical aggregation results generated by the neural network model in a specific intermediate layer when processing the corresponding sample sets.

[0057] For step S301, sample instructions can be obtained by collecting existing public datasets or through manual construction. In one implementation, semantically neutral, general instructions that normally trigger model responses can be selected as the first sample set, and instructions containing common restricted topics or rejection triggers can be selected as the second sample set. This paired or grouped sample construction method helps to highlight the key feature differences that determine the switching of behavioral patterns in comparison.

[0058] For step S302, the instruction texts in the first and second sample sets can be converted into embedded features and then input into the neural network model. In a predetermined specific intermediate layer, the activation features corresponding to each instruction can be recorded respectively.

[0059] For step S303, the semantic control feature can be determined by calculating the average difference. For example, the average of multiple first sample activation features corresponding to the first sample set can be calculated to obtain the first central feature, and the average of multiple second sample activation features corresponding to the second sample set can be calculated to obtain the second central feature. Subsequently, the difference between the first central feature and the second central feature is calculated, and this difference is determined as the semantic control feature. This semantic control feature represents the semantic space direction from the second behavioral pattern to the first behavioral pattern, for example, the semantic direction of a rejection response.

[0060] According to some embodiments, step S202, shifting the original activation features using preset semantic control features to obtain target activation features, may include: subtracting the semantic control features from the original activation features to obtain the target activation features.

[0061] Therefore, by performing a subtraction operation, a specific directional component represented by semantic control features can be selectively removed or suppressed from the original activation features. This linear feature space operation is computationally inexpensive and can precisely control the model's internal representation, thereby forcing subsequent layers to reason based on features with the directional component removed, thus achieving effective control over the model's output behavior.

[0062] Furthermore, this method enables the optimized test features to accurately learn how to suppress the directional component at the input, thus allowing them to switch the output response behavior of the neural network model simply by changing the input data without interfering with the model's internal calculations.

[0063] In some embodiments, the subtraction operation is typically performed at the tensor level. Specifically, the tensor of the original activation features is subtracted element-wise from the tensor of the semantic control features. Furthermore, to control the intensity of the intervention, a weighting coefficient can be introduced, which is the product of the weighting coefficient and the semantic control features subtracted from the original activation features.

[0064] In some embodiments, in step S203, the target activation features obtained in step S202 can be further processed using subsequent layers located after the intermediate layers in the neural network model to obtain the target result, that is, the final output of the neural network model. This result reflects the intervention of semantic control features on the output behavior of the neural network model.

[0065] According to some embodiments, step S204, which combines the test feature to be optimized with the target instruction feature and inputs it into the neural network model, and obtains the controlled activation feature output by the intermediate layer and the controlled result output by the subsequent layer, may include: concatenating the test feature to be optimized as a prefix with the target instruction feature, and inputting the concatenation result into the neural network model.

[0066] Therefore, this prefix concatenation method can indirectly control the model's understanding and processing of the target instruction by changing the context environment without modifying the target instruction itself, thus achieving non-intrusive guidance of the model's behavior at the input end.

[0067] In some embodiments, the concatenation operation is performed along the sequence length dimension. For example, the sequence length corresponding to the test feature to be optimized is M, the sequence length corresponding to the target instruction feature is N, and the length of the concatenated input feature along the sequence dimension is M+N. The test feature precedes the target instruction feature, forming the starting part of the combined input feature.

[0068] After inputting the above combined input sequence into the neural network model, conventional forward propagation computation is performed. When the computation reaches a specific intermediate layer, the tensor output by that layer is extracted, and the entire tensor or the portion corresponding to the target instruction feature is used as controlled activation features. As the computation continues to the final output layer of the model, the probability distribution (e.g., logits distribution), features, text sequences, or other results generated by the model are obtained as controlled results.

[0069] According to some embodiments, such as Figure 4 As shown, step S205, optimizing the test features through backpropagation based on the target activation features, controlled activation features, target results, and controlled results to obtain target test data, may include: step S401, determining a first loss value based on the target activation features and controlled activation features; step S402, determining a second loss value based on the target results and controlled results; step S403, combining the first loss value and the second loss value to obtain a comprehensive loss value; and step S404, optimizing the test features through backpropagation using the comprehensive loss value.

[0070] Therefore, by constructing a comprehensive loss value that includes intermediate layer feature constraints (first loss value) and output layer result constraints (second loss value), the test features can be jointly constrained from two dimensions: the semantic representation of the intermediate layer and the final output result.

[0071] Specifically, the first loss value characterizes the difference between the target activation feature and the controlled activation feature. It primarily drives the test feature to learn and master the semantic direction required to switch the neural network model's behavior patterns, enabling the optimized test feature to regulate the model's output response at the input end. The second loss value characterizes the difference between the target result and the controlled result. It primarily applies language model-level normalization constraints to the final generated content, ensuring that while introducing test features changes the model's internal state, the neural network model's output data maintains correct semantic coherence, logic, and usability. This effectively avoids problems such as garbled text, meaningless character sequences, or structural collapse caused by over-optimizing intermediate layer features. By combining the two loss values, the semantic alignment of the intermediate layers can be guaranteed while also considering the generation quality of the output layer, achieving end-to-end control over the model's behavior.

[0072] Furthermore, compared to using only a single-level loss function, this multi-level constraint approach can more accurately guide the update direction of test features, avoid getting trapped in local optima, and thus improve the convergence speed of backpropagation optimization and the effectiveness of the generated test data.

[0073] According to some embodiments, step S401, determining the first loss value based on the target activation feature and the controlled activation feature, may include: determining the first loss value based on the L2 distance or cosine similarity between the target activation feature and the controlled activation feature. The L2 distance focuses on the absolute difference in the numerical magnitude and direction of the feature vectors, while the cosine similarity focuses on the consistency of the feature vectors in the semantic space direction.

[0074] According to some embodiments, step S402, determining the second loss value based on the target result and the controlled result, may include: determining the second loss value based on the KL divergence between the target result and the controlled result. The target result and the controlled result may be probability distributions over a vocabulary. KL divergence can effectively measure the degree of difference between these two probability distributions.

[0075] In some embodiments, in step S403, a first weighting coefficient (e.g., λ1) and a second weighting coefficient (e.g., λ2) can be preset. The first loss value can be multiplied by the first weighting coefficient, and the second loss value can be multiplied by the second weighting coefficient. The two products are then added together to obtain the comprehensive loss value. By adjusting these two weighting coefficients, the relative importance of intermediate layer feature alignment and output layer result alignment in the optimization process can be flexibly adjusted.

[0076] In some embodiments, the test feature to be optimized can be implemented as a learnable embedding tensor independent of the neural network model parameters. Initially, it can be initialized by randomly sampling tokens from the vocabulary or using random noise.

[0077] In some embodiments, in step S404, the gradient of the comprehensive loss value relative to the test feature to be optimized can be calculated using the chain rule based on the comprehensive loss value. Subsequently, the value of the test feature is updated along the negative gradient direction using a gradient descent algorithm (e.g., SGD, Adam, etc.) or a projective gradient descent algorithm until a preset number of iterations is reached or the comprehensive loss value converges to a preset threshold. During this process, the weight parameters of the neural network model remain frozen.

[0078] Figure 5 A schematic diagram illustrating the optimization process of test features according to exemplary embodiments of the present disclosure is shown. Figure 5 As shown, this optimization process mainly involves two parallel data flow paths. It should be noted that both data flow paths use the same neural network model.

[0079] In the path shown above, the target instruction features are input into the neural network model. After processing by the preceding layers, the original activation features h are output at the intermediate layer m. Subsequently, a feature shift operation is performed, that is, the preset semantic control feature v is subtracted from the original activation features h to obtain the target activation features h'. This target activation feature h' replaces the original activation features and continues to propagate forward. After processing by subsequent layers, the target result p_target is finally output at layer n.

[0080] In the path shown below, the test feature to be optimized is combined with the target instruction feature to obtain a combined input. This combined input is processed by the same neural network model. In the intermediate layer m, the controlled activation feature h_prefix, which is influenced by the test feature, is obtained. Finally, the final controlled result p_prefix is ​​obtained in layer n.

[0081] Based on the outputs of the two paths mentioned above, the loss value is calculated. As shown by the dashed arrow in the figure, the calculation of this loss value is based on two dimensions of constraints: one is the feature alignment constraint of the intermediate layer dimension, that is, calculating the difference between the target activation feature h' and the controlled activation feature h_prefix; the other is the result alignment constraint of the output layer dimension, that is, calculating the difference between the target result p_target and the controlled result p_prefix.

[0082] Finally, as shown by the solid arrow in the figure, backpropagation optimization is performed on the input test features based on the calculated loss value to obtain the final target test data.

[0083] According to some embodiments, the test data generation method may further include (not shown in the figure): combining the target test data with the target instruction features and inputting them into the model to be tested to obtain the test results output by the model to be tested.

[0084] Therefore, by directly applying the generated target test data to the input of the model under test, the effectiveness of the test data in real-world application scenarios can be verified. This approach achieves non-intrusive evaluation of model behavior, meaning that potential security risks or robustness defects in the model under test can be detected without accessing the model's internal parameters or gradients during the testing phase.

[0085] In some embodiments, the model to be tested can be a model with the same neural network model architecture and parameters as the one used in the aforementioned generation stage, thereby realizing a white-box verification scenario, or it can be a black-box model with a different neural network model architecture or that can only be accessed through an API interface, thereby realizing a migration attack or black-box testing scenario.

[0086] In one exemplary embodiment, target test data (e.g., an optimized text prefix) can be appended to the target instruction feature and sent as a new input sequence to the model under test. If the test result output by the model under test conforms to the second behavior pattern (e.g., outputting a specific answer to a restricted instruction), it can be determined that the model under test has a security vulnerability under the current target instruction.

[0087] According to another aspect of this disclosure, a test data generation apparatus for neural network models is provided. For example... Figure 6 As shown, the device 600 includes: a first acquisition unit 610, configured to input target instruction features into a neural network model and acquire the original activation features output by the intermediate layers of the neural network model; an offset unit 620, configured to offset the original activation features using preset semantic control features to obtain target activation features; a second acquisition unit 630, configured to acquire the target result output by subsequent layers in the neural network model based on the target activation features; a third acquisition unit 640, configured to input the test features to be optimized and the target instruction features into the neural network model and acquire the controlled activation features output by the intermediate layers and the controlled result output by the subsequent layers; and an optimization unit 650, configured to perform backpropagation optimization on the test features based on the target activation features, controlled activation features, target result, and controlled result to obtain target test data.

[0088] It is understood that the operation of units 610-650 in device 600 can refer to the description of steps S201-S205 in method 200 above, and will not be repeated here.

[0089] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0090] According to embodiments of this disclosure, an electronic device, a readable storage medium, and a computer program product are also provided.

[0091] refer to Figure 7The present invention describes a structural block diagram of an electronic device 700 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0092] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the electronic device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0093] Multiple components in electronic device 700 are connected to I / O interface 705, including: input unit 706, output unit 707, storage unit 708, and communication unit 709. Input unit 706 can be any type of device capable of inputting information to electronic device 700. Input unit 706 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 707 can be any type of device capable of presenting information, and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 708 may include, but is not limited to, disk and optical disk. Communication unit 709 allows electronic device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0094] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods, processes, and / or processes described above. For example, in some embodiments, these methods, processes, and / or processes may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the methods, processes, and / or processes described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform these methods, processes, and / or processes by any other suitable means (e.g., by means of firmware).

[0095] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0096] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0097] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0098] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0099] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0100] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0101] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0102] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A method for generating test data for a neural network model, comprising: The target instruction features are input into the neural network model, and the original activation features of the intermediate layer output of the neural network model are obtained. The original activation features are shifted using preset semantic control features to obtain the target activation features; Obtain the target result output by the subsequent layers in the neural network model after the intermediate layer, based on the target activation features; The test features to be optimized are combined with the target instruction features and input into the neural network model to obtain the controlled activation features output by the intermediate layer and the controlled results output by the subsequent layer. as well as Based on the target activation features, the controlled activation features, the target result, and the controlled result, the test features are optimized through backpropagation to obtain the target test data.

2. The method according to claim 1, wherein, The backpropagation optimization of the test features based on the target activation features, the controlled activation features, the target result, and the controlled result to obtain target test data includes: Based on the target activation features and the controlled activation features, a first loss value is determined; Based on the target result and the controlled result, a second loss value is determined; The first loss value and the second loss value are combined to obtain the comprehensive loss value; and The test features are optimized using backpropagation based on the comprehensive loss value.

3. The method according to claim 2, wherein, The step of determining the first loss value based on the target activation feature and the controlled activation feature includes: The first loss value is determined based on the L2 distance or cosine similarity between the target activation feature and the controlled activation feature.

4. The method according to claim 2, wherein, The determination of the second loss value based on the target result and the controlled result includes: The second loss value is determined based on the KL divergence between the target result and the controlled result.

5. The method according to any one of claims 1-4, wherein, The step of combining the test features to be optimized with the target instruction features and inputting them into the neural network model, and obtaining the controlled activation features output by the intermediate layer and the controlled results output by the subsequent layers includes: The test feature to be optimized is used as a prefix and concatenated with the target instruction feature, and the concatenation result is input into the neural network model.

6. The method according to any one of claims 1-4, wherein, The semantic control feature is used to switch the output response of the neural network model from a first behavior mode to a second behavior mode. The subsequent layers of the neural network model are configured to output an original result corresponding to the first behavior mode based on the original activation feature. The original result is different from the target result.

7. The method according to claim 6, wherein, The semantic control features are obtained through the following operations: Obtain a first sample set that can trigger the first behavior pattern and a second sample set that can trigger the second behavior pattern; The first sample set and the second sample set are respectively input into the neural network model, and the first sample activation feature output by the intermediate layer based on the first sample set and the second sample activation feature output based on the second sample set are obtained. as well as The semantic control features are determined based on the first sample activation features and the second sample activation features.

8. The method according to any one of claims 1-4, wherein, The step of shifting the original activation features using preset semantic control features to obtain target activation features includes: The target activation feature is obtained by subtracting the semantic control feature from the original activation feature.

9. The method according to any one of claims 1-4, wherein, The neural network model is a large language model. The target instruction features are obtained by embedding the target instruction text. The target results include a target probability distribution on the vocabulary and a target text result generated based on the target probability distribution. The controlled results include a controlled probability distribution on the vocabulary and a controlled text result generated based on the controlled probability distribution.

10. The method according to any one of claims 1-4, further comprising: The target test data and the target instruction features are combined and then input into the model to be tested to obtain the test results output by the model to be tested.

11. A test data generation device for a neural network model, comprising: The first acquisition unit is configured to input the target instruction features into the neural network model and acquire the original activation features output by the intermediate layer of the neural network model; The offset unit is configured to offset the original activation feature using a preset semantic control feature to obtain the target activation feature; The second acquisition unit is configured to acquire the target result output by the subsequent layer after the intermediate layer in the neural network model based on the target activation feature; The third acquisition unit is configured to combine the test features to be optimized with the target instruction features and input them into the neural network model, and acquire the controlled activation features output by the intermediate layer and the controlled results output by the subsequent layer. as well as The optimization unit is configured to perform backpropagation optimization on the test features based on the target activation features, the controlled activation features, the target result, and the controlled result to obtain target test data.

12. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-10.

13. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, Computer instructions are used to cause a computer to perform the method according to any one of claims 1-10.

14. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method of any one of claims 1-10.

Images