Safety prevention and control method, device and equipment based on large model neurons

Abstract

The embodiment of the invention discloses a safety prevention and control method, device and equipment based on large model neurons. In the embodiment of the invention, problem information is acquired; in response to the fact that a target neuron is activated in the reasoning process of the problem information, the activation value of the target neuron is adjusted, the target neuron is preset according to explanation of a plurality of neurons in a sparse auto-encoder, and the sparse auto-encoder is arranged on any layer in a large language model; according to the adjusted activation value, reply information is generated, and the reply information meets the safety prevention and control requirement. Through the method, interpretability of the neurons is realized, and on the basis of interpretability of the neurons, directional refined prevention and control are carried out on the output of the large model in the large model.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically, to a security prevention and control method, device and equipment based on large model neurons. Background Art

[0002] Large Language Model (LLM) is one of the groundbreaking technologies in the field of artificial intelligence. It can be applied to many scenarios such as human-computer interaction, content creation, and knowledge acquisition. Users input questions into the LLM, and the LLM returns the answers after processing. In some cases, the questions raised by users may pose security risks, and the LLM needs to be security-controlled to return answers that meet security control requirements.

[0003] In the existing technology, large-scale language models use endogenous prevention and control and external guardrails for security control. Specifically, endogenous prevention and control refers to the targeted cleaning of corpus during pre-training of large language models, or the acquisition of endogenous security capabilities through fine-tuning and reinforcement learning during post-training, and automatic safe response to harmful inputs; the outer guardrail will detect the content generated by the large language model, and if it adapts to specific rules, the generated content will be intercepted or rewritten; the above-mentioned endogenous protection has a long iteration cycle and high training cost; and the outer guardrail involves the linkage of multiple small models, and the adaptation rules are complex. After the large language model is iterated, the small models linked to it also need to be retrained.

[0004] To sum up, how to achieve targeted and refined prevention and control without retraining large-scale language models is a problem that needs to be solved at present. Summary of the Invention

[0005] In view of this, the embodiments of the present invention provide a security control method, device and equipment based on large model neurons to achieve the interpretability of neurons, and on the basis of the interpretability of neurons, perform targeted and refined control of the output of the large model within the large model.

[0006] In a first aspect, an embodiment of the present invention provides a security control method based on large model neurons, the method comprising: obtaining question information; in response to a target neuron being activated during the reasoning process of the question information, adjusting the activation value of the target neuron, wherein the target neuron is pre-set based on the interpretation of multiple neurons in a sparse autoencoder, and the sparse autoencoder is set at any layer in a large language model; generating reply information based on the adjusted activation value, wherein the reply information meets the security control requirements.

[0007] Optionally, the target neurons include risk neurons and safety neurons, wherein the risk neurons are neurons that increase the risk of the reply information, and the safety neurons are neurons that increase the security of the reply information.

[0008] Optionally, adjusting the activation value of the target neuron specifically includes: reducing the activation value of the risk neuron and increasing the activation value of the safety neuron.

[0009] Optionally, the method further includes: determining a plurality of neurons in the sparse autoencoder.

[0010] Optionally, determining a plurality of neurons in the sparse autoencoder specifically includes: obtaining pre-training data; training the sparse autoencoder according to the pre-training data; and determining a plurality of neurons in the trained sparse autoencoder.

[0011] Optionally, the training of the sparse autoencoder according to the pre-training data specifically includes: determining the input signal of the sparse autoencoder according to the pre-training data, wherein the input signal is a known output signal of a hidden layer in the large language model that is forward connected to the sparse autoencoder; determining the output signal of the sparse autoencoder according to the input signal; and training the sparse autoencoder according to the input signal, output signal and optimization target of the sparse autoencoder, wherein the optimization target includes reconstruction loss and sparsity loss.

[0012] Optionally, the method further includes: generating explanations of multiple neurons in the sparse autoencoder.

[0013] Optionally, the generating of explanations of multiple neurons in the sparse autoencoder specifically includes: obtaining sample data; for any neuron, generating multiple activation values ​​based on the sample data; normalizing the multiple activation values ​​and mapping them to a plurality of pre-set intervals; generating first prompt information based on a training set corresponding to the activation values ​​of the plurality of intervals, wherein the training set includes a plurality of the sample data; inputting the first prompt information into a first large model to generate a candidate explanation of any neuron; generating second prompt information based on the candidate explanation of any neuron and a verification set; inputting the second prompt information into a second large model to generate a verification activation value of any neuron on the verification set; obtaining a true activation value on the verification set; determining a correlation value between the verification activation value and the true activation value; and determining that the candidate explanation of any neuron is the explanation of any neuron in response to the correlation value being greater than a set threshold.

[0014] In second aspect, an embodiment of the present invention provides a security control device based on large model neurons, the device comprising: an acquisition unit for acquiring question information; a processing unit for adjusting the activation value of the target neuron in response to the target neuron being activated during the reasoning process of the question information, wherein the target neuron is pre-set according to the interpretation of multiple neurons in a sparse autoencoder, and the sparse autoencoder is set at any layer in a large language model; a generation unit for generating reply information based on the adjusted activation value, wherein the reply information meets the security control requirements.

[0015] Optionally, the target neurons include risk neurons and safety neurons, wherein the risk neurons are neurons that increase the risk of the reply information, and the safety neurons are neurons that increase the security of the reply information.

[0016] Optionally, the processing unit is specifically configured to: reduce the activation value of the risk neuron and increase the activation value of the safety neuron.

[0017] Optionally, the device further includes: a determination unit, configured to determine a plurality of neurons in the sparse autoencoder.

[0018] Optionally, the determining unit is specifically used to: obtain pre-training data; train the sparse autoencoder according to the pre-training data; and determine a plurality of neurons in the trained sparse autoencoder.

[0019] Optionally, the determination unit is further specifically used to: determine the input signal of the sparse autoencoder based on the pre-training data, wherein the input signal is a known output signal of a hidden layer in the large language model that is forward connected to the sparse autoencoder; determine the output signal of the sparse autoencoder based on the input signal; train the sparse autoencoder based on the input signal, output signal and optimization target of the sparse autoencoder, wherein the optimization target includes reconstruction loss and sparsity loss.

[0020] Optionally, the generating unit is further used to generate explanations of multiple neurons in the sparse autoencoder.

[0021] Optionally, the generation unit is specifically used to: obtain sample data; for any neuron, generate multiple activation values ​​based on the sample data; normalize the multiple activation values ​​and map them to a plurality of pre-set intervals; generate a first prompt information based on a training set corresponding to the activation values ​​of the plurality of intervals, wherein the training set includes a plurality of the sample data; input the first prompt information into a first large model to generate a candidate explanation of the any neuron; generate a second prompt information based on the candidate explanation of the any neuron and a verification set; input the second prompt information into a second large model to generate a verification activation value of the any neuron on the verification set; obtain the true activation value on the verification set; determine a correlation value between the verification activation value and the true activation value; in response to the correlation value being greater than a set threshold, determine that the candidate explanation of the any neuron is the explanation of the any neuron.

[0022] In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement a method as described in the first aspect or any possible embodiment of the first aspect.

[0023] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the method as described in the first aspect or any possible embodiment of the first aspect.

[0024] In an embodiment of the present invention, question information is obtained; in response to the activation of a target neuron during the reasoning process of the question information, the activation value of the target neuron is adjusted, wherein the target neuron is pre-set based on the interpretation of multiple neurons in a sparse autoencoder, and the sparse autoencoder is set at any layer in a large language model; and reply information is generated based on the adjusted activation value, wherein the reply information meets the security control requirements. Through the above method, the interpretability of neurons is achieved, and based on the interpretability of neurons, the output of the large model is subjected to targeted and refined control within the large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0026] Figure 1 This is a flow chart of a safety control method based on large model neurons in an embodiment of the present invention;

[0027] Figure 2This is a schematic diagram of a Transformer architecture in an embodiment of the present invention;

[0028] Figure 3 is a flow chart of a method for determining a plurality of neurons in the sparse autoencoder according to an embodiment of the present invention;

[0029] Figure 4 is a flow chart of a method for generating explanations of multiple neurons in the sparse autoencoder according to an embodiment of the present invention;

[0030] Figure 5 Schematic diagram of a safety control device based on a large model neuron in an embodiment of the present invention;

[0031] Figure 6 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present application is described below based on the following embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without the description of these details. To avoid obscuring the essence of the present application, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0033] Furthermore, persons of ordinary skill in the art will appreciate that the figures provided herein are for illustration purposes only and are not necessarily drawn to scale.

[0034] Unless the context clearly requires otherwise, words like “include”, “comprising” and the like throughout this application should be interpreted as including rather than exclusive or exhaustive; that is, as meaning “including but not limited to”.

[0035] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0036] In the existing technology, large language models (LLM) use two methods for security control: endogenous control and external guardrails. Among them, method 1, the above-mentioned endogenous control refers to the targeted cleaning of corpus during pre-training of large language models, or the acquisition of endogenous security capabilities through fine-tuning and reinforcement learning (RL) during post-training, and automatic safe response to harmful inputs. Among them, the targeted cleaning of corpus during pre-training refers to accurately identifying and retaining high-quality data related to the target scenario to be processed by the large language model, while eliminating noise and redundant information, which can significantly improve data quality, thereby enhancing the quality of large language models. The performance and reliability of the language model, flexibly adjust the cleaning strategy according to the target scenario, and continuously iterate and optimize; the post-training process refers to the key stage from pre-training to actual deployment of the large language model, through the data of specific tasks and optimization methods, to improve the performance of the large language model in the target scenario. The iteration cycle of the above-mentioned endogenous protection is long and the training cost is high; Method 2, the outer guardrail will detect the content generated by the large language model, and if it adapts to specific rules, the generated content will be intercepted or rewritten; the outer guardrail involves the linkage of multiple small models, and the adaptation rules are complex. After the large language model is iterated, the small models linked to it also need to be retrained. Therefore, how to achieve targeted and refined prevention and control without retraining the large language model is a problem that needs to be solved at present.

[0037] In an embodiment of the present invention, the large language model can also be referred to as an artificial intelligence (AI) model or a large model, wherein the large language model is a deep learning model based on a transformer architecture, which can process and generate natural language text. It is usually trained on a large amount of text data, has the ability to understand and generate language, and is widely used in dialogue systems, text generation and other natural language processing tasks.

[0038] In order to solve the above problems, a safety control method based on large model neurons is proposed in the embodiment of the present invention. Figure 1 As shown, the method includes:

[0039] Step S101: Obtain question information.

[0040] Specifically, the large language model obtains question information input by the user, and the question information is a question that the user needs the large language model to answer. For example, the question information is "How to start an electric car without a key?" This is only an example. The specific question information is determined according to the actual situation. The embodiment of the present invention does not limit the scenario, type, etc. of the question information.

[0041] In one possible implementation, the user inputs question information into the front-end screen of the smart terminal, and the large language model is placed in the smart terminal to obtain the question information input by the user into the screen. This is only an exemplary description.

[0042] Step S102: In response to a target neuron being activated during the reasoning process of the question information, adjusting the activation value of the target neuron.

[0043] The target neuron is pre-set according to the interpretation of multiple neurons in a sparse autoencoder (SAE), and the sparse autoencoder is set at any layer in a large language model, such as a self-attention layer, a multi-layer perceptron (MLP) layer, a hidden layer output layer, a residual hidden layer output layer, etc. Figure 2 As shown, the Transformer architecture of any of the above layers includes core components such as the Attention Mechanism 201, the Feed-Forward Network (FFN) 202, and the Hidden States 203. The Sparse Autoencoder SAE can be connected to any of the above components. Figure 2 In the embodiment, the sparse autoencoder SAE is connected to the Hidden States. The sparse autoencoder SAE may also be connected to the FFN or to the Attention, which is determined according to the actual situation.

[0044] In an embodiment of the present invention, the sparse autoencoder SAE is a special autoencoder whose goal is to learn an efficient and sparse representation of input data by introducing sparsity constraints in the hidden layer. The SAE includes multiple neurons, and each neuron in the SAE is interpretable, that is, each neuron has its corresponding explanation. When a user uses a large language model in a certain scenario, the user pre-selects target neurons that may be used by the large language model from multiple neurons. Assume that the SAE includes 2000 neurons, each neuron has a corresponding explanation. The user first enters a search term to filter out multiple neurons related to its usage scenario from the 2000 neurons, and then selects a target neuron from the multiple filtered neurons, wherein the target neuron includes risk neurons and safety neurons, wherein the risk neuron is a neuron that increases the risk of the reply information, and the safety neuron is a neuron that increases the security of the reply information.

[0045] In one possible implementation, after determining that the risk neuron and the safety neuron have been activated, the activation value of the risk neuron can be reduced and the activation value of the safety neuron can be increased. By adjusting the activation values ​​of the risk neuron and the safety neuron, hidden layer protection of the large model is achieved.

[0046] Step S103: Generate reply information according to the adjusted activation value, wherein the reply information meets the security control requirements.

[0047] Specifically, since the activation value of the risky neurons is suppressed and the activation value of the safe neurons is enhanced, the possibility of the reply information containing risky content is reduced.

[0048] In an embodiment of the present invention, the above steps 101 to S103 are the process of using the large language model that has been trained. Prior to this, two processes are required: determining multiple neurons in the sparse autoencoder and generating interpretations of multiple neurons in the sparse autoencoder.

[0049] In a possible implementation, the method for determining a plurality of neurons in the sparse autoencoder is as follows: Figure 3 As shown, the following steps are included:

[0050] Step S301: Obtain pre-training data.

[0051] Specifically, the output data of any layer in the large language model is selected as the pre-training data of SAE, that is, training. The any layer includes the self-attention layer, the multi-layer perceptron layer, the hidden layer output layer, the residual hidden layer output layer, etc., which is determined according to the actual situation.

[0052] In an embodiment of the present invention, the pre-training data is relatively comprehensive data in the scenario in which the large-scale language model is to be used.

[0053] Step S302: training the sparse autoencoder according to the pre-training data.

[0054] Specifically, the input signal of the sparse autoencoder is determined according to the pre-training data, wherein the input signal is a known output signal of a hidden layer in the large language model that is forward connected to the sparse autoencoder; the output signal of the sparse autoencoder is determined according to the input signal; the sparse autoencoder is trained according to the input signal, output signal and optimization target of the sparse autoencoder, wherein the optimization target includes reconstruction loss (LOSS) and sparsity loss, the reconstruction loss is used to represent the reconstruction quality of the feature, and the sparsity loss is used to ensure the quality of feature learning.

[0055] In an embodiment of the present invention, the SAE learns the sparse representation of data through an encoder and decoder structure, in which neurons perform the following core functions: in the encoder stage, the input data is mapped to a higher-dimensional latent space through neurons, but the number of neurons actually activated is extremely small; in the decoder stage, the sparse coding vector reconstructs the original input through neurons, requiring neurons to have nonlinear activation capabilities to capture complex features, and the sparsity constraints of SAE enable large language models to rely only on a small number of key neurons to express features.

[0056] Step S303: Determine a plurality of neurons in the trained sparse autoencoder.

[0057] Assume that it is determined that the sparse autoencoder includes 2000 neurons. This is only an exemplary description and is determined based on actual conditions.

[0058] In one possible implementation, the method for generating the explanation of multiple neurons in the sparse autoencoder is as follows: Figure 4 As shown, the following steps are included:

[0059] Step S401: Obtain sample data.

[0060] Specifically, the sample data is relatively comprehensive data for any scenario or any task. The sample data is divided into positive sample data and negative sample data. The positive sample data can also be called white sample data, and the negative sample data can also be called black sample data.

[0061] Step S402: For any neuron, generate multiple activation values ​​according to the sample data.

[0062] In one possible implementation, the SAE includes multiple neurons, and each neuron needs to be interpreted one by one. For each neuron, reasoning is performed by token based on the sample data to obtain an activation value by token activation. When reasoning by token, the sample data needs to be divided into tokens first. For example, "How to start an electric car without a key?" is divided into "How to start an electric car without a key". The multiple activation values ​​of any neuron are determined based on the divided tokens. This is only an example and will be determined based on actual conditions.

[0063] In an embodiment of the present invention, the activation value can be a positive value or a negative value. Before subsequent processing, it needs to be filtered by polarity to retain the positive value or the negative value. If the positive value is retained, subsequent processing is performed directly. If the negative value is retained, the modulus value of the negative value is determined before subsequent processing.

[0064] Step S403: normalize the multiple activation values ​​and map them to multiple preset intervals.

[0065] Assume that the preset multiple intervals are 0-10, with a total of 11 intervals. The multiple activation values ​​are normalized and mapped to the above 11 intervals. This is only an exemplary description, and the specific number of intervals is determined according to actual conditions.

[0066] Step S404: Generate first prompt information according to the training set corresponding to the activation values ​​of the multiple intervals.

[0067] The training set includes a plurality of sample data.

[0068] Specifically, the training set is a portion of the sample data corresponding to all activation values. For example, 80% of the sample data corresponding to all activation values ​​is selected as the training set. It can also be other quantities, such as 70%, 75%, etc., which is determined according to actual conditions.

[0069] In one possible implementation, the first prompt information includes the following content: 1. System prompt; 2. 3 examples; 3. 10 samples obtained from the highest activation value interval; 3. 2 samples obtained from other activation value intervals; 5. 5 completely random samples. The above first prompt information is only an example description and will be determined based on actual conditions.

[0070] Step S405: input the first prompt information into the first large model to generate a candidate explanation of any neuron.

[0071] Specifically, the first prompt information is input into the first large model, and a candidate explanation of any neuron is generated through random sampling and iteration.

[0072] Step S406: Generate second prompt information based on the interpretation and verification set of any neuron.

[0073] Specifically, the validation set is a portion of the sample data corresponding to all activation values. For example, 20% of the sample data corresponding to all activation values ​​is selected as the validation set. It can also be other quantities, such as 25%, 30%, etc., which is determined according to actual conditions.

[0074] In one possible implementation, a second prompt information is generated based on the token list corresponding to the sample data in the candidate interpretation and verification set. The second prompt information includes the following contents: 1. System prompt; 2. Test instructions; 3. 3 examples and random unknown things; 4. A token list for predicting the activation of all activated verb tokens masked as "unknown". The above first prompt information is only an exemplary description and will be determined based on actual conditions.

[0075] Step S407: input the second prompt information into the second large model to generate a verification activation value of any neuron on the verification set.

[0076] Specifically, the second large model may be a large language model used in training SAE, or may be other models.

[0077] In one possible implementation, the second large model performs a forward propagation on the second prompt information, and uses the unnormalized prediction score (logits) of each unpositioned word, and the unnormalized prediction score of 0-10 words of the word source masked by the unknown word as an activation function (softmax) to generate an expected verification activation value, which is in the range of 0-10; or, the second large model directly generates the verification activation value of any neuron on the verification set based on the input second prompt information.

[0078] Step S408: Obtain the true activation value on the verification set.

[0079] Specifically, the actual activation value is also in the range of 0-10.

[0080] Step S409: Determine the correlation value between the verification activation value and the true activation value.

[0081] Specifically, the data corresponding to the verification activation value and the data corresponding to the true activation value are determined respectively, and the correlation data between the data corresponding to the verification activation value and the data corresponding to the true activation value is calculated. The correlation value represents the confidence level, and the higher the correlation value, the higher the confidence level of the candidate explanation of the neuron.

[0082] Step S410: In response to the correlation value being greater than a set threshold, determining the candidate interpretation of the any neuron as the interpretation of the any neuron.

[0083] Among them, the set threshold is determined according to the actual situation and is not limited here. When the correlation value is greater than the set threshold, it means that the confidence of the candidate explanation is high and the corresponding neuron can be explained. Therefore, the candidate explanation can be determined as the explanation of any neuron.

[0084] In the embodiment of the present invention, through the above Figure 4 The demonstrated method can determine the explanation of each neuron in SAE, and on this basis, perform security control on the output of the large model. Compared with the traditional black box large model, there is no need to iteratively train the large model, and it realizes point-to-point and targeted refined control based on the interpretability of neurons.

[0085] In an embodiment of the present invention, a safety control device based on a large model neuron is provided. Figure 5 As shown, it specifically includes: an acquisition unit 501, a processing unit 502 and a generation unit 503;

[0086] Among them, the acquisition unit 501 is used to obtain question information; the processing unit 502 is used to adjust the activation value of the target neuron in response to the target neuron being activated during the reasoning process of the question information, wherein the target neuron is pre-set according to the interpretation of multiple neurons in the sparse autoencoder, and the sparse autoencoder is set at any layer in a large language model; the generation unit 503 is used to generate reply information according to the adjusted activation value, wherein the reply information meets the security control requirements.

[0087] Furthermore, the target neurons include risk neurons and safety neurons, wherein the risk neurons are neurons that increase the risk of the reply information, and the safety neurons are neurons that increase the safety of the reply information.

[0088] Furthermore, the processing unit is specifically configured to reduce the activation value of the risk neuron and increase the activation value of the safety neuron.

[0089] Furthermore, the device also includes: a determination unit, configured to determine a plurality of neurons in the sparse autoencoder.

[0090] Furthermore, the determining unit is specifically configured to: obtain pre-training data; train the sparse autoencoder according to the pre-training data; and determine a plurality of neurons in the trained sparse autoencoder.

[0091] Furthermore, the determination unit is specifically used to: determine the input signal of the sparse autoencoder based on the pre-training data, wherein the input signal is a known output signal of a hidden layer in the large language model that is forward connected to the sparse autoencoder; determine the output signal of the sparse autoencoder based on the input signal; train the sparse autoencoder based on the input signal, output signal and optimization target of the sparse autoencoder, wherein the optimization target includes reconstruction loss and sparsity loss.

[0092] Furthermore, the generating unit is further configured to generate explanations of multiple neurons in the sparse autoencoder.

[0093] Furthermore, the generation unit is specifically used to: obtain sample data; for any neuron, generate multiple activation values ​​based on the sample data; normalize the multiple activation values ​​and map them to a plurality of pre-set intervals; generate a first prompt information based on a training set corresponding to the activation values ​​of the plurality of intervals, wherein the training set includes a plurality of the sample data; input the first prompt information into a first large model to generate a candidate interpretation of the any neuron; generate a second prompt information based on the interpretation and verification set of the any neuron; input the second prompt information into a second large model to generate a verification activation value of the any neuron on the verification set; obtain the true activation value on the verification set; determine the correlation value between the verification activation value and the true activation value; in response to the correlation value being greater than a set threshold, determine that the candidate interpretation of the any neuron is the interpretation of the any neuron.

[0094] Figure 6 FIG. 1 is a schematic diagram of the structure of the electronic device according to an embodiment of the present invention. Figure 6 As shown, it includes a general computer hardware structure, which includes at least a processor 601 and a memory 602. The processor 601 and the memory 602 are connected via a bus 603. The memory 602 is suitable for storing instructions or programs executable by the processor 601. The processor 601 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 601 executes the instructions stored in the memory 602, thereby executing the method flow of the embodiment of the present invention described above to process data and control other devices. The bus 603 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 604 and the display device and the input / output (I / O) device 605. The input / output (I / O) device 605 can be a mouse, keyboard, modem, network interface, touch input device, somatosensory input device, printer, and other devices known in the art. Typically, the input / output device 605 is connected to the system via an input / output (I / O) controller 606.

[0095] In which, the instructions stored in the memory 602 are executed by at least one processor 601 to implement: obtaining question information; in response to the target neuron being activated during the reasoning process of the question information, adjusting the activation value of the target neuron, wherein the target neuron is pre-set based on the interpretation of multiple neurons in a sparse autoencoder, and the sparse autoencoder is set at any layer in a large language model; generating reply information based on the adjusted activation value, wherein the reply information meets the security control requirements.

[0096] Specifically, the electronic device includes: one or more processors 601 and a memory 602, Figure 6 Take a processor 601 as an example. The processor 601 and the memory 602 may be connected via a bus or other means. Figure 6 In the example, a bus connection is used. Memory 602, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. Processor 601 executes the non-volatile software programs, instructions, and modules stored in memory 602 to execute various functional applications and data processing of the device, thereby implementing the aforementioned large-model neuron-based security control method.

[0097] The memory 602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store a list of options, etc. In addition, the memory 602 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 602 may optionally include a memory remotely located relative to the processor 601, and these remote memories may be connected to an external device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0098] One or more modules are stored in the memory 602, and when executed by one or more processors 601, the security prevention and control method based on large model neurons in any of the above method embodiments is executed.

[0099] As will be appreciated by those skilled in the art, various aspects of embodiments of the present invention may be implemented as systems, methods, or computer program products. Thus, various aspects of embodiments of the present invention may take the form of a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software aspects with hardware aspects, which may all be generally referred to herein as a "circuit," "module," or "system." Additionally, various aspects of embodiments of the present invention may take the form of a computer program product implemented in one or more computer-readable media having computer-readable program code implemented thereon.

[0100] Any combination of one or more computer-readable media can be utilized. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples (non-exhaustive enumeration) of computer-readable storage media will include the following: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the context of an embodiment of the present invention, a computer-readable storage medium can be any tangible medium that can contain or store a program used by an instruction execution system, device, or apparatus, or a program used in conjunction with an instruction execution system, device, or apparatus.

[0101] A computer-readable signal medium may include a propagated digital signal having computer-readable program code embodied therein, such as in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or apparatus.

[0102] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0103] The computer program code for performing the operations for various aspects of the embodiments of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" programming language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer as a stand-alone software package; partially on the user's computer and partially on a remote computer; or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] The flowchart legends and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention described above describe various aspects of embodiments of the present invention. It will be understood that each block of the flowchart legends and / or block diagrams and the combination of blocks in the flowchart legends and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that the instructions (executed by the processor of the computer or other programmable data processing device) create a device for implementing the function / action specified in the flowchart and / or block diagram block or block.

[0105] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing device, or other apparatus to operate in a particular manner, so that the instructions stored in the computer-readable medium produce an article of manufacture that includes instructions for implementing the functions / actions specified in the flowchart and / or block diagram blocks or blocks.

[0106] The computer program instructions may also be loaded onto a computer, other programmable data processing device or other apparatus to cause a series of operable steps to be performed on the computer, other programmable device or other apparatus to produce a computer-implemented process, such that the instructions executed on the computer or other programmable device provide a process for implementing the functions / actions specified in the flowchart and / or block diagram block or blocks.

[0107] The foregoing is merely a preferred embodiment of the present application and is not intended to limit the present application. Persons skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application are intended to be within the scope of protection of the present application.

[0108] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation portals for users to choose to authorize or refuse. The user's refusal to process personal information other than the necessary information for basic functions will not affect the user's use of basic functions.

Claims

1. A safety control method based on large model neurons, characterized in that: The method comprises: Get problem information; In response to a target neuron being activated during the reasoning process of the question information, adjusting the activation value of the target neuron, wherein the target neuron is pre-set based on the interpretation of multiple neurons in a sparse autoencoder, and the sparse autoencoder is set in any layer of a large language model; Generate reply information according to the adjusted activation value, wherein the reply information meets the security control requirements.

2. The method according to claim 1, characterized in that The target neurons include risk neurons and safety neurons, wherein the risk neurons are neurons that increase the risk of the reply information, and the safety neurons are neurons that increase the safety of the reply information.

3. The method according to claim 2, characterized in that The adjusting the activation value of the target neuron specifically includes: The activation value of the risk neuron is reduced, and the activation value of the safety neuron is increased.

4. The method according to claim 1, wherein The method further comprises: A plurality of neurons in the sparse autoencoder is determined.

5. The method according to claim 4, characterized in that The determining of a plurality of neurons in the sparse autoencoder specifically includes: Get pre-training data; training the sparse autoencoder based on the pre-training data; A plurality of neurons in the trained sparse autoencoder is determined.

6. The method according to claim 5, characterized in that The step of training the sparse autoencoder according to the pre-training data specifically includes: Determining an input signal of the sparse autoencoder based on the pre-training data, wherein the input signal is a known output signal of a hidden layer in the large language model that is forward-connected to the sparse autoencoder; Determining an output signal of the sparse autoencoder according to the input signal; The sparse autoencoder is trained according to an input signal, an output signal and an optimization target of the sparse autoencoder, wherein the optimization target includes a reconstruction loss and a sparsity loss.

7. The method according to claim 5, characterized in that The method further comprises: Generate explanations for multiple neurons in the sparse autoencoder.

8. The method according to claim 7, characterized in that Generating an explanation of the plurality of neurons in the sparse autoencoder specifically comprises: Get sample data; For any neuron, generating multiple activation values ​​according to the sample data; Normalizing the multiple activation values ​​and mapping them to multiple pre-set intervals; generating first prompt information according to a training set corresponding to the activation values ​​of the plurality of intervals, wherein the training set includes a plurality of the sample data; Inputting the first prompt information into a first large model to generate a candidate explanation for any of the neurons; generating second prompt information according to the candidate explanation and verification set of any one of the neurons; Inputting the second prompt information into the second large model to generate a verification activation value of any neuron on the verification set; Obtaining the true activation value on the verification set; Determining a correlation value between the verification activation value and the true activation value; In response to the correlation value being greater than a set threshold, the candidate interpretation of the any one neuron is determined to be the interpretation of the any one neuron.

9. A safety control device based on a large model neuron, characterized in that: The device comprises: An acquisition unit, used to obtain question information; a processing unit, in response to a target neuron being activated during the reasoning process of the question information, for adjusting an activation value of the target neuron, wherein the target neuron is pre-set based on an interpretation of a plurality of neurons in a sparse autoencoder, the sparse autoencoder being provided in any layer of a large language model; A generating unit is used to generate a reply message according to the adjusted activation value, wherein the reply message meets the security control requirements.

10. An electronic device comprising a memory and a processor, characterized in that: The memory is configured to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images