Power system large model security enhancement method and related device

Abstract

The invention belongs to the technical field of electric power intelligence, and discloses an electric power system large model security enhancement method and related device.The method comprises the steps that according to an adversarial attack target, an adversarial embedding vector is constructed in combination with visual component features of a joint embedding space of an electric power system large model; constructing an adversarial image according to the adversarial embedding vector and the original image; wherein the adversarial image and the original image meet a preset similarity constraint, and an embedding vector after the adversarial image is mapped to the joint embedding space is located in a dangerous area defined by the adversarial embedding vector; and matching the adversarial image with the text prompt to obtain a training sample, and training the large model of the power system according to the training sample. And training the large model of the power system according to the training sample to effectively improve the safety of the large model of the power system, thereby realizing the safety guarantee of the large model of the power system in the application of the power system, enhancing the safety and credibility of the large model of the power system, and further ensuring the safe and stable operation of the power system.

Description

Technical Field

[0001] The present invention belongs to the field of electric power intelligence technology and relates to a method for enhancing the safety of a large model of an electric power system and a related device. Background Art

[0002] We are currently at the intersection of energy transformation and digital technology innovation. Artificial intelligence technology represented by large language models is becoming a key strategic tool to lead this transformation. The large model of the power system, which combines the large language model with the power system, will surely have a huge driving force on the development of the power system and technological progress, especially in the process of building a new power system with new energy as the main body. It will surely play a core and key role. At the same time, the cloud-edge computing architecture provides an innovative solution for optimizing the energy efficiency and edge deployment of large language models. On this basis, the co-evolution strategy between large and small models is further integrated into the power system to achieve more efficient resource utilization and more optimized decision support. However, the large model is essentially based on deep learning, and the robustness and vulnerability of its algorithm and its susceptibility to adversarial attacks also bring great security risks to the widespread application of large models of power systems, including model theft and output errors caused by various attacks. The security vulnerabilities of the large model of the power system may be exploited by attackers, which in turn puts the associated businesses of the large model of the power system at risk of overall failure, and further threatens the application ecology built on the large model of the power system and related industries. Therefore, it is necessary and of practical significance to study the security issues of large power system models, improve the security of large power system models, and detect malicious samples.

[0003] The security assessment of large power system models refers to the process of systematically analyzing and evaluating the security of large power system models in specific application scenarios. Among them, verifying whether the large power system model can stably provide the expected output for abnormal or malicious changes in data is a key indicator for verifying the security performance of large power system models. At present, malicious attacks on large power system models mainly use adversarial text input and prompt word injection. Such attacks can mislead large models to produce erroneous outputs and are transferable, allowing them to bypass the security barriers of various large power system models.

[0004] However, text-based attacks can be easily detected by automatic filters or detectors, leading to the generation of security patches, which reduces its effectiveness as a persistent threat to assess model security. With the development of multimodal power system large models, the multimodal form brings new adversarial attack vulnerabilities. Malicious attackers exploit the vulnerability of cross-modal alignment in multimodal power system large models and only change the image to generate targeted attacks to induce the power system large model to output incorrect text representations. Such malicious attacks on the visual components of power system large models are not only severely destructive, but also usually difficult to detect, which makes them an important indicator affecting the security performance of power system large models. How to effectively enhance the security defense capabilities of power system large models and effectively identify and avoid these hidden attacks has become an urgent problem to be solved. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide a method for enhancing the safety of a large model of an electric power system and related devices.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for enhancing the safety of a large model of a power system, comprising: constructing an adversarial embedding vector based on an adversarial attack target and combining visual component features of a joint embedding space of a large model of the power system; constructing an adversarial image based on the adversarial embedding vector and an original image; wherein the adversarial image satisfies a preset similarity constraint with the original image, and the embedding vector of the adversarial image after being mapped to the joint embedding space is within a danger zone defined by the adversarial embedding vector; matching the adversarial image with a text prompt to obtain a training sample, and training the large model of the power system based on the training sample.

[0008] Optionally, constructing the adversarial image according to the adversarial embedding vector and the original image includes: iteratively optimizing the adversarial image malicious trigger according to a preset optimization objective function, and using the iteratively optimized adversarial image malicious trigger to modify the original image to obtain the adversarial image; wherein the optimization objective function is:

[0009]

[0010] in, is the original image modified by the adversarial image malicious trigger, and L2 is the calculation The function of the gap between harm To counter the embedding vector, I φ is a pre-trained image encoder, is the original image, B is and Similarity constraints between .

[0011] Optionally, the adversarial image malicious trigger is: a visual trigger, or a trigger combining optical character recognition text and vision; the pre-trained image encoder is pre-trained using a multimodal learning framework CLIP; the calculation h harm and The function of the distance between them is the Euclidean distance function; the original image is a random noise distribution image, a white background image or an undisturbed image.

[0012] Optionally, it also includes: obtaining a text description output by the power system large model based on the input sample, and generating a comparison image using a text-to-image generator based on the text description; obtaining the similarity between the input image and the comparison image in the input sample, and obtaining an adversarial detection result of the input image based on the similarity between the input image and the comparison image.

[0013] Optionally, the obtaining of the similarity between the input image and the comparison image includes: traversing a plurality of preset pre-trained image encoders: obtaining the encoding vector of the input image and the encoding vector of the comparison image using the current pre-trained image encoder, and obtaining the similarity between the encoding vector of the input image and the encoding vector of the comparison image under the current pre-trained image encoder; obtaining the average of the similarities between the encoding vector of the input image and the encoding vector of the comparison image under all pre-trained image encoders as the similarity between the input image and the comparison image.

[0014] Optionally, the adversarial detection result of the input image obtained by the similarity between the input image and the comparison image includes: when the similarity between the input image and the comparison image is greater than a preset similarity threshold, the adversarial detection result of the input image is a non-adversarial image; when the similarity between the input image and the comparison image is not greater than a preset similarity threshold, the adversarial detection result of the input image is an adversarial image.

[0015] According to a second aspect of the present invention, a system for enhancing the safety of a large model of a power system is provided, comprising: a vector construction module, for constructing an adversarial embedding vector according to an adversarial attack target and in combination with visual component features of a joint embedding space of a large model of a power system; an image construction module, for constructing an adversarial image according to the adversarial embedding vector and an original image; wherein the adversarial image satisfies a preset similarity constraint with the original image, and the embedding vector of the adversarial image after being mapped to the joint embedding space is within a danger zone defined by the adversarial embedding vector; and a training module, for matching the adversarial image with a text prompt to obtain a training sample, and training the large model of the power system according to the training sample.

[0016] Optionally, it also includes: an image generation module, used to obtain the text description output by the power system large model based on the input sample, and generate a comparison image using a text-to-image generator according to the text description; an adversarial detection module, used to obtain the similarity between the input image and the comparison image in the input sample, and obtain the adversarial detection result of the input image according to the similarity between the input image and the comparison image.

[0017] According to a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for enhancing the safety of a large power system model when executing the computer program.

[0018] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for enhancing the safety of a large power system model are implemented.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] The method for enhancing the security of a large model of a power system of the present invention constructs an adversarial embedding vector according to the adversarial attack target combined with the visual component features of the joint embedding space of the large model of the power system, and then constructs an adversarial image according to the adversarial embedding vector and the original image, and generates a more concealed adversarial image through a malicious attack method based on the embedding space. Next, the adversarial image is matched with the text prompt to obtain a training sample, and in view of the vulnerability of cross-text and image modal alignment, the semantic alignment of the large model of the power system is destroyed by matching the adversarial image with the text prompt, and then the large model of the power system is trained according to the training sample, so as to effectively improve the security of the large model of the power system, thereby realizing the security of the large model of the power system in the application of the power system, enhancing the security and reliability of the large model of the power system, and thus ensuring the safe and stable operation of the power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The present invention is a flowchart of a method for enhancing the safety of a large power system model according to an embodiment of the present invention.

[0022] Figure 2 Schematic diagram of the adversarial image training principle of a large model of a power system according to an embodiment of the present invention.

[0023] Figure 3 This is a flow chart of a method for enhancing the safety of a large power system model according to another embodiment of the present invention.

[0024] Figure 4 Schematic diagram of the adversarial image detection principle of an embodiment of the present invention.

[0025] Figure 5 The present invention is a block diagram of the structure of a large-scale model security enhancement system for a power system according to an embodiment of the present invention.

[0026] Figure 6 This is a structural block diagram of a power system large model security enhancement system according to another embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0030] See also Figure 1 and 2 In one embodiment of the present invention, a method for enhancing the security of a large power system model is provided. Aiming at the vulnerability of cross-modal alignment of the large power system model, adversarial images are matched with text prompts to form training samples, and training is performed based on the training samples to enhance the security of the large power system model.

[0031] Specifically, the method for enhancing the safety of a large power system model of the present invention comprises the following steps:

[0032] S1: According to the adversarial attack target, the adversarial embedding vector is constructed by combining the visual component features of the joint embedding space of the large power system model.

[0033] S2: constructing an adversarial image according to the adversarial embedding vector and the original image; wherein the adversarial image satisfies a preset similarity constraint with the original image, and the embedding vector after the adversarial image is mapped to the joint embedding space is located in a danger zone defined by the adversarial embedding vector.

[0034] S3: Match the adversarial images with the text prompts to obtain training samples, and train the power system large model based on the training samples.

[0035] The method for enhancing the security of a large model of a power system of the present invention constructs an adversarial embedding vector according to the adversarial attack target combined with the visual component features of the joint embedding space of the large model of the power system, and then constructs an adversarial image according to the adversarial embedding vector and the original image, and generates a more concealed adversarial image through a malicious attack method based on the embedding space. Then, the adversarial image is matched with the text prompt to obtain a training sample, and in view of the vulnerability of cross-text and image modal alignment, the semantic alignment of the large model of the power system is destroyed by matching the adversarial image with the text prompt, so as to achieve a security assessment of the large model, ensure the security and reliability of the large model in practical applications, and then train the large model of the power system according to the training sample, so as to effectively improve the security of the large model of the power system, thereby achieving the security guarantee of the large model of the power system in the application of the power system, enhancing the security and reliability of the large model of the power system, and thus ensuring the safe and stable operation of the power system.

[0036] Explainable, cross-modal power system models usually pre-train an image encoder with fixed parameters. φ (·) and the large language model f θ (·) Alignment, using the projection layer W I Aligning the joint embedding space Z, which also covers the features extracted by the text encoder T(·), the final output Y of the power system large model can be expressed as:

[0037] Y=f θ ([h i ,h t ]),h i =W I ·I(x i ),h t =T(x t )

[0038] Among them, h i and h t Respectively represent the image input x i and text input x t Obtain visual features and text features from the image encoder and text encoder, h i ,h t ∈Z. The trainable parameters of the cross-modal power system model include θ and WI , Image Encoder I φ (·) Usually a pre-trained model that is kept frozen during ensemble.

[0039] The adversarial attack is carried out on the cross-modal power system big model. The goal of the adversarial attack is to make the power system big model output wrong results and create composable malicious components (such as adversarial embedding vectors) in the joint embedding space Z of the power system big model. These malicious components can be used as inputs of the big model. In the joint embedding space, typical harmful prompts are divided into two different parts: the common text instruction feature h t g and image features containing malicious perturbations These embeddings are collectively input into a large power system model:

[0040]

[0041] The text instruction usually asks a vague question, such as "What safety hazards exist in the current scenario?" When the text instruction is combined with the embedding vector generated by the image encoder from the adversarial image, it will be mapped to the wrong target embedding, causing the power system large model to identify and output incorrect safety hazard information.

[0042] In a possible implementation, constructing the adversarial image according to the adversarial embedding vector and the original image includes: iteratively optimizing the adversarial image malicious trigger according to a preset optimization objective function, and using the iteratively optimized adversarial image malicious trigger to modify the original image to obtain the adversarial image;

[0043] Among them, the optimization objective function is:

[0044]

[0045] in, is the original image modified by the adversarial image malicious trigger, L2 is the calculation h harm and The function of the gap between harm To counter the embedding vector, I φ is a pre-trained image encoder, is the original image, B is and Similarity constraints between .

[0046] Explanatory, the attack to obtain the adversarial embedding vector comes from malicious inputs of text input or image input, which can usually be discovered by automatic filters or detectors. Therefore, in order to ensure the concealment of the attack method, these malicious inputs are hidden in images that look beneficial. Based on this, conducting adversarial attacks on the joint embedding space requires constructing a specific class of adversarial images, and the embedding vectors of these adversarial images after being mapped to the joint embedding space are located in the danger zone defined by the adversarial embedding vector.

[0047] For a given adversarial embedding vector h harm , construct an adversarial image so that its corresponding embedding vector is consistent with the adversarial embedding vector h in the joint embedding space harm For example, for the original image x adv , initialization can use images such as random noise distribution images, white background images or unperturbed images to generate the initial embedding vector h adv =I(x adv ), and then optimize the adversarial image to minimize the embedding vector h adv and adversarial embedding vector h harm The gap between them finally obtains the optimal adversarial image.

[0048] For example, minimizing the embedding vector h adv and adversarial embedding vector h harm The gap between them can be obtained by minimizing the embedding vector h adv and adversarial embedding vector h harm The Euclidean distance between them is calculated as harm and The function of the gap between them can be selected as the Euclidean distance function. Based on this, the optimization purpose becomes to embed the vector h adv and adversarial embedding vector h harm The Euclidean distance between them is the loss, and this loss is minimized through back propagation to finally obtain the optimal adversarial image.

[0049] For example, regarding the adversarial image malicious trigger, a visual trigger or a trigger combining optical character recognition (OCR) text and vision can be selected to achieve an embedding vector h for guiding the generation of adversarial images. adv .

[0050] Among them, visual triggers act directly on the pixel values ​​or features of the image, and construct adversarial images by adding tiny perturbations or changing the image's color, texture and other attributes. The design of visual triggers is usually based on adversarial attack algorithms, such as the fast gradient sign method (FGSM) and projected gradient descent (PGD).

[0051] The trigger that combines optical character recognition text and vision not only considers the visual features of the image, but also combines OCR technology to identify and modify the text content in the image. By modifying the text information in the image and combining it with small visual perturbations, a more covert and effective adversarial image can be constructed.

[0052] Exemplarily, the pre-trained image encoder is pre-trained using the multimodal learning framework CLIP (Contrastive Language–Image Pre-training), so that the image encoder can learn rich visual concepts and language representations. Based on this design, adversarial images can be generated without directly accessing the power system model, only by accessing the general image encoder, which reduces the generation cost of adversarial images and can effectively trigger and guide the power system model to generate inappropriate text output.

[0053] See also Figure 3 and 4 In a possible implementation manner, the method for enhancing the safety of a large power system model further includes the following steps:

[0054] S4: Obtain a text description output by the power system large model based on the input sample, and generate a comparison image using a text-to-image generator according to the text description.

[0055] S5: Obtain the similarity between the input image and the comparison image in the input sample, and obtain the adversarial detection result of the input image according to the similarity between the input image and the comparison image.

[0056] Specifically, on the basis of enhancing the security of large power system models based on adversarial image training, an adversarial detection method for large power system models was designed, thereby enhancing the ability of large power system models to resist adversarial image attacks and further improving the security of large power system models.

[0057] By designing an adversarial image detection method based on a text-to-image generator, a comparison image is generated based on the text output by the large model of the power system using the text-to-image generator. Adversarial images are detected by comparing the embedding similarity between the input image and the comparison image in the feature space. This method does not require additional training and can achieve excellent results with zero samples, effectively reducing the computational cost and time of deploying defense mechanisms. It is suitable for real-time adversarial image detection and can play an important role in the security defense problem of the large model of the power system, further ensuring the safe and stable operation of the power system.

[0058] Explanatory, in this implementation, it is assumed that the attacker only has black-box access to the large model of the power system. In this setting, the attacker cannot understand the internal architecture, parameter configuration, or training strategy of the large model of the power system. At the same time, existing detection methods cannot obtain the specific details used in the process of constructing adversarial images. The core goal of the attacker is to carry out targeted attacks by generating text descriptions consistent with the preset target response under the adversarial constraints defined by the L-norm, while ensuring that the difference between the adversarial image and the original clean sample image is within an acceptable range. Accordingly, the task of the adversarial image detection method is to accurately identify adversarial images to maintain the security and robustness of the large model of the power system.

[0059] First, the characteristics of the adversarial image itself are analyzed. The adversarial attack is to generate an adversarial image based on the clean image by adding an adversarial perturbation δ, while ensuring that the perturbation amplitude is controlled within the preset limit ε. The purpose is to make the output text description inconsistent with the original image, thereby affecting the correct decision of the power system.

[0060] According to the characteristics of adversarial images, a text-to-image generator is designed y t Represents text output from a large power system model, It indicates that the new image generated according to the text output of the large model of the power system is the comparison image, and then the adversarial detection result of the input image is obtained according to the similarity between the input image and the comparison image. In the scenario of processing clean images, since the generated text can accurately reflect the image content, the input image and the comparison image will have a high similarity, while in the case of processing adversarial images, the similarity between the input image and the comparison image is usually low.

[0061] Exemplarily, the adversarial detection result of the input image obtained by the similarity between the input image and the comparison image includes: when the similarity between the input image and the comparison image is greater than a preset similarity threshold, the adversarial detection result of the input image is a non-adversarial image; when the similarity between the input image and the comparison image is not greater than a preset similarity threshold, the adversarial detection result of the input image is an adversarial image.

[0062] Explanatory, using pre-trained image encoders I θ (x)→z, evaluate the input image x i and compare images Then, an adversarial detector D(x)→[0,1] is introduced to detect whether the input image is an adversarial image:

[0063]

[0064] Wherein, τ represents a preset similarity threshold, and sim represents a similarity calculation function.

[0065] Explanatory, cross-modal power system models are similar to bag-of-words models, which may not be reliable in optimizing cross-modal similarity, so models that compare text-image similarity have challenges in handling positional relationships in sentences and verb usage changes. Therefore, we choose to calculate the input image x i and compare images The similarity between them is not directly compared with the input image x i and text output of large power system models t .

[0066] In a possible implementation, obtaining the similarity between the input image and the comparison image includes: traversing a plurality of preset pre-trained image encoders: obtaining the encoding vector of the input image and the encoding vector of the comparison image using the current pre-trained image encoder, and obtaining the similarity between the encoding vector of the input image and the encoding vector of the comparison image under the current pre-trained image encoder; obtaining the average value of the similarity between the encoding vector of the input image and the encoding vector of the comparison image under all pre-trained image encoders as the similarity between the input image and the comparison image.

[0067] Explanation: the plurality of pre-trained image encoders may be one pre-trained image encoder or at least two pre-trained image encoders. The at least two pre-trained image encoders are designed to realize a complementary strategy for integrating multiple image encoders, which can solve the problem of inaccurate similarity evaluation of a single image encoder.

[0068] The similarity is evaluated by calculating the average output of multiple image encoders, that is:

[0069]

[0070] Where N is the number of image encoders, s is the similarity between the input image and the comparison image, Represents different types of image encoders.

[0071] Explanatory,integrating the output results of different image encoders can further,ensure the accuracy of similarity evaluation and improve the,detection accuracy of adversarial images.

[0072] Explanatory, the structure of Variational Auto-Encoder (VAE) is used to provide a theoretical basis for the malicious sample detection method. In the structure of VAE, the way of reconstructing the error can determine whether the sample is within or outside the training distribution. VAE is divided into two structures, encoder and decoder, in which the encoder encodes the input data into a continuous latent space, and the decoder reconstructs the input from the latent space information. The malicious sample detection method is regarded as a special VAE.

[0073] Unlike typical VAEs, the designed detection method relies on a discrete, categorical latent space containing text outputs generated from images and text prompts. Ignoring the part of the text prompt words, the image-to-text output stage of the cross-modal power system large model acts as the encoder encoding stage, which maps high-dimensional visual data to a discrete latent space of text representation, which can be expressed as:

[0074] q φ (z|x) = Cat(z;π(x))

[0075] Among them, x represents the input image, z represents the potential text description, Cat represents the Categorical distribution, and π(x) represents the discrete latent variable distribution conditioned on the image input, with its parameter φ.

[0076] The text-to-image stage acts as a decoder, reconstructing visual data from the text description:

[0077] p θ (x|z)=Bernoulli(x;σ(z))

[0078] Here, σ(z) models the probability of generating a potential text description z and its parameter is θ.

[0079] The process of sampling the image description text using a non-zero Softmax function is similar to sampling from the latent space using the Gumbel-Softmax reparameterization technique in VAE. The Gumbel distribution is an extreme value distribution that is often used to simulate the maximum or minimum value of independent and identically distributed random variables.

[0080] Therefore, using the reconstruction error as the basis for judging whether the input data is affected by adversarial attacks is consistent with the method of using the reconstruction error to judge whether the sample deviates from the data distribution when using VAE. This indicates that the designed adversarial image detection method can detect adversarial attacks well.

[0081] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0082] See also Figure 5 In another embodiment of the present invention, a large-model safety enhancement system for a power system is provided, which can be used to implement the above-mentioned large-model safety enhancement method for a power system. Specifically, the large-model safety enhancement system for a power system includes a vector construction module, an image construction module and a training module.

[0083] Among them, the vector construction module is used to construct an adversarial embedding vector according to the adversarial attack target and the visual component features of the joint embedding space of the large model of the power system; the image construction module is used to construct an adversarial image based on the adversarial embedding vector and the original image; wherein the adversarial image and the original image meet the preset similarity constraint, and the embedding vector after the adversarial image is mapped to the joint embedding space is located in the danger zone defined by the adversarial embedding vector; the training module is used to match the adversarial image with the text prompt to obtain a training sample, and train the large model of the power system according to the training sample.

[0084] See also Figure 6 In a possible implementation, the power system large model security enhancement system further includes: an image generation module and an adversarial detection module.

[0085] Among them, the image generation module is used to obtain the text description output by the power system large model based on the input sample, and use the text to image generator to generate a comparison image according to the text description; the adversarial detection module is used to obtain the similarity between the input image and the comparison image in the input sample, and obtain the adversarial detection result of the input image according to the similarity between the input image and the comparison image.

[0086] All relevant contents of each step involved in the aforementioned embodiment of the power system large model security enhancement method can be referred to the functional description of the functional modules corresponding to the power system large model security enhancement system in the embodiment of the present invention, and will not be repeated here.

[0087] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0088] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the power system large model security enhancement method.

[0089] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the method for enhancing the safety of the large model of the power system in the above embodiment.

[0090] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0092] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for enhancing the safety of a large power system model, characterized in that: include: According to the adversarial attack target, an adversarial embedding vector is constructed by combining the visual component features of the joint embedding space of the large power system model; Constructing an adversarial image according to the adversarial embedding vector and the original image; wherein the adversarial image satisfies a preset similarity constraint with the original image, and the embedding vector after the adversarial image is mapped to the joint embedding space is located in a danger zone defined by the adversarial embedding vector; The adversarial images are matched with text prompts to obtain training samples, and the power system large model is trained based on the training samples.

2. The method for enhancing the safety of a large power system model according to claim 1, characterized in that: The constructing the adversarial image according to the adversarial embedding vector and the original image comprises: Iteratively optimizing the adversarial image malicious trigger according to a preset optimization objective function, and using the iteratively optimized adversarial image malicious trigger to modify the original image to obtain an adversarial image; Among them, the optimization objective function is: in, is the original image modified by the adversarial image malicious trigger, L2 is the calculation h harm and The function of the gap between harm To counter the embedding vector, I φ is a pre-trained image encoder, is the original image, B is and Similarity constraints between .

3. The method for enhancing the safety of a large power system model according to claim 2, characterized in that: The adversarial image malicious trigger is: a visual trigger, or a trigger combining optical character recognition text and vision; the pre-trained image encoder is pre-trained using the multimodal learning framework CLIP; the calculation h harm and The function of the distance between them is the Euclidean distance function; the original image is a random noise distribution image, a white background image or an undisturbed image.

4. The method for enhancing the safety of a large power system model according to claim 1, characterized in that: Also includes: Obtaining a text description output by the power system large model based on the input sample, and generating a comparison image using a text-to-image generator according to the text description; The similarity between the input image and the comparison image in the input sample is obtained, and the adversarial detection result of the input image is obtained according to the similarity between the input image and the comparison image.

5. The method for enhancing the safety of a large power system model according to claim 4, characterized in that: The obtaining of the similarity between the input image and the comparison image comprises: Traversing a plurality of preset pre-trained image encoders: using the current pre-trained image encoder to obtain the encoding vector of the input image and the encoding vector of the comparison image, and obtaining the similarity between the encoding vector of the input image and the encoding vector of the comparison image under the current pre-trained image encoder; The average value of the similarity between the encoding vector of the input image and the encoding vector of the comparison image under all pre-trained image encoders is obtained as the similarity between the input image and the comparison image.

6. The method for enhancing the safety of a large power system model according to claim 4, characterized in that: The adversarial detection result of the input image obtained by measuring the similarity between the input image and the comparison image includes: When the similarity between the input image and the comparison image is greater than a preset similarity threshold, the adversarial detection result of the input image is a non-adversarial image; When the similarity between the input image and the comparison image is not greater than a preset similarity threshold, the adversarial detection result of the input image is an adversarial image.

7. A large-scale model safety enhancement system for a power system, characterized in that: include: A vector construction module is used to construct an adversarial embedding vector based on the adversarial attack target and the visual component features of the joint embedding space of the large power system model; An image construction module is used to construct an adversarial image based on the adversarial embedding vector and the original image; wherein the adversarial image satisfies a preset similarity constraint with the original image, and the embedding vector of the adversarial image after being mapped to the joint embedding space is located in a danger zone defined by the adversarial embedding vector; The training module is used to match the adversarial images with the text prompts to obtain training samples, and train the power system large model based on the training samples.

8. The power system large model safety enhancement system according to claim 7, characterized in that: Also includes: An image generation module is used to obtain a text description output by the power system large model based on an input sample, and generate a comparison image using a text-to-image generator according to the text description; The adversarial detection module is used to obtain the similarity between the input image and the comparison image in the input sample, and obtain the adversarial detection result of the input image according to the similarity between the input image and the comparison image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for enhancing the safety of a large power system model as claimed in any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for enhancing the safety of a large power system model as claimed in any one of claims 1 to 6 are implemented.

Images