Edge confrontation sample detection method and system for edge cloud collaboration of large electric power model

Abstract

The invention discloses an edge confrontation sample detection method and system oriented to electric power large model edge cloud collaboration, and belongs to the technical field of electric power artificial intelligence security. The method comprises the following steps: introducing a general and power industry customized probe set, generating a cross-modal attention map through a lightweight large model after side quantization, and extracting hierarchical features by using a multi-layer multi-head attention mechanism; extracting statistical domain features and frequency domain features from the attention map, and splicing and normalizing to obtain a joint feature vector; and inputting the joint feature vector into a lightweight classifier, quickly distinguishing a clean sample from an adversarial sample, and triggering an edge-cloud cooperative defense strategy of refusing to answer, alarming or returning to the cloud. The method solves the problems of limited edge node resources and low confrontation detection efficiency, has the characteristics of low delay, low power consumption and high robustness, effectively improves the ability of an electric power artificial intelligence system to resist confrontation attacks, and is suitable for multiple electric power scenes such as power transmission line inspection and substation monitoring.

Description

Technical Field

[0001] This invention belongs to the field of power artificial intelligence security technology, specifically relating to an edge adversarial sample detection method and system for edge-cloud collaboration in power large-scale models. Background Technology

[0002] In recent years, with the rapid development of new power systems and the mature application of artificial intelligence (AI) technology in multimodal perception, prediction, and decision-making, the edge-cloud collaborative architecture for power AI has gradually become mainstream. This architecture organically combines centralized computing power in the cloud with near-source intelligence at the edge, supporting intelligent upgrades across multiple scenarios, from intelligent transmission line inspection and real-time substation monitoring to distribution network fault location, load forecasting, and scheduling optimization. The cloud-side system, relying on massive computing power, can achieve high-precision modeling and continuous optimization of complex power tasks, while the edge, facing the field, possesses advantages such as low latency and real-time response, thereby improving operation and maintenance efficiency and intelligence levels while ensuring the safe and stable operation of the power system. In this collaborative system, edge devices typically undertake preliminary tasks such as real-time data acquisition, feature extraction, and preliminary screening, while the cloud platform relies on powerful computing and storage resources to run multimodal power models, completing global reasoning and comprehensive analysis. While this layered design balances performance and efficiency, it faces new challenges in dealing with adversarial attacks. Adversarial examples are malicious inputs that implant minute perturbations into the original input data, thereby misleading the model to output incorrect results. Their potential harm is particularly serious in the power sector. In transmission line image recognition, adversarial examples may cause defect identification failures; in load forecasting, adversarial perturbations may interfere with the prediction model, leading to dispatching deviations and even grid security risks. Therefore, improving the adversarial robustness of power AI systems has become one of the key technical bottlenecks for the large-scale deployment of edge-cloud collaborative applications.

[0003] Existing adversarial detection methods are mostly based on detectors deeply coupled with the main task model, requiring complex neural network structures and substantial computational and storage resources. While these methods are feasible in cloud environments, they are difficult to deploy on low-power, resource-constrained edge nodes. If edge nodes lack adversarial detection capabilities, potential malicious samples can only be directly uploaded to the cloud for identification and filtering by cloud models. This approach not only increases network transmission overhead and latency but may also lead to a decrease in real-time response capabilities at the edge, thereby weakening the overall advantages of the edge-cloud collaborative architecture. Furthermore, large-scale data backhaul increases communication energy consumption and bandwidth pressure, which is detrimental to long-term operation in complex power grid scenarios.

[0004] Therefore, in the context of edge-cloud collaboration in power AI, there is an urgent need for a lightweight, low-power, deployable adversarial detection mechanism that can be deployed on edge nodes to balance detection performance and system response efficiency under resource constraints, thereby enhancing the security and robustness of large power models in practical applications. Summary of the Invention

[0005] The technical problem this invention aims to solve is to address the shortcomings of the existing technology by providing an edge adversarial sample detection method and system for edge-cloud collaboration in power large-scale models. This method identifies anomalous interference samples by analyzing the attention map and hierarchical features of the intermediate layers of the large model. A lightweight anomaly pattern detection model is trained based on joint feature vectors to determine whether the input is an adversarial sample. This addresses the technical problem of limited edge node resources and a lack of efficient adversarial detection capabilities in power artificial intelligence edge-cloud collaborative systems. It effectively identifies potential adversarial samples while ensuring low latency and low power consumption, thereby improving the security and robustness of power artificial intelligence systems.

[0006] The present invention adopts the following technical solution: An edge adversarial example detection method for edge-cloud collaboration in a large power model is characterized by the following steps: S1. Introduce a probe set containing general problems and power industry-customized problems. Input the image sample to be tested and the probe set into a lightweight large model after side quantization. Generate a cross-modal attention map through the lightweight large model and extract hierarchical distribution features from the cross-modal attention map using a multi-layer multi-head attention mechanism. S2. Extract statistical domain features and frequency domain features from the cross-modal attention map, wherein the statistical domain features include sparsity, entropy and centroid shift, and the frequency domain features include high-frequency energy ratio and spectral perturbation information extracted by discrete Fourier transform and discrete cosine transform. Concatenate the statistical domain features and frequency domain features to form a joint feature vector. S3. Input the normalized joint feature vector into the lightweight classifier. The lightweight classifier enables rapid differentiation between clean samples and adversarial samples. Based on the discrimination result, trigger edge-cloud collaborative defense strategies, including rejection strategies, alarm strategies, or cloud-based deep analysis strategies.

[0007] Preferably, the probe set is obtained through a heuristic strategy optimization. The heuristic strategy is based on the structural features of the sample and the known distribution pattern of the attack mode. A preset feature sensitivity function scores the response intensity of the initial probe set, and a streamlined probe set is obtained by screening and reconstructing it to amplify the abnormal response mode of the adversarial sample.

[0008] Preferably, the process of generating the cross-modal attention map is as follows: The lightweight large model generates cross-modal attention tensors at different levels and attention heads, and aggregates the outputs of all levels and all attention heads to obtain a complete cross-modal attention map; The extraction process of the hierarchical distribution features is as follows: through a multi-layer multi-head attention mechanism, shallow local patterns and low-level differences, deep abstract semantics and cross-modal dependencies are captured respectively, so as to achieve complete feature coverage from local to global.

[0009] Preferably, the cross-modal attention tensor is:

[0010] in, Representation layer index, Representation layer index, Indicates the first Layer One's attention is focused on something.

[0011] Preferably, the extraction process of the frequency domain features is as follows: The discrete Fourier transform and discrete cosine transform are performed on the cross-modal attention map to extract high-frequency energy ratio and spectral perturbation information. The local mutation and fragmentation features of adversarial examples are captured through the frequency domain features.

[0012] Preferably, the statistical domain features include one or more of sparsity, entropy, and centroid shift, which characterize the global distribution pattern against perturbations.

[0013] Preferably, the statistical domain feature includes the mean. ,variance skewness Kurtosis, mean ,variance skewness They are respectively:

[0014]

[0015]

[0016] in, These are the dimensions of the map. This is the normalized matrix of the attention map.

[0017] Preferably, the lightweight classifier is a quantized fully connected network or a shallow classifier, and the output of the lightweight classifier is:

[0018] in, y indicates that the input image sample to be tested is a clean sample, and y indicates that the input image sample to be tested is an adversarial sample.

[0019] Preferably, the execution logic of the edge-cloud collaborative defense strategy is as follows: The detection method runs as an additional module in the inference process on the edge device side, and is executed in parallel with the inference calculation of the edge model. When the detection results determine that the input image sample is an adversarial sample, the edge model prevents the generation of erroneous outputs and sends the suspected adversarial sample back to the cloud for in-depth analysis and confirmation by the cloud-based power big data model.

[0020] Secondly, embodiments of the present invention provide an edge adversarial sample detection system for edge-cloud collaboration in a large power model, comprising: The extraction module is used to introduce a probe set containing general problems and power industry-customized problems. The input image sample to be tested and the probe set are input into a lightweight large model after side quantization. A cross-modal attention tensor is generated and aggregated to obtain a hierarchical attention map set. Hierarchical features are extracted through a multi-layer multi-head attention mechanism. The construction module is used to extract statistical domain features and frequency domain features from the hierarchical attention map set, and concatenate the two types of features to form a joint feature vector; The discrimination module is used to input the joint feature vector into a lightweight classifier, and output a discrimination result of whether the input test image sample is a clean sample or an adversarial sample through the lightweight classifier. Based on the discrimination result, an edge-cloud collaborative defense strategy is executed. If the input test image sample is determined to be an adversarial sample, a rejection strategy or a strategy of sending back to the cloud for in-depth analysis is triggered.

[0021] Preferably, the probe set is obtained through a heuristic strategy optimization. The heuristic strategy is based on the structural features of the sample and the known distribution pattern of the attack mode. A preset feature sensitivity function scores the response intensity of the initial probe set, and a streamlined probe set is obtained by screening and reconstructing it to amplify the abnormal response mode of the adversarial sample.

[0022] Preferably, the process of generating the cross-modal attention map is as follows: The lightweight large model generates cross-modal attention tensors at different levels and attention heads, and aggregates the outputs of all levels and all attention heads to obtain a complete cross-modal attention map; The extraction process of the hierarchical distribution features is as follows: through a multi-layer multi-head attention mechanism, shallow local patterns and low-level differences, deep abstract semantics and cross-modal dependencies are captured respectively, so as to achieve complete feature coverage from local to global.

[0023] Preferably, the extraction process of the frequency domain features is as follows: The discrete Fourier transform and discrete cosine transform are performed on the cross-modal attention map to extract high-frequency energy ratio and spectral perturbation information. The local mutation and fragmentation features of adversarial examples are captured through the frequency domain features.

[0024] Preferably, the statistical domain features include one or more of sparsity, entropy, and centroid shift, which characterize the global distribution pattern against perturbations; the statistical domain features include the mean. ,variance skewness Kurtosis, mean ,variance skewness They are respectively:

[0025]

[0026]

[0027] in, These are the dimensions of the map. This is the normalized matrix of the attention map.

[0028] Preferably, the lightweight classifier is a quantized fully connected network or a shallow classifier, and the output of the lightweight classifier is:

[0029] in, y indicates that the input image sample to be tested is a clean sample, and y indicates that the input image sample to be tested is an adversarial sample.

[0030] Preferably, the execution logic of the edge-cloud collaborative defense strategy is as follows: The detection method runs as an additional module in the inference process on the edge device side, and is executed in parallel with the inference calculation of the edge model. When the detection results determine that the input image sample is an adversarial sample, the edge model prevents the generation of erroneous outputs and sends the suspected adversarial sample back to the cloud for in-depth analysis and confirmation by the cloud-based power big data model.

[0031] Thirdly, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the above-described edge adversarial sample detection method for edge-cloud collaboration in a large power model.

[0032] Fourthly, embodiments of the present invention provide a computer-readable storage medium including a computer program, which, when executed by a processor, implements the steps of the above-described edge adversarial sample detection method for edge-cloud collaboration in a large power model.

[0033] Fifthly, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the above-described edge adversarial sample detection method for edge-cloud collaboration in a large power model.

[0034] In a sixth aspect, embodiments of the present invention provide an electronic device, including a computer program, which, when executed by the electronic device, implements the steps of the above-described edge adversarial sample detection method for edge-cloud collaboration in a large power model.

[0035] Compared with the prior art, the present invention has at least the following beneficial effects: An edge-cloud collaborative method for detecting adversarial examples in a large-scale power model activates the model's cross-modal attention through a probe-driven mechanism, combining statistical and frequency domain feature representations, and ultimately achieving efficient discrimination through a lightweight classifier. By introducing a set of general and power industry-customized probes, natural language prompts guide the model to generate a cross-modal attention map, explicitly amplifying the abnormal response patterns of adversarial examples and solving the problem of traditional detection's insensitivity to disturbances. The joint modeling of statistical and frequency domain features not only characterizes the global distribution pattern through sparsity and entropy, but also captures local abrupt changes using discrete Fourier transform and discrete cosine transform, achieving complementary modeling of the global and local characteristics of adversarial disturbances and avoiding detection blind spots in a single feature space. The normalized feature vectors are input into the lightweight classifier, ensuring rapid sample differentiation with low computational and storage resources at edge nodes, balancing detection efficiency and real-time performance. The edge-cloud collaborative defense strategy covers modes such as rejection, alarm, and cloud transmission, preventing erroneous outputs from misleading business operations, and accumulating attack data through deep cloud analysis to support model iteration, achieving a dual improvement in security and robustness.

[0036] Furthermore, the heuristic strategy, based on sample structural features and attack pattern distribution patterns, pre-sets a feature sensitivity function to score the response intensity of initial probes. This accurately eliminates invalid or redundant probes, avoiding detection interference caused by the arbitrariness of traditional probe design. The optimized, streamlined probe set not only maintains stable responses under multiple attack modes but also amplifies anomalous response signals from adversarial examples, enhancing sensitivity to subtle perturbations. Simultaneously, the streamlined design reduces the computational load of probe-model interaction, lowers the operational load on edge nodes, and adapts to the low-power, resource-constrained operating environment of edge devices. Moreover, combined with the scenario adaptability of customized probes for the power industry, the optimized probe set improves the adaptability of the detection method in specific power scenarios such as transmission line inspection and substation monitoring, ensuring cross-task generalization effectiveness.

[0037] Furthermore, by generating cross-modal attention tensors at different levels and attention heads using a lightweight large model, and then aggregating the outputs of all levels and attention heads, this approach can comprehensively cover the interaction information between the input image sample and the probe, improving the completeness of the attention map compared to a single-layer attention mechanism. Hierarchical feature extraction, through a multi-layer, multi-head attention mechanism, captures shallow local patterns and low-level differences, as well as deep abstract semantics and cross-modal dependencies, achieving full-dimensional coverage from local features to global semantics, solving the problem that traditional feature extraction struggles to balance details and the overall picture. This full-dimensional feature characterization significantly enhances the ability to identify various types of adversarial attacks. Whether it's local perturbations or cross-modal feature shift attacks, effective detection can be achieved through the corresponding feature dimensions, providing rich feature support for subsequent statistical-frequency domain joint modeling and improving overall detection accuracy.

[0038] Furthermore, the tensor expression explicitly defines the meaning of the layer index, attention head index, and association weight matrix, quantifying the interaction strength between the input token and the probe token. This clearly reveals the dependencies between different modalities, avoiding the ambiguity of traditional attention feature generation. The cross-modal attention map obtained through this tensor aggregation accurately reflects the impact of adversarial perturbations on modal interactions, providing high-quality, quantifiable foundational data for subsequent feature extraction. The explicit mathematical expression allows for precise replication of the technical solution's implementation process, facilitating subsequent engineering deployment and optimization iterations. It also provides a clear basis for parameter adjustments in different scenarios, enhancing the flexibility and adaptability of the detection method and ensuring stable output of effective features in various power edge-cloud collaborative scenarios.

[0039] Furthermore, by performing Discrete Fourier Transform and Discrete Cosine Transform on the cross-modal attention map, features are mapped from the spatial domain to the frequency domain, transforming subtle perturbations in adversarial examples that are difficult to detect into significant spectral differences. The extracted high-frequency energy ratio and spectral perturbation information can accurately locate local texture anomalies and high-frequency noise caused by adversarial perturbations, solving the problem of traditional detection methods missing subtle perturbations. The introduction of frequency domain features complements statistical domain features, capturing local mutations through high-frequency information and characterizing the global distribution through statistical features, achieving a comprehensive characterization of adversarial perturbations. This extraction process has low computational complexity, is suitable for environments with limited edge node resources, and significantly improves the robustness of the detection method against multiple types of adversarial attacks without increasing the load.

[0040] Furthermore, statistical domain features such as sparsity, entropy, and centroid shift can accurately capture unnatural pixel distribution changes introduced by adversarial examples. Compared to single statistical indicators, multi-dimensional features can reflect global distribution anomalies from different perspectives, improving feature discriminative power. The global characterization capability of statistical domain features complements the local capture capability of frequency domain features, avoiding detection blind spots in a single feature space and ensuring effective response to different types of adversarial attacks (whether global perturbations or local attacks). In addition, the calculation process of statistical domain features is simple and efficient, requiring no complex neural network structure, and is suitable for the low-power, low-computing-resource operating conditions of edge nodes, ensuring both detection performance and system response efficiency.

[0041] Furthermore, this approach provides a quantitative basis for characterizing global distribution features, enhancing the rigor and reproducibility of the detection method. Explicit mathematical expressions quantify the distribution pattern of the attention map; the mean reflects the overall distribution center, the variance reflects the degree of dispersion, and skewness and kurtosis characterize the symmetry and steepness of the distribution, accurately capturing distribution shifts and abnormal fluctuations caused by adversarial examples. Quantified statistical features enhance the fusion effect with frequency domain features, giving the joint feature vector better discriminative power and providing accurate discrimination criteria for lightweight classifiers. The explicit mathematical expressions allow for precise replication of the technical solution's implementation process, facilitating subsequent model training and parameter optimization, while also providing theoretical support for the rationality of the detection results, enhancing the credibility and engineering application value of the patented technology.

[0042] Furthermore, the choice between quantized fully connected networks or shallow classifiers significantly reduces the number of model parameters and computational complexity. Compared to complex neural networks, it has lower storage overhead, faster inference speed, and can run in parallel on edge devices without affecting the real-time response requirements of the power system. The output expression clearly defines the discrimination criteria between clean samples and adversarial samples, providing a clear basis for interpreting the detection results and offering precise support for triggering subsequent edge-cloud collaborative defense strategies. While ensuring low computational load, the lightweight classifier achieves efficient discrimination through normalized joint feature vectors, avoiding redundant computations of traditional classifiers while ensuring discrimination accuracy, achieving an organic balance between low latency, low power consumption, and high accuracy.

[0043] Furthermore, the detection method, as an additional module, runs in parallel with the edge model, avoiding increased latency caused by serial computation and ensuring the real-time response capability of the power system. When adversarial examples are detected, the rejection strategy directly prevents erroneous output, avoiding misleading critical operations such as transmission line inspection and load dispatching; the feedback strategy uploads suspected samples to the cloud, utilizing the powerful computing capabilities of the cloud for in-depth analysis and confirmation, improving the reliability of the detection results. The cloud accumulates an attack pattern library based on the feedback samples, providing data support for the continuous optimization of the edge detection model, enabling iterative upgrades of the defense system, allowing the detection method to adapt to constantly changing attack patterns, and ensuring the long-term safe and stable operation of the power artificial intelligence system.

[0044] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0045] In summary, this invention achieves efficient and accurate adversarial example detection at the edge by employing probe-driven attention feature extraction, multi-domain feature fusion, and lightweight classifier design, thus addressing the security challenges faced by power AI systems in edge-cloud collaborative scenarios. It combines low power consumption, high precision, and strong adaptability, providing reliable AI security protection for critical power infrastructure.

[0046] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0047] Figure 1 This is a diagram illustrating the cross-modal attention map generation process of the present invention; Figure 2 This is a diagram illustrating the process of identifying adversarial examples in this invention. Figure 3 A schematic diagram of a computer device provided in an embodiment of the present invention; Figure 4 This is a block diagram of a chip provided according to an embodiment of the present invention.

[0048] Among them, 60. Computer equipment; 61. Processor; 62. Memory; 63. Computer program; 600. Electronic device; 610. Processing unit; 620. Storage unit; 6201. Random access memory unit; 6202. Cache memory unit; 6203. Read-only memory unit; 6204. Program / utility; 6205. Program module; 630. Bus; 640. Display unit; 650. Input / output interface; 660. Network adapter; 700. External device. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0051] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0052] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this invention generally indicates that the preceding and following objects have an "or" relationship.

[0053] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0054] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0055] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0056] This invention provides an edge adversarial sample detection method for power large-scale models with edge-cloud collaboration. First, based on the input image sample to be tested, an attention map is generated using a lightweight large-scale model after edge quantization, combined with general and power industry-customized probes. A multi-layer multi-head attention mechanism is then used to extract hierarchical distribution features. Second, statistical domain features and frequency domain features are extracted from the obtained attention map to simultaneously characterize the global distribution pattern and local mutation features of adversarial perturbations. Finally, the normalized feature vector is input into a lightweight classifier to quickly distinguish between clean and adversarial samples. During the online detection phase, high-risk samples can trigger rejection, alarms, or data transmission to the cloud for in-depth analysis, thereby constructing an efficient edge-cloud collaborative defense system. This method is low-overhead, easy to deploy, and can efficiently identify adversarial samples at the edge, improving the security and robustness of power large-scale models.

[0057] Please see Figure 1 This invention provides a method for detecting edge adversarial examples in a power large-scale model with edge-cloud collaboration, comprising the following steps: S1. Introduce general and power industry customized probes on the input image samples to be tested, generate cross-modal attention maps through the lightweight large model after edge quantization, and extract hierarchical features using a multi-layer multi-head attention mechanism. S101. In traditional adversarial detection, the input image sample is usually directly fed into the model for inference, lacking additional intervention methods to explicitly amplify the feature differences brought about by adversarial perturbations. This invention addresses this by using the input image sample... Based on this, a set of probe problems is designed and introduced: (1) Using human-constructed natural language prompts, the model is guided to generate cross-modal attention outputs relevant to the input content. The probes are divided into two categories: One type is the general question (such as "Is this image an outdoor scene?"). The second type of question is customized for the power industry (such as "Does the diagram include substation equipment?" or "Does it contain high-voltage switches?").

[0058] This probe set not only enhances the sensitivity to disturbance triggering, but also improves the adaptability of the method to industry tasks by combining the characteristics of power scenarios.

[0059] To avoid arbitrary probe design, this invention further employs a heuristic strategy to optimize the probe set, ensuring its effectiveness and stability across multiple attack modes. Specifically, the heuristic strategy first filters and reconstructs the initial probe set based on the structural features of the samples and the known distribution patterns of attack modes. In multi-class adversarial examples, some perturbations exhibit strong locality, while others manifest as cross-modal feature shifts. The heuristic algorithm scores the response intensity of probes using a preset feature sensitivity function, thereby quickly eliminating invalid or redundant probes and forming a more concise and targeted candidate set.

[0060] S102. Set up the corresponding probe questions, use a lightweight side-by-side large model for inference to generate a cross-modal attention map, and input the image sample to be tested. With probe set Input the lightweight large model after edge quantization together The model generates cross-modal attention tensors at different levels and with different attention heads: (2) in, Representation layer index, Representation layer index, Indicates the first Layer Each attention head contains a correlation weight matrix between the input token and the probe token. Each element of the matrix characterizes the interaction strength between the input feature fragment and the probe feature under the attention mechanism, explicitly revealing the dependencies between different modalities. By aggregating the outputs of all levels and all attention heads, a complete set of hierarchical attention maps is obtained: (3) This multi-layered and multi-headed attention aggregation approach significantly improves expressive power and discrimination performance compared to existing methods that rely solely on single-layer attention. Specifically, in terms of depth, shallow attention typically focuses on capturing local patterns and underlying differences in input features, while deep attention can reveal more abstract and semantic cross-modal dependencies. By fusing attention maps of different depths layer by layer, complete coverage from local features to global semantics can be achieved, ensuring a multi-layered characterization of the input and probe interaction patterns.

[0061] S2. After generating the attention map, further feature extraction is needed to capture potential anomalous patterns. Considering that adversarial examples often manifest as subtle perturbations, which may be difficult to distinguish from normal samples in the spatial domain, but can significantly amplify their differences in the frequency and statistical feature spaces, this step utilizes both frequency and statistical features to characterize the attention map, enhancing its robustness against adversarial perturbations.

[0062] S201. From the perspective of frequency domain features, counter-perturbations often manifest as high-frequency noise or local texture anomalies. By performing a Discrete Fourier Transform (DFT) on the attention map, it is mapped from the spatial domain to the frequency domain. Changes in the energy distribution in the spectrum often reveal potential perturbation signals. Therefore, the frequency domain feature extraction process is defined as follows: (4) in, The normalized matrix representing the attention map. These are the dimensions of the map. This represents the corresponding spectral component.

[0063] By calculating a lightweight classifier (5) A complete frequency domain representation is obtained, and indicators such as the proportion of low-frequency energy, the proportion of high-frequency energy, and spectral entropy are further extracted as frequency domain feature vectors. The low-frequency part corresponds to the overall structural information of the sample, while the high-frequency part contains more perturbation information.

[0064] S202. Global and local statistics of the attention map can effectively characterize the distribution characteristics of the perturbation. Adversarial examples often introduce unnatural pixel distribution changes; therefore, higher-order statistical features such as mean, variance, and skewness can enhance detection capabilities. The specific definitions are as follows: 1) Mean: (6) 2) Variance: (7) 3) Deviation: (8) The above statistical features can be used to capture the overall shift and abnormal fluctuations in the distribution of the attention map. Finally, the frequency domain features and statistical domain features are concatenated to form a joint feature vector: (9) in, It includes indicators such as low-frequency energy ratio, high-frequency energy ratio, and spectral entropy. It includes metrics such as mean, variance, skewness, and kurtosis. This joint feature ensures both sensitivity to perturbation signals and improves overall discriminative power. Employing both frequency domain and statistical domain features effectively distinguishes normal samples from adversarial samples. Frequency domain features enhance the model's ability to perceive local fine-grained perturbations, while statistical domain features provide a stable characterization of the global distribution. The combination of both significantly improves detection accuracy and robustness, avoiding blind spots that may exist in a single feature space.

[0065] To address the resource constraints of edge nodes in the edge-cloud collaborative system of power AI, this paper combines general-purpose and power industry-customized probes with input image samples to be tested. A lightweight large model after edge quantization generates a cross-modal attention map, and a multi-layer multi-head attention mechanism is used to extract hierarchical distribution features. This design amplifies the abnormal response patterns of adversarial examples, improves the detectability of adversarial perturbations, and thus achieves efficient anomaly perception capabilities at the edge.

[0066] This method utilizes statistical domain features such as sparsity, entropy, and centroid shift to characterize the global distribution pattern. Simultaneously, it combines Discrete Fourier Transform and Discrete Cosine Transform to extract high-frequency energy ratio and spectral perturbation information, capturing local mutations and fragmentation features in adversarial examples. This feature fusion method enables complementary modeling of the global and local characteristics of adversarial perturbations, enhancing the robustness and generalization ability of the detection method.

[0067] S3. After feature extraction, the obtained feature vectors are input into a lightweight classifier for discrimination. The classifier quickly distinguishes the categories of the input test image samples based on the difference patterns of the feature vectors in the statistical and frequency domains, thus achieving efficient identification of clean samples and adversarial samples. The adversarial sample identification process is as follows: Figure 2 As shown. Due to its lightweight design, this classifier has low computational complexity and fast inference speed, enabling it to operate stably under conditions of limited resources on edge devices, ensuring real-time performance and low power consumption in the detection process.

[0068] S301. After completing the frequency domain and statistical domain feature extraction, the joint feature vector will be... Input to lightweight classifier Output the final judgment result: (10) in, y=1 indicates that the input image sample to be tested is a clean sample, and y=1 indicates that the input image sample to be tested is an adversarial sample.

[0069] To ensure the detection module can operate efficiently on resource-constrained edge devices, this invention employs a lightweight classifier structure. This classifier can be a discriminator from a fully connected neural network or a shallow convolutional network. This lightweight design not only significantly reduces the number of model parameters and computational complexity but also guarantees extremely low storage overhead and computational latency without sacrificing discrimination accuracy.

[0070] S302. Online detection and edge-cloud collaboration. In actual deployment, the method of this invention runs as an additional module in the inference process on the edge device side. While the edge model completes inference calculations, the detection module performs parallel and rapid screening of the input image sample to be tested. Once the detection result indicates that the input image sample to be tested is a high-risk sample, i.e., it may be an adversarial sample, the system will immediately trigger the corresponding defense strategy: 1) Rejection strategy: If a sample is determined to have a high risk of attack, the edge model will prevent the generation of potentially attacked erroneous outputs to avoid misleading the business system.

[0071] 2) Backhaul strategy: Suspected adversarial samples are backhauled to the cloud for further analysis, verification and confirmation by a large model with stronger computing power and higher accuracy, to ensure the accuracy and reliability of the detection results.

[0072] By combining rapid edge screening with in-depth verification in the cloud, this invention establishes a low-latency, low-power, and highly robust edge-cloud collaborative defense mechanism. This mechanism not only ensures efficient interception of adversarial examples but also accumulates detection samples and attack patterns in the cloud, providing support for the continuous optimization and iteration of subsequent models.

[0073] After feature modeling, a fast edge-based discrimination mechanism based on a lightweight classifier is proposed. The normalized feature vectors are input into a quantized fully connected network or a shallow classifier to quickly distinguish between clean and adversarial samples. Combined with a collaborative edge-cloud defense system, the edge can trigger rejection, alarm, or data feedback strategies when detecting high-risk samples, allowing the cloud to perform in-depth analysis and iterative optimization. This mechanism combines low latency, low power consumption, and high robustness, effectively improving the ability of power AI systems to resist adversarial attacks in real-world scenarios.

[0074] In another embodiment of the present invention, an edge adversarial sample detection system for power large-scale model edge-cloud collaboration is provided. This system can be used to implement the above-mentioned edge adversarial sample detection method for power large-scale model edge-cloud collaboration. Specifically, the edge adversarial sample detection system for power large-scale model edge-cloud collaboration includes an extraction module, a construction module, and a discrimination module.

[0075] The extraction module is used to introduce a probe set containing general problems and power industry-customized problems, input the image sample to be tested and the probe set into a lightweight large model after side quantization, generate a cross-modal attention tensor and aggregate it to obtain a hierarchical attention map set, and extract hierarchical features through a multi-layer multi-head attention mechanism. The construction module is used to extract statistical domain features and frequency domain features from the hierarchical attention map set, and concatenate the two types of features to form a joint feature vector; The discrimination module is used to input the joint feature vector into a lightweight classifier, and output a discrimination result of whether the input test image sample is a clean sample or an adversarial sample through the lightweight classifier. Based on the discrimination result, an edge-cloud collaborative defense strategy is executed. If the input test image sample is determined to be an adversarial sample, a rejection strategy or a strategy of sending back to the cloud for in-depth analysis is triggered.

[0076] The probe set is obtained through a heuristic strategy. The heuristic strategy is based on the structural features of the samples and the known distribution patterns of attack patterns. A preset feature sensitivity function scores the response intensity of the initial probe set, and the probe set is filtered and reconstructed to amplify the abnormal response patterns of the adversarial samples.

[0077] The process of generating the cross-modal attention map is as follows: The lightweight large model generates cross-modal attention tensors at different levels and attention heads, and aggregates the outputs of all levels and all attention heads to obtain a complete cross-modal attention map; The extraction process of the hierarchical distribution features is as follows: through a multi-layer multi-head attention mechanism, shallow local patterns and low-level differences, deep abstract semantics and cross-modal dependencies are captured respectively, so as to achieve complete feature coverage from local to global.

[0078] The extraction process of the frequency domain features is as follows: The discrete Fourier transform and discrete cosine transform are performed on the cross-modal attention map to extract high-frequency energy ratio and spectral perturbation information. The local mutation and fragmentation features of adversarial examples are captured through the frequency domain features.

[0079] The statistical domain features include one or more of sparsity, entropy, and centroid shift, which characterize the global distribution pattern against perturbations; the statistical domain features include mean. ,variance skewness Kurtosis, mean ,variance skewness They are respectively:

[0080]

[0081]

[0082] in, These are the dimensions of the map. This is the normalized matrix of the attention map.

[0083] The lightweight classifier is a quantized fully connected network or a shallow classifier, and the output of the lightweight classifier is as follows:

[0084] in, y indicates that the input image sample to be tested is a clean sample, and y indicates that the input image sample to be tested is an adversarial sample.

[0085] The execution logic of the edge-cloud collaborative defense strategy is as follows: The detection method runs as an additional module in the inference process on the edge device side, and is executed in parallel with the inference calculation of the edge model. When the detection results determine that the input image sample is an adversarial sample, the edge model prevents the generation of erroneous outputs and sends the suspected adversarial sample back to the cloud for in-depth analysis and confirmation by the cloud-based power big data model.

[0086] This invention provides a terminal device comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, graphics processing units (GPUs), tensor processing units (TPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve corresponding method flows or corresponding functions. The processor described in this embodiment can be used for the operation of an edge adversarial sample detection method for power large-scale model edge-cloud collaboration, including: A probe set containing general and power industry-specific problems is introduced. Input image samples to be tested are fed into a lightweight large model after edge quantization. This lightweight large model generates a cross-modal attention map, and a multi-layer multi-head attention mechanism is used to extract hierarchical distribution features from the cross-modal attention map. Statistical domain features and frequency domain features are extracted from the cross-modal attention map. The statistical domain features include sparsity, entropy, and centroid shift, while the frequency domain features include high-frequency energy ratio and spectral perturbation information extracted through discrete Fourier transform and discrete cosine transform. The statistical domain features and frequency domain features are concatenated to form a joint feature vector. The normalized joint feature vector is input into a lightweight classifier, which enables rapid differentiation between clean and adversarial samples. Based on the discrimination result, an edge-cloud collaborative defense strategy is triggered, including a rejection strategy, an alarm strategy, or a strategy to transmit data back to the cloud for in-depth analysis.

[0087] Please see Figure 3The terminal device is a computer device. In this embodiment, the computer device 60 includes a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When executed by the processor 61, the computer program 63 implements the edge adversarial example detection method for power large-scale model edge-cloud collaboration in this embodiment. To avoid repetition, these details are not elaborated here. Alternatively, when executed by the processor 61, the computer program 63 implements the functions of each model / unit in the edge adversarial example detection system for power large-scale model edge-cloud collaboration in this embodiment. To avoid repetition, these details are not elaborated here.

[0088] Computer device 60 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. Computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will understand that... Figure 3 This is merely an example of computer device 60 and does not constitute a limitation on computer device 60. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0089] The processor 61 may be a Central Processing Unit (CPU), or other general-purpose processors, graphics processing units (GPUs), tensor processing units (TPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0090] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or RAM of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device 60.

[0091] Furthermore, the memory 62 may include both internal storage units of the computer device 60 and external storage devices. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0092] Please see Figure 4 The terminal device is an electronic device 600, which is manifested in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0093] The storage unit stores program code, which can be executed by the processing unit 610 to perform the steps described in the method section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 610 can perform actions such as... Figure 1 The steps are shown in the figure.

[0094] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0095] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0096] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the multiple bus structures.

[0097] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem). This communication can be performed via input / output interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network, wide area network, and / or public network, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0098] Example 4 This invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It is understood that the computer-readable storage medium here can include both built-in storage media in the terminal device and extended storage media supported by the terminal device; it can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor, which can be one or more computer programs (including program code). More specific examples of the computer-readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical fiber, portable compact disk read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0099] Computer-readable storage media also include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium can also be any readable medium other than a readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, radio frequency, etc., or any suitable combination thereof.

[0100] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0101] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the edge adversarial example detection method for edge-cloud collaboration in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor in the following steps: A probe set containing general and power industry-specific problems is introduced. Input image samples to be tested are fed into a lightweight large model after edge quantization. This lightweight large model generates a cross-modal attention map, and a multi-layer multi-head attention mechanism is used to extract hierarchical distribution features from the cross-modal attention map. Statistical domain features and frequency domain features are extracted from the cross-modal attention map. The statistical domain features include sparsity, entropy, and centroid shift, while the frequency domain features include high-frequency energy ratio and spectral perturbation information extracted through discrete Fourier transform and discrete cosine transform. The statistical domain features and frequency domain features are concatenated to form a joint feature vector. The normalized joint feature vector is input into a lightweight classifier, which enables rapid differentiation between clean and adversarial samples. Based on the discrimination result, an edge-cloud collaborative defense strategy is triggered, including a rejection strategy, an alarm strategy, or a strategy to transmit data back to the cloud for in-depth analysis.

[0102] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0103] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0104] 1. Simulation experimental data (1) Experimental environment Edge device: A power edge gateway (CPU: ARM Cortex-A53, memory: 2GB, storage: 16GB); Cloud device: Server (CPU: Intel Xeon 8375C, GPU: NVIDIA A100, Memory: 128GB); Dataset: Power scenario dataset (containing 100,000 data points including transmission line images, substation equipment data, and load forecasting data). Adversarial examples were generated using three typical attack algorithms: FGSM, PGD, and CW, with perturbation intensity ε∈[0.01,0.1]. Adversarial examples account for 30% of the dataset. Comparison methods: Traditional Deeply Coupled Detector (DCD), Single Statistical Domain Detection Method (SDD), and Single Frequency Domain Detection Method (FDD).

[0105] (2) Experimental results

[0106] (3) Data Conclusions Detection accuracy: The method of this invention improves by 13.4 percentage points compared with traditional DCD and by more than 15% compared with single feature methods, thanks to statistical-frequency domain joint modeling and probe-driven feature enhancement; Inference latency: The latency of the method of this invention is only 19.5% of that of traditional DCD, and 30% lower than that of the single feature method, demonstrating the advantages of lightweight design; Resource consumption: The number of model parameters is reduced by 90.2% compared to traditional DCD, and the power consumption is reduced by 74.3%, making it fully adaptable to resource-constrained environments at edge nodes.

[0107] Compared with existing methods that rely on intermediate layer anomaly detection or deeply coupled detectors, this invention has significant advantages in causal logic: The attention map is generated by using an edge quantization model, which effectively reduces the computation and storage costs and enables the detection module to run in real time at the edge nodes, thereby ensuring low latency and low power consumption. The probe design combines general problems with customized problems in the power scenario, which can amplify the difference signals when semantic alignment is violated, and improve the sensitivity to adversarial examples and cross-task generalization ability. The extraction of multi-layer, multi-head attention features can simultaneously capture abnormal responses of shallow semantics and deep details, significantly enhancing the ability to identify multiple types of attacks. Introducing joint feature modeling in the statistical and frequency domains can complement each other in terms of global distribution and local mutation patterns, thereby improving detection robustness; Using a lightweight classifier to complete the final discrimination not only ensures detection speed and edge deployability, but also reduces the need to transmit data to the cloud and reduces communication energy consumption; Finally, through edge-cloud collaborative defense, the edge can complete rapid initial screening, while the cloud can perform in-depth confirmation and continuous optimization, thus achieving a security protection system that balances real-time performance and high reliability.

[0108] In summary, this invention presents an edge adversarial sample detection method and system for power large-scale model edge-cloud collaboration. By introducing probe-driven attention mapping, multi-layer multi-head feature extraction, and joint modeling of the statistical and frequency domains into the power artificial intelligence edge-cloud collaboration system, a lightweight and deployable edge adversarial sample detection method is established. This invention effectively improves the ability of power artificial intelligence systems to resist adversarial attacks without increasing the computational burden of the main model, and has both engineering feasibility and practical application value.

[0109] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0110] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0111] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0112] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0115] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random-access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0116] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. An edge adversarial example detection method for edge-cloud collaboration in a large-scale power model, characterized in that, Includes the following steps: S1. Introduce a probe set containing general problems and power industry-customized problems. Input the image sample to be tested and the probe set into a lightweight large model after side quantization. Generate a cross-modal attention map through the lightweight large model and extract hierarchical distribution features from the cross-modal attention map using a multi-layer multi-head attention mechanism. S2. Extract statistical domain features and frequency domain features from the cross-modal attention map, wherein the statistical domain features include sparsity, entropy and centroid shift, and the frequency domain features include high-frequency energy ratio and spectral perturbation information extracted by discrete Fourier transform and discrete cosine transform. Concatenate the statistical domain features and frequency domain features to form a joint feature vector. S3. Input the normalized joint feature vector into the lightweight classifier. The lightweight classifier enables rapid differentiation between clean samples and adversarial samples. Based on the discrimination result, trigger edge-cloud collaborative defense strategies, including rejection strategies, alarm strategies, or cloud-based deep analysis strategies.

2. The edge adversarial example detection method for edge-cloud collaboration in a large power model according to claim 1, characterized in that, The probe set is obtained through a heuristic strategy. The heuristic strategy is based on the structural features of the samples and the known distribution patterns of attack patterns. A preset feature sensitivity function scores the response intensity of the initial probe set, and the probe set is filtered and reconstructed to amplify the abnormal response patterns of the adversarial samples.

3. The edge adversarial sample detection method for edge-cloud collaboration in a large power model according to claim 1, characterized in that, The process of generating the cross-modal attention map is as follows: The lightweight large model generates cross-modal attention tensors at different levels and attention heads, and aggregates the outputs of all levels and all attention heads to obtain a complete cross-modal attention map; The extraction process of the hierarchical distribution features is as follows: through a multi-layer multi-head attention mechanism, shallow local patterns and low-level differences, deep abstract semantics and cross-modal dependencies are captured respectively, so as to achieve complete feature coverage from local to global.

4. The edge adversarial sample detection method for edge-cloud collaboration in a large power model according to claim 3, characterized in that, The cross-modal attention tensor is: in, Representation layer index, Representation layer index, Indicates the first Layer One's attention is focused on something.

5. The edge adversarial sample detection method for edge-cloud collaboration in a large power model according to claim 1, characterized in that, The extraction process of the frequency domain features is as follows: The discrete Fourier transform and discrete cosine transform are performed on the cross-modal attention map to extract high-frequency energy ratio and spectral perturbation information. The local mutation and fragmentation features of adversarial examples are captured through the frequency domain features.

6. The edge adversarial sample detection method for edge-cloud collaboration in a large power model according to claim 1, characterized in that, The statistical domain features include one or more of sparsity, entropy, and centroid offset, which characterize the global distribution pattern against perturbations.

7. The edge adversarial sample detection method for edge-cloud collaboration in a large power model according to claim 6, characterized in that, The statistical domain features include the mean. ,variance skewness Kurtosis, mean ,variance skewness They are respectively: in, These are the dimensions of the map. This is the normalized matrix of the attention map.

8. The edge adversarial sample detection method for edge-cloud collaboration in a large power model according to claim 1, characterized in that, The lightweight classifier is a quantized fully connected network or a shallow classifier, and the output of the lightweight classifier is as follows: in, y indicates that the input image sample to be tested is a clean sample, and y indicates that the input image sample to be tested is an adversarial sample.

9. The edge adversarial sample detection method for edge-cloud collaboration in a large power model according to claim 1, characterized in that, The execution logic of the edge-cloud collaborative defense strategy is as follows: The detection method runs as an additional module in the inference process on the edge device side, and is executed in parallel with the inference calculation of the edge model. When the detection results determine that the input image sample is an adversarial sample, the edge model prevents the generation of erroneous outputs and sends the suspected adversarial sample back to the cloud for in-depth analysis and confirmation by the cloud-based power big data model.

10. An edge adversarial sample detection system for edge-cloud collaboration in a large-scale power model, characterized in that, include: The extraction module is used to introduce a probe set containing general problems and power industry-customized problems. The input image sample to be tested and the probe set are input into a lightweight large model after side quantization. A cross-modal attention tensor is generated and aggregated to obtain a hierarchical attention map set. Hierarchical features are extracted through a multi-layer multi-head attention mechanism. The construction module is used to extract statistical domain features and frequency domain features from the hierarchical attention map set, and concatenate the two types of features to form a joint feature vector; The discrimination module is used to input the joint feature vector into a lightweight classifier, and output a discrimination result of whether the input test image sample is a clean sample or an adversarial sample through the lightweight classifier. Based on the discrimination result, an edge-cloud collaborative defense strategy is executed. If the input test image sample is determined to be an adversarial sample, a rejection strategy or a strategy of sending back to the cloud for in-depth analysis is triggered.

11. The edge adversarial sample detection system for edge-cloud collaboration in a large power model according to claim 10, characterized in that, The probe set is obtained through a heuristic strategy. The heuristic strategy is based on the structural features of the samples and the known distribution patterns of attack patterns. A preset feature sensitivity function scores the response intensity of the initial probe set, and the probe set is filtered and reconstructed to amplify the abnormal response patterns of the adversarial samples.

12. The edge adversarial sample detection system for edge-cloud collaboration in a large power model according to claim 10, characterized in that, The process of generating the cross-modal attention map is as follows: The lightweight large model generates cross-modal attention tensors at different levels and attention heads, and aggregates the outputs of all levels and all attention heads to obtain a complete cross-modal attention map; The extraction process of the hierarchical distribution features is as follows: through a multi-layer multi-head attention mechanism, shallow local patterns and low-level differences, deep abstract semantics and cross-modal dependencies are captured respectively, so as to achieve complete feature coverage from local to global.

13. The edge adversarial sample detection system for edge-cloud collaboration in a large power model according to claim 10, characterized in that, The extraction process of the frequency domain features is as follows: The discrete Fourier transform and discrete cosine transform are performed on the cross-modal attention map to extract high-frequency energy ratio and spectral perturbation information. The local mutation and fragmentation features of adversarial examples are captured through the frequency domain features.

14. The edge adversarial sample detection system for edge-cloud collaboration in a large power model according to claim 10, characterized in that, The statistical domain features include one or more of sparsity, entropy, and centroid shift, which characterize the global distribution pattern against perturbations; the statistical domain features include mean. ,variance skewness Kurtosis, mean ,variance skewness They are respectively: in, These are the dimensions of the map. This is the normalized matrix of the attention map.

15. The edge adversarial sample detection system for edge-cloud collaboration in a large power model according to claim 1, characterized in that, The lightweight classifier is a quantized fully connected network or a shallow classifier, and the output of the lightweight classifier is as follows: in, y indicates that the input image sample to be tested is a clean sample, and y indicates that the input image sample to be tested is an adversarial sample.

16. The edge adversarial sample detection system for edge-cloud collaboration in a large power model according to claim 1, characterized in that, The execution logic of the edge-cloud collaborative defense strategy is as follows: The detection method runs as an additional module in the inference process on the edge device side, and is executed in parallel with the inference calculation of the edge model. When the detection results determine that the input image sample is an adversarial sample, the edge model prevents the generation of erroneous outputs and sends the suspected adversarial sample back to the cloud for in-depth analysis and confirmation by the cloud-based power big data model.

17. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform the method of any one of claims 1 to 9.

18. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including steps for performing the method of any one of claims 1 to 9.

Images