Industrial vision adversarial sample defense method and system based on physical simulation, electronic device and storage medium

Abstract

The application provides an industrial vision adversarial sample defense method and system based on physical simulation, an electronic device and a storage medium, and belongs to the technical field of general image data processing or generation, and comprises the following steps: obtaining original image data; generating a physical simulation image through a physical simulation engine based on material surface visual information in the original image data; iteratively generating an adversarial sample through a double-channel generative adversarial network generator according to the physical simulation image; performing adversarial training through a double-channel generative adversarial network discriminator based on a normal sample set and the adversarial sample set, and calculating a discriminator loss; calculating a generator loss based on the physical simulation image according to the discriminator loss, iteratively generating the adversarial sample, and outputting a target adversarial sample until a generator loss difference value is less than a preset value, so that the defense of the industrial vision adversarial sample is realized. The application can enhance the robustness and reliability of the industrial vision system by simulating the physical characteristics of the real world.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing or generation in general, and in particular to a physical simulation based industrial vision adversarial sample defense method and system, an electronic device and a storage medium. BACKGROUND

[0002] In the field of artificial intelligence and machine learning, adversarial attacks can easily deceive machine learning models and make them output incorrect or misleading predictions through carefully designed adversarial examples without attracting human attention. This attack method not only challenges the robustness and reliability of machine learning models, but also poses a potential threat to the widespread application of artificial intelligence technology.

[0003] In modern production and manufacturing, industrial vision is widely used, but the threat of adversarial samples is particularly serious. The robustness and reliability of industrial vision systems are extremely high in industrial environments, and any incorrect judgment may lead to production accidents, economic losses, and even safety hazards. Traditional adversarial sample defense methods often have limited effect in industrial vision scenarios, are only designed for adversarial samples in the digital domain, and are difficult to cope with complex attacks in the physical world. They may also introduce additional computational overhead, affecting the real-time performance of industrial vision systems. SUMMARY

[0004] To solve the above problems, the present application provides a physical simulation based industrial vision adversarial sample defense method, system, electronic device and storage medium, which simulates the physical properties of the real world to enhance the robustness and reliability of the industrial vision system.

[0005] The present application provides a physical simulation based industrial vision adversarial sample defense method, comprising:

[0006] Obtain the original image data of the target industrial scene, based on the material surface visual information in the original image data, simulate the optical properties of the material surface in the target industrial scene through a physical simulation engine, and generate a physical simulation image;

[0007] According to the physical simulation image, iteratively generate adversarial samples through a dual-channel generative adversarial network generator;

[0008] Based on a normal sample set composed of original images and an adversarial sample set composed of adversarial samples, perform adversarial training through a dual-channel generative adversarial network discriminator, calculate the discriminator loss, and update the discriminator parameters using the gradient descent method;

[0009] According to the discriminator loss, a generator loss is calculated based on the physical simulation image, a generator loss difference is obtained based on a current generator loss and a previous generator loss, when the generator loss difference is not less than a preset value, generator parameters are updated using a gradient descent method, and the generation of the adversarial sample is iterated, until the generator loss difference is less than the preset value, and the target adversarial sample is output, so as to realize the defense of the industrial vision adversarial sample.

[0010] Further, the material surface visual information in the original image data includes a material parameter and an illumination parameter, the material parameter includes a diffuse reflection coefficient, a specular reflection coefficient and a glossiness, and the illumination parameter includes a light source intensity, an incident angle and an observation angle.

[0011] Further, the generation formula of the pixel value of the physical simulation image is as follows:

[0012]

[0013] In the formula, represents the pixel value of the physical simulation image at the pixel coordinate , represents the diffuse reflection coefficient, represents the light source intensity, represents the incident angle, represents the cosine value of the incident angle, represents the specular reflection coefficient, represents the observation angle, represents the cosine value of the observation angle, represents the glossiness, represents the material disturbance factor.

[0014] Further, the calculation formula of the material disturbance factor is as follows:

[0015]

[0016] In the formula, represents the original image, represents Gaussian noise with a mean value of 0 and a variance of , represents the disturbance intensity, represents the gradient of the original image.

[0017] Further, the iteration generation of the adversarial sample based on the physical simulation image through the dual-channel generative adversarial network generator includes generating the adversarial sample through the following formula:

[0018]

[0019] In the formula, represents the first Adversarial examples generated in the next iteration; This represents the physical simulation image, which serves as the base image for the adversarial example; Indicates the magnitude of the disturbance; Representing the hyperbolic tangent function, it maps the perturbation value output by the dual-channel generative adversarial network generator to... ; This indicates a dual-channel generative adversarial network generator. With the physical simulation image and the Random noise in the next iteration For input, use the first Generator parameters for the next iteration Output disturbance value.

[0020] Furthermore, the set of normal samples composed of original images and the set of adversarial samples composed of adversarial samples are used for adversarial training through a dual-channel generative adversarial network discriminator. The discriminator loss is calculated as follows:

[0021] The first channel discriminator of the dual-channel generative adversarial network discriminator processes the normal sample set to obtain the expected loss of the normal sample set;

[0022] The second channel discriminator of the dual-channel generative adversarial network discriminator processes the adversarial sample set to obtain the expected loss of the adversarial sample set;

[0023] The discriminator loss is obtained by adding the expected loss of the normal sample set to the expected loss of the adversarial sample set.

[0024] Furthermore, the formula for calculating the discriminator loss is as follows:

[0025]

[0026] In the formula, Indicates the first Discriminator loss in the next iteration, Indicates normal samples Expectations This represents the normal sample distribution. This indicates that the first channel discriminator distinguishes normal samples. The negative logarithmic loss, This indicates that the first channel discriminator distinguishes normal samples. The output probability, Indicates the first Discriminator parameters for the next iteration Indicates the first Adversarial examples in the next iteration Expectations Indicates the first Adversarial examples in the next iteration The distribution, This indicates that the second channel discriminator is for the first... Negative log loss for adversarial examples in the next iteration; This indicates that the second channel discriminator is for the first... The output probability of the adversarial example in the next iteration.

[0027] Furthermore, the formula for calculating the generator loss is as follows:

[0028]

[0029] In the formula, Indicates the first The generator loss in the next iteration, Indicates the first Adversarial examples in the next iteration Expectations This indicates that the second-channel discriminator is effective against adversarial examples. The classification probability is taken as the negative natural logarithm; Indicates the regularization weight. This represents the square of the L2 norm.

[0030] The present invention provides an industrial visual adversarial sample defense system based on physical simulation, comprising a first processing module, a second processing module, a third processing module, and a fourth processing module;

[0031] The first processing module is used to acquire the original image data of the target industrial scene, and based on the visual information of the material surface in the original image data, simulate the optical properties of the material surface in the target industrial scene through a physical simulation engine to generate a physical simulation image.

[0032] The second processing module is used to iteratively generate adversarial examples based on the physical simulation image using a dual-channel generative adversarial network generator.

[0033] The third processing module is used to perform adversarial training on a dual-channel generative adversarial network discriminator based on a set of normal samples composed of the original images and a set of adversarial samples composed of adversarial samples, calculate the discriminator loss, and update the discriminator parameters using the gradient descent method.

[0034] The fourth processing module is used to calculate the generator loss based on the discriminator loss and the physical simulation image, obtain the generator loss difference based on the current generator loss and the previous generator loss, and update the generator parameters using the gradient descent method when the generator loss difference is not less than a preset value, continue to iterate and generate adversarial examples until the generator loss difference is less than the preset value, and output the target adversarial example to achieve defense against industrial visual adversarial examples.

[0035] The electronic device provided by the present application comprises a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to realize the steps of the industrial visual adversarial sample defense method based on physical simulation.

[0036] The storage medium provided by the present application has a computer program stored thereon, and the computer program is executed by a processor to realize the steps of the industrial visual adversarial sample defense method based on physical simulation.

[0037] The present application provides a kind of based on physical simulation of industrial visual adversarial sample defense method, system, electronic equipment and storage medium, by obtaining the original image data of target industrial scene, based on the material surface visual information in the original image data, through physical simulation engine, simulate the optical properties of material surface in the target industrial scene, generate physical simulation image;According to the physical simulation image, through double channel generation adversarial network generator iteratively generates adversarial sample;Based on the normal sample set of original image composition and the adversarial sample set of adversarial sample composition, through double channel generation adversarial network discriminator carries out adversarial training, calculates discriminator loss, and utilizes gradient descent method to update discriminator parameter;According to the discriminator loss, based on the physical simulation image, generator loss is calculated, based on current generator loss and previous generator loss generator loss difference value is obtained, when generator loss difference value is not less than preset value, generator parameter is updated using gradient descent method, continue iteratively generates adversarial sample, until generator loss difference value is less than preset value, output target adversarial sample, to realize the defense of industrial visual adversarial sample, through simulating the physical characteristics of real world, it can enhance the robustness and reliability of industrial visual system. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 It is the flowchart of the industrial visual adversarial sample defense method based on physical simulation of the embodiment of the present application.

[0039] Figure 2 It is the structure diagram of the industrial visual adversarial sample defense system based on physical simulation of the embodiment of the present application.

[0040] Mark explanation: 10, first processing module;20, second processing module;30, third processing module;40, fourth processing module. DETAILED DESCRIPTION

[0041] The following will be combined with specific embodiments and drawings Figures 1-2 Detailed description of the invention, so that those skilled in the art can more fully understand the purpose, features and effects of the present application.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. If there is a conflict between the definitions of terms in this disclosure and that understood by those of ordinary skill in the art, the definitions provided in this disclosure control.

[0043] The present application can play a greater role in the field of industrial vision by providing a physical simulation-based industrial vision adversarial sample defense method, system, electronic device and storage medium, which adopts deep integration of physical simulation technology and artificial intelligence, and provides more secure and reliable technical support for intelligent manufacturing and industrial automation.

[0044] Embodiment one

[0045] As a specific embodiment of the present application, the embodiment provides a physical simulation-based industrial vision adversarial sample defense method, which refers to Figure 1 , and the specific steps are as follows:

[0046] S100, obtaining original image data of a target industrial scene, simulating optical properties of a material surface in the target industrial scene based on material surface visual information in the original image data through a physical simulation engine, and generating a physical simulation image;

[0047] S200, iteratively generating an adversarial sample through a dual-channel generative adversarial network generator according to the physical simulation image;

[0048] S300, performing adversarial training through a dual-channel generative adversarial network discriminator based on a normal sample set composed of original images and an adversarial sample set composed of adversarial samples, calculating a discriminator loss, and updating discriminator parameters using a gradient descent method;

[0049] S400, calculating a generator loss based on the physical simulation image according to the discriminator loss, obtaining a generator loss difference value based on the current generator loss and the previous generator loss, updating generator parameters using a gradient descent method when the generator loss difference value is not less than a preset value, continuing to iteratively generate adversarial samples until the generator loss difference value is less than the preset value, and outputting a target adversarial sample to achieve defense against industrial vision adversarial samples.

[0050] The physical simulation-based industrial vision adversarial sample defense method provided in the embodiment simulates complex interaction between light and a material surface based on material surface visual information in an industrial scene through a physical simulation engine, which conforms to the laws of optical physics.

[0051] The industrial vision adversarial sample defense method based on physical simulation provided in the embodiment can finally generate the target adversarial sample which can simulate the real interference that may occur in the industrial scene, such as strong light, shadow, and material reflection change, so as to enhance the robustness of the industrial vision system. The realistic adversarial sample (i.e., the target adversarial sample) is used to enhance the training data set of the industrial vision model. By introducing the realistic adversarial sample in the training, the real interference is simulated, and the industrial vision model can learn how to keep stable output under the interference such as light change, so as to improve the defense ability to the real adversarial scene.

[0052] The industrial vision adversarial sample defense method based on physical simulation provided in the embodiment can finally generate the target adversarial sample which can simulate the real interference that may occur in the industrial scene, such as strong light, shadow, and material reflection change, so as to enhance the robustness of the industrial vision system. The realistic adversarial sample (i.e., the target adversarial sample) is used to enhance the training data set of the industrial vision model. By introducing the realistic adversarial sample in the training, the real interference is simulated, and the industrial vision model can learn how to keep stable output under the interference such as light change, so as to improve the defense ability to the real adversarial scene.

[0053] Specifically, in S100, first, the original image in the target industrial scene is acquired , and then the material surface visual information is extracted from the original image . The material surface visual information in the original image data includes material parameters and illumination parameters. The material parameters include diffuse reflection coefficient , specular reflection coefficient , and glossiness , and the illumination parameters include light source intensity , incident angle , and observation angle .

[0054] Further, the physical simulation engine simulates the reflection behavior of light on the material surface by calculating the diffuse reflection component and the specular reflection component. The diffuse reflection component is proportional to the cosine of the incident angle , and represents the uniform scattering intensity of light on the material surface in each direction. The uniform scattering effect of light on the material surface is calculated according to the light source intensity and the incident angle . The specular reflection component is related to the cosine of the observation angle and the power of the glossiness . The higher the glossiness , the more concentrated the highlight area, and the specular reflection characteristics of a smooth surface are simulated. The highlight effect is calculated according to the light source intensity , the observation angle , and the glossiness .

[0055] Optionally, the physical simulation engine is Blender Cycles.

[0056] Further, to enhance the fidelity of the physical simulation image, a material disturbance factor is introduced, the micro-inhomogeneity of the material surface is simulated through image gradient and Gaussian noise, which can truly reflect the light change and material characteristics in the industrial scene, and provides high-quality basic data for subsequent adversarial sample generation, thereby effectively simulating the optical characteristics in the real industrial environment.

[0057] Preferably, in the embodiment, the generation formula of the pixel value of the physical simulation image is as follows:

[0058]

[0059] In the formula, represents the pixel value of the physical simulation image at the pixel coordinate ; represents the diffuse reflection coefficient, reflecting the uniform scattering characteristics of the material surface under light; represents the light source intensity, affecting the brightness of the diffuse reflection component and the specular reflection component, simulating the light intensity in the industrial scene; represents the incident angle; represents the cosine value of the incident angle; represents the specular reflection coefficient, reflecting the gloss characteristics of the material surface; represents the observation angle; represents the cosine value of the observation angle; represents the glossiness; represents the material disturbance factor, simulating the micro-inhomogeneity of the material surface.

[0060] wherein, is calculated by the following formula:

[0061]

[0062] In the formula, represents the original image, represents the Gaussian noise with mean value 0 and variance , simulating the random disturbance of the material surface, enhancing the fidelity, generated by a random number generator; represents the disturbance intensity, controlling the amplitude of the Gaussian noise, determined according to the material characteristics (such as roughness) of the target industrial scene; represents the gradient of the original image, calculated by an image processing algorithm, and the image processing algorithm is optionally a Sobel operator.

[0063] By using a physical simulation engine to simulate the complex interaction between light and material surfaces in a real industrial environment, the generated physical simulation images not only conform to the laws of optical physics, but also reflect the real effects of lighting changes in industrial scenarios.

[0064] Specifically, in S200, the adversarial example is generated using the following formula:

[0065]

[0066] In the formula, Indicates the first Adversarial examples generated in the next iteration; The physical simulation image represents the base image for adversarial examples, providing realistic optical properties; It indicates the magnitude of the disturbance and controls the strength of the countermeasures against the disturbance; Representing the hyperbolic tangent function, it maps the perturbation value output by the dual-channel generative adversarial network generator to... To avoid excessive perturbation that could distort the generated adversarial examples; This indicates a dual-channel generative adversarial network generator. With the physical simulation image and the Random noise in the next iteration For input, use the first Generator parameters for the next iteration Output disturbance value.

[0067] In this embodiment, adversarial examples are iteratively generated using a dual-channel generative adversarial network generator based on the generated physical simulation image to enhance the robustness of industrial vision systems, such as industrial vision defect detection models, under complex lighting conditions.

[0068] The dual-channel generative adversarial network generator introduces random noise in each iteration, which follows a standard normal distribution, to increase the diversity of generated adversarial examples.

[0069] Furthermore, to ensure that the generated perturbation is numerically controllable, it is normalized by the hyperbolic tangent function, which maps the perturbation value output by the dual-channel generative adversarial network generator to the [-1,1] interval, thus avoiding excessive perturbation that could lead to distortion of the generated adversarial samples.

[0070] In this embodiment, using the physical simulation image as a basis ensures that the generated adversarial sample conforms to the optical characteristics of the real scene, thereby enhancing the physical realism of the adversarial sample.

[0071] The physical simulation-based industrial vision adversarial sample defense method provided by the embodiment generates diversified adversarial samples through the introduction of random noise and normalization processing, while retaining the optical characteristics of the physical simulation image, simulates the disturbance effect under complex lighting conditions, ensures controllable and realistic disturbance of the adversarial samples through disturbance amplitude control and normalization processing of the hyperbolic tangent function, avoids distortion, and significantly enhances the adaptability of the industrial vision system to lighting changes and adversarial disturbances.

[0072] Specifically, in S300, in order to improve the distinguishing ability of the dual-channel generative adversarial network discriminator for normal samples and adversarial samples, a batch of original images are obtained to form a normal sample set, and adversarial samples generated by the dual-channel generative adversarial network generator are obtained to form an adversarial sample set, and the dual-channel generative adversarial network discriminator is used for adversarial training.

[0073] The dual-channel generative adversarial network discriminator processes the normal sample set and the adversarial sample set through two channel discriminators respectively, and calculates a discriminator loss.

[0074] The first channel discriminator of the dual-channel generative adversarial network discriminator is responsible for processing the normal sample set, and outputs a probability value between 0 and 1 for each normal sample in the normal sample set, indicating the probability of the processed normal sample being determined as real. Then the natural logarithm of the probability is calculated, and the negative value is taken as the loss component of the first channel discriminator for processing each normal sample. Preferably, the average loss of the normal sample set is used to approximate the expected loss of the entire normal sample set, indicating the average classification performance of the normal sample set under the current discriminator parameters.

[0075] The second channel discriminator of the dual-channel generative adversarial network discriminator is responsible for processing the adversarial sample set, and outputs a probability value between 0 and 1 for each adversarial sample in the adversarial sample set, indicating the probability of the processed adversarial sample being determined as real. Further, 1 is subtracted from the probability and the natural logarithm is calculated, and the negative value is taken as the loss component of the second channel discriminator for processing each adversarial sample. Preferably, the average loss of the adversarial sample set is used to approximate the expected loss of the entire adversarial sample set, indicating the average classification performance of the adversarial sample set under the current discriminator parameters.

[0076] The expected loss of the entire normal sample set obtained by the first channel discriminator processing the normal sample set and the expected loss of the entire adversarial sample set obtained by the second channel discriminator processing the adversarial sample set are added together to obtain a total discriminator loss, which comprehensively measures the classification performance of the dual-channel discriminator and balances the distinguishing ability for normal samples and adversarial samples.

[0077] The purpose of calculating the expected loss of the first channel discriminator when processing the normal sample set is to optimize the discriminator's ability to classify real industrial images, establish a real data benchmark, support the balance of adversarial training, improve the accuracy of real image classification, ensure that the discriminator can distinguish between real images and adversarial samples generated by the generator, provide adversarial pressure to the generator, and enhance the overall robustness of the model. The expected loss of the adversarial sample set serves to provide a comparative benchmark, enhance the generator's optimization direction, maintain training stability, and ultimately support the generation of realistic adversarial samples, thereby improving the industrial vision system's ability to defend against interference.

[0078] Further, the discriminator loss is calculated based on the expected loss of the normal sample set and the expected impairment of the adversarial sample set. The formula for calculating the discriminator loss is:

[0079]

[0080] In the formula, Indicates the first The discriminator loss in the next iteration measures the discriminator's ability to distinguish between normal samples and adversarial samples. Indicates normal samples Expectations This represents the normal sample distribution; Indicates the first channel discriminator For normal samples The negative logarithmic loss; Indicates the first channel discriminator For normal samples The output probability; Indicates the first Discriminator parameters for the next iteration; Indicates the first Adversarial examples in the next iteration Expectations Indicates the first Adversarial examples in the next iteration Distribution; Indicates the second channel discriminator For the Negative log loss for adversarial examples in the next iteration; Indicates the second channel discriminator For the The output probability of the adversarial example in the next iteration.

[0081] Furthermore, after calculating the total discriminator loss, the discriminator parameters are updated using gradient descent to obtain the th... Discriminator parameters for the next iteration Gradient descent is a first-order optimization algorithm that iteratively updates parameters in the opposite direction of the function gradient to find local minima, widely used in machine learning and artificial intelligence.

[0082] The discriminator parameter update as part of the adversarial training ends with the overall iteration stop, which is controlled by the generator loss difference in the subsequent step. The discriminator loss is calculated and the discriminator parameters are updated to enable the discriminator to better distinguish between normal samples and adversarial samples, thereby providing feedback to the generator to generate more realistic adversarial samples.

[0083] In this embodiment, the first channel discriminator and the second channel discriminator are used to process the normal sample set and the adversarial sample set respectively, and the discriminator loss is calculated. The gradient descent method is used to update the discriminator parameters, thereby improving the adaptability to illumination changes and adversarial perturbations.

[0084] The purpose of updating the discriminator parameters using the gradient descent method is to optimize the discriminator's ability to distinguish between real data and generated data, which improves the classification performance of the discriminator, generates higher quality data for the generator, and maintains the adversarial balance of the generative adversarial network training. Optimizing the discriminator parameters can reduce the expected loss.

[0085] The normal samples and the adversarial samples jointly drive the dual-channel generative adversarial network discriminator to learn the difference between real and generated images, simulate the dynamic changes of light and material in industrial environments, and simulate the optical properties of real industrial environments through adversarial training to improve the defense capabilities of industrial vision systems.

[0086] The physical simulation-based industrial vision adversarial sample defense method of this embodiment uses the dual-channel generative adversarial network discriminator to perform adversarial training using the normal sample set and the adversarial sample set, processes real images and generated physical simulation images, optimizes the classification performance of the discriminator, and the dual-channel design enables the discriminator to effectively distinguish between normal samples and adversarial samples. The difference between the two is measured comprehensively, the dynamic changes of light and material in industrial environments are simulated, and the adaptability of the discriminator to complex lighting conditions and adversarial perturbations is improved, thereby enhancing the defense capabilities of the industrial vision system.

[0087] Specifically, in S400, the process of calculating the generator loss based on the discriminator loss and the physical simulation image, obtaining the generator loss difference based on the current generator loss and the previous generator loss, updating the generator parameters using gradient descent when the generator loss difference is not less than a preset value, and continuing to iterate and generate adversarial examples until the generator loss difference is less than the preset value, and then outputting the target adversarial example to achieve defense against industrial visual adversarial examples includes: calculating the expected loss of the adversarial example set obtained by processing the adversarial example set by the second channel discriminator; calculating the generator loss based on the physical simulation image; obtaining the generator loss difference based on the current generator loss and the previous generator loss; updating the generator parameters using gradient descent when the generator loss difference is not less than a preset value; continuing to iterate and generate adversarial examples until the generator loss difference is less than the preset value; and then outputting the target adversarial example to achieve defense against industrial visual adversarial examples.

[0088] In this embodiment, the expected loss of the adversarial sample set obtained by the second channel discriminator after processing the adversarial sample set is used as the adversarial loss, thereby better measuring the ability of the adversarial samples generated by the generator to deceive the discriminator.

[0089] Furthermore, to ensure the optical consistency between the adversarial example and the physical simulation image, a regularization loss is calculated: the square of the L2 norm of the adversarial example and the physical simulation image is multiplied by the regularization weight to obtain the regularization loss, ensuring that the adversarial example is close to the physical simulation image in visual and optical characteristics, simulating real lighting interference.

[0090] The adversarial loss and the regularization loss are added together to obtain the generator loss. Preferably, the generator loss is calculated using the following formula:

[0091]

[0092] In the formula, Indicates the first The generator loss in the next iteration; Indicates the first Adversarial examples in the next iteration Expectations Indicates the second channel discriminator Adversarial examples The classification probability is taken as the negative natural logarithm, which is used to measure the deceptive ability of the generator to generate samples. This represents the regularization weight, ensuring optical consistency between adversarial examples and physically simulated images, and preventing excessive perturbation; The L2 norm squared represents the pixel-level difference between the adversarial example and the physical simulation image, ensuring that the adversarial example is realistic and close to the physical simulation image.

[0093] The generator loss difference value is calculated by comparing the generator loss of the current iteration with the generator loss of the previous iteration. If the difference value is less than , it indicates that the generator loss converges, and the iteration is terminated. The current adversarial sample is output as the final realistic adversarial sample, i.e., the target adversarial sample. Otherwise, the iteration continues. If the iteration needs to continue, the generator parameters are updated using the gradient descent method based on the generator loss, and the generator parameters of the th iteration are obtained. The steps S200-S400 are repeated for iteration until the generator loss difference value is less than In other feasible embodiments, the generator loss difference value can also be adjusted according to the degree of realism of the final target adversarial sample to be output.

[0094] In this embodiment, the dual-channel generative adversarial network generator first generates an adversarial sample based on the physical simulation image and random noise, aiming to simulate the light changes or noise interference in the industrial scene (such as strong light highlights, shadows), so that the adversarial sample is initially adversarial, trying to deceive the dual-channel generative adversarial network discriminator, while retaining the optical characteristics of the physical simulation image; the dual-channel generative adversarial network generator optimizes the parameters according to the generator loss difference value fed back by the dual-channel generative adversarial network discriminator, i.e., the discrimination result of the dual-channel generative adversarial network discriminator on the previous adversarial sample, to continue generating a new adversarial sample, further improving the deception ability of the adversarial sample, making it difficult for the dual-channel generative adversarial network discriminator to distinguish the difference between the adversarial sample and the real sample, while maintaining similarity with the physical simulation image through regularization, ensuring that the adversarial sample is realistic and conforms to the optical law.

[0095] The adversarial sample generated by the generator for the first time may not be able to effectively deceive the discriminator, because the generator parameters have not been fully optimized, and the generated perturbation may be too simple or unrealistic, making it difficult to simulate complex industrial scene interference. The quality of the adversarial sample generated through multiple iterations is improved, which can simulate real interference in the industrial scene and enhance the defense capability of the visual system.

[0096] In this embodiment, the generator loss difference value is first calculated by comparing the generator loss of the current iteration with the generator loss of the previous iteration, and it is determined whether to continue iteration according to the generator loss difference value. If the iteration needs to continue, the generator parameters are updated using the gradient descent method, the generator parameters are updated through iteration, the generator is optimized, and a more realistic and more adversarial adversarial sample is generated to deceive the discriminator and maintain optical consistency with the physical simulation image, simulate light interference and material changes in the industrial vision scene, and improve the robustness and anti-interference ability of the industrial vision system.

[0097] The physical simulation-based industrial vision adversarial sample defense method of the embodiment combines an adversarial loss and a regularization loss, calculates a generator loss, and optimizes generator parameters through iteration, and a convergence judgment mechanism of a loss difference value can effectively control the iteration process, generate realistic adversarial samples, and a closed-loop optimization mechanism enables the generated adversarial samples to better simulate the light interference and material changes in a real industrial scene, further improving the anti-interference ability of the industrial vision system.

[0098] The physical simulation-based industrial vision adversarial sample defense method provided by the embodiment can solve the problems of insufficient robustness of an industrial vision system under complex lighting conditions, lack of physical authenticity of a traditional data enhancement method, difficulty in balancing realism and diversity in adversarial sample generation, and insufficient anti-interference ability of an industrial vision system in existing adversarial sample defense methods.

[0099] Embodiment Two

[0100] As another specific embodiment of the application, the embodiment provides a physical simulation-based industrial vision adversarial sample defense system, referring to Figure 2 , comprising a first processing module 10, a second processing module 20, a third processing module 30, and a fourth processing module 40.

[0101] The first processing module 10 is configured to obtain original image data of a target industrial scene, simulate the optical properties of a material surface in the target industrial scene based on the material surface visual information in the original image data through a physical simulation engine, and generate a physical simulation image.

[0102] The second processing module 20 is configured to generate adversarial samples iteratively through a double-channel generative adversarial network generator based on the physical simulation image.

[0103] The third processing module 30 is configured to perform adversarial training through a double-channel generative adversarial network discriminator based on a normal sample set composed of original images and an adversarial sample set composed of adversarial samples, calculate a discriminator loss, and update discriminator parameters using a gradient descent method.

[0104] The fourth processing module 40 is configured to calculate a generator loss based on the physical simulation image according to the discriminator loss, obtain a generator loss difference value based on the current generator loss and the previous generator loss, update generator parameters using a gradient descent method when the generator loss difference value is not less than a preset value, continue to iteratively generate adversarial samples, output a target adversarial sample until the generator loss difference value is less than the preset value, and achieve defense against industrial vision adversarial samples.

[0105] Embodiment Three

[0106] As another specific embodiment of the present application, the embodiment provides an electronic device comprising a memory, a processor and a computer program stored on the memory, the processor executing the computer program to implement the steps of the industrial visual adversarial sample defense method based on physical simulation:

[0107] S100, obtaining original image data of a target industrial scene, simulating optical properties of a material surface in the target industrial scene based on material surface visual information in the original image data through a physical simulation engine, and generating a physical simulation image;

[0108] S200, iteratively generating adversarial samples through a double-channel generative adversarial network generator according to the physical simulation image;

[0109] S300, performing adversarial training through a double-channel generative adversarial network discriminator based on a normal sample set composed of original images and an adversarial sample set composed of adversarial samples, calculating a discriminator loss, and updating discriminator parameters using a gradient descent method;

[0110] S400, calculating a generator loss based on the physical simulation image according to the discriminator loss, obtaining a generator loss difference value based on the current generator loss and the previous generator loss, updating generator parameters using a gradient descent method when the generator loss difference value is not less than a preset value, continuing to iteratively generate adversarial samples until the generator loss difference value is less than the preset value, and outputting target adversarial samples to achieve defense of industrial visual adversarial samples.

[0111] Embodiment four

[0112] As another specific embodiment of the present application, the embodiment provides a storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the industrial visual adversarial sample defense method based on physical simulation:

[0113] S100, obtaining original image data of a target industrial scene, simulating optical properties of a material surface in the target industrial scene based on material surface visual information in the original image data through a physical simulation engine, and generating a physical simulation image;

[0114] S200, iteratively generating adversarial samples through a double-channel generative adversarial network generator according to the physical simulation image;

[0115] S300, performing adversarial training through a double-channel generative adversarial network discriminator based on a normal sample set composed of original images and an adversarial sample set composed of adversarial samples, calculating a discriminator loss, and updating discriminator parameters using a gradient descent method;

[0116] S400. Based on the discriminator loss and the physical simulation image, calculate the generator loss. Obtain the generator loss difference between the current generator loss and the previous generator loss. When the generator loss difference is not less than a preset value, update the generator parameters using the gradient descent method and continue to iterate and generate adversarial examples until the generator loss difference is less than the preset value. Output the target adversarial example to achieve defense against industrial visual adversarial examples.

[0117] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A method for defending against adversarial examples in industrial vision based on physical simulation, characterized in that, The method includes: The original image data of the target industrial scene is acquired. Based on the visual information of the material surface in the original image data, the optical properties of the material surface in the target industrial scene are simulated through a physical simulation engine to generate a physical simulation image. Based on the physical simulation image, adversarial examples are iteratively generated using a dual-channel generative adversarial network generator. Based on a set of normal samples composed of original images and a set of adversarial samples composed of adversarial samples, a dual-channel generative adversarial network discriminator is trained adversarially, the discriminator loss is calculated, and the discriminator parameters are updated using the gradient descent method. Based on the discriminator loss, the generator loss is calculated based on the physical simulation image. The generator loss difference is obtained based on the current generator loss and the previous generator loss. When the generator loss difference is not less than a preset value, the generator parameters are updated using the gradient descent method, and adversarial examples are generated iteratively until the generator loss difference is less than the preset value. The target adversarial example is then output to achieve defense against industrial visual adversarial examples. The formula for generating the pixel values ​​of the physical simulation image is as follows: In the formula, This indicates that the physical simulation image is located at pixel coordinates. Pixel value at that location, Indicates the diffuse reflectance coefficient. Indicates the intensity of the light source. Indicates the angle of incidence. The value represents the cosine of the angle of incidence. Indicates the specular reflection coefficient. Indicates the observation angle. This represents the cosine value of the observed angle. Indicates glossiness. The material perturbation factor is represented by the following formula: In the formula, Represents the original image. This indicates that the mean is 0 and the variance is 0. Gaussian noise, Indicates the intensity of the disturbance. This represents the gradient of the original image.

2. The industrial visual adversarial example defense method based on physical simulation according to claim 1, characterized in that, The visual information of the material surface in the original image data includes material parameters and lighting parameters. The material parameters include diffuse reflection coefficient, specular reflection coefficient, and gloss. The lighting parameters include light source intensity, incident angle, and viewing angle.

3. The industrial visual adversarial example defense method based on physical simulation according to claim 1, characterized in that, The step of iteratively generating adversarial examples based on the physical simulation image using a dual-channel generative adversarial network generator includes generating the adversarial examples using the following formula: In the formula, Indicates the first Adversarial examples generated in the next iteration; This represents the physical simulation image, which serves as the base image for the adversarial example; Indicates the magnitude of the disturbance; Representing the hyperbolic tangent function, it maps the perturbation value output by the dual-channel generative adversarial network generator to... ; This indicates a dual-channel generative adversarial network generator. With the physical simulation image and the Random noise in the next iteration For input, use the first Generator parameters for the next iteration Output disturbance value.

4. The industrial visual adversarial example defense method based on physical simulation according to claim 1, characterized in that, The set of normal samples composed of original images and the set of adversarial samples composed of adversarial samples are used for adversarial training through a dual-channel generative adversarial network discriminant. The discriminant loss is calculated as follows: The first channel discriminator of the dual-channel generative adversarial network discriminator processes the normal sample set to obtain the expected loss of the normal sample set; The second channel discriminator of the dual-channel generative adversarial network discriminator processes the adversarial sample set to obtain the expected loss of the adversarial sample set; The discriminator loss is obtained by adding the expected loss of the normal sample set to the expected loss of the adversarial sample set.

5. The industrial visual adversarial sample defense method based on physical simulation according to claim 4, characterized in that, The formula for calculating the discriminator loss is as follows: In the formula, Indicates the first Discriminator loss in the next iteration Indicates normal samples Expectations This represents the normal sample distribution. This indicates that the first channel discriminator distinguishes normal samples. The negative logarithmic loss, This indicates that the first channel discriminator distinguishes normal samples. The output probability, Indicates the first Discriminator parameters for the next iteration Indicates the first Adversarial examples in the next iteration Expectations Indicates the first Adversarial examples in the next iteration The distribution, This indicates that the second channel discriminator is for the first... Negative log loss for adversarial examples in the next iteration; This indicates that the second channel discriminator is for the first... The output probability of the adversarial example in the next iteration.

6. A physical simulation-based industrial visual adversarial sample defense system, characterized in that, The system includes a first processing module (10), a second processing module (20), a third processing module (30), and a fourth processing module (40). The first processing module (10) is used to acquire the original image data of the target industrial scene, and based on the visual information of the material surface in the original image data, simulate the optical properties of the material surface in the target industrial scene through a physical simulation engine to generate a physical simulation image. The second processing module (20) is used to iteratively generate adversarial samples based on the physical simulation image using a dual-channel generative adversarial network generator; The third processing module (30) is used to perform adversarial training on a dual-channel generative adversarial network discriminator based on a set of normal samples composed of original images and a set of adversarial samples composed of adversarial samples, calculate the discriminator loss, and update the discriminator parameters using the gradient descent method. The fourth processing module (40) is used to calculate the generator loss based on the discriminator loss and the physical simulation image, obtain the generator loss difference based on the current generator loss and the previous generator loss, and update the generator parameters using the gradient descent method when the generator loss difference is not less than a preset value, continue to iterate and generate adversarial examples until the generator loss difference is less than the preset value, and output the target adversarial example to achieve defense against industrial visual adversarial examples. The formula for generating the pixel values ​​of the physical simulation image is as follows: In the formula, This indicates that the physical simulation image is located at pixel coordinates. Pixel value at that location, Indicates the diffuse reflectance coefficient. Indicates the intensity of the light source. Indicates the angle of incidence. The value represents the cosine of the angle of incidence. Indicates the specular reflection coefficient. Indicates the observation angle. This represents the cosine value of the observed angle. Indicates glossiness. The material perturbation factor is represented by the following formula: In the formula, Represents the original image. This indicates that the mean is 0 and the variance is 0. Gaussian noise, Indicates the intensity of the disturbance. This represents the gradient of the original image.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-5.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-5.

Images