Confrontation and defense method and device for driver state recognition and storage medium

Abstract

The invention discloses a confrontation and defense method and device for driver state recognition and a storage medium. The method comprises the steps that a confrontation and defense model connected with a state recognition model is constructed; collecting a plurality of first normal samples, and respectively generating a plurality of first adversarial samples corresponding to the first normal samples by using a plurality of different adversarial sample generation algorithms and based on the plurality of first normal samples; determining a plurality of first mixed sample strategies, and forming a plurality of first sample pairs according to the plurality of first mixed sample strategies and based on the plurality of first normal samples and the first adversarial samples corresponding to the first normal samples; inputting the first plurality of sample pairs into a confrontation and defense model, and training the confrontation and defense model; and performing confrontation detection and descrambling processing on the to-be-detected image by using the first confrontation defense network under the condition that training of the confrontation defense model is completed. Therefore, the confrontation and defense model can cope with different types of confrontation attacks, and the defense mobility of the confrontation and defense model is improved.

Description

Technical Field

[0001] This application relates to the fields of deep learning and safe driving technology, and in particular to an adversarial defense method, device and storage medium for driver state recognition. Background Technology

[0002] To ensure driver safety during driving, intelligent vehicles are often equipped with driver state recognition systems. These systems monitor the driver's state through image recognition and can issue warnings and make appropriate adjustments to the vehicle's driver assistance functions when the driver is distracted or fatigued. However, driver state recognition models are vulnerable to malicious attacks, primarily involving forged video images. Attackers can generate adversarial examples to attack the state recognition model and mislead it into making incorrect judgments, thereby affecting the vehicle's driver assistance functions and potentially causing traffic accidents.

[0003] In the field of deep learning, adversarial examples generally refer to special samples created by attackers using specific algorithms that are not perceptible to human senses but can still cause deep learning models to make mistakes. These adversarial examples can induce state recognition models to make incorrect decisions.

[0004] Currently, to address the aforementioned adversarial attacks, defense schemes such as adversarial training, defensive distillation, and adversarial attack detection have been proposed. However, these defense schemes all have certain shortcomings in the application of driver state recognition tasks. For example, some defense schemes have weak defense transferability and can only work against specific state recognition models. Once the state recognition model is changed, the construction and training of the defense network needs to be re-executed, which is cumbersome and the workload is related to the complexity of the replaced state recognition model. That is, there is a one-to-one correspondence between the defense scheme and the state recognition model. Furthermore, some defense schemes have low accuracy in identifying different types of adversarial attacks; in practical applications, the state recognition model cannot distinguish between real images and adversarial examples.

[0005] The publication number is CN111783551A, and the title is "An Adversarial Example Defense Method Based on Bayesian Convolutional Neural Networks." This method includes: selecting multiple traffic sign images as a training set and initial training set for the traffic signal recognition task of an autonomous driving image recognition system; constructing a Bayesian convolutional neural network model for the autonomous driving image recognition system and training the model to determine its parameters; setting perturbation values ​​and perturbation value increment steps to generate multiple adversarial examples; using the adversarial examples as training set data, and combining them with the initial training set to train the model and update its parameters; and improving the autonomous driving image recognition system based on the updated model parameters.

[0006] The publication number is CN113537463A, and the title is "A Method and Apparatus for Adversarial Example Defense Based on Data Perturbation." The method includes the following steps: adding pixels capable of interfering with vehicle recognition of road signs as data perturbations to the input samples to form defense samples; inputting the defense samples into a target neural network model for optimization, outputting trained data perturbations; adding the trained data perturbations to the samples to be recognized by the recognition neural network for recognition; the recognition neural network is a neural network model implanted in an autonomous vehicle.

[0007] There are currently no effective solutions to the technical problems of weak defense mobility and low accuracy in identifying different types of adversarial attacks in the existing technologies mentioned above. Summary of the Invention

[0008] The embodiments of this disclosure provide an adversarial defense method, apparatus, and storage medium for driver state recognition, so as to at least solve the technical problems of weak defense mobility and low recognition accuracy for different types of adversarial attacks in some existing defense schemes.

[0009] According to one aspect of the present disclosure, an adversarial defense method for driver state recognition is provided, comprising: constructing an adversarial defense model connected to a state recognition model, wherein the state recognition model is used to monitor the driver's driving state, and the adversarial defense model includes a first adversarial defense network; collecting multiple first normal samples, and generating multiple first adversarial samples corresponding to each first normal sample using multiple different adversarial sample generation algorithms based on the multiple first normal samples, wherein the adversarial attack type and / or adversarial attack intensity of each first adversarial sample is different; determining multiple first mixed sample strategies, and forming multiple first sample pairs according to the multiple first mixed sample strategies and based on the multiple first normal samples and the first adversarial samples corresponding to each first normal sample, wherein the first mixed sample strategies are used to instruct that the multiple first adversarial samples be mixed according to a pre-set adversarial attack type and adversarial attack intensity; inputting the first multiple sample pairs into the adversarial defense model and training the adversarial defense model; and, after the adversarial defense model has been trained, using the first adversarial defense network to perform adversarial detection and descrambling processing on the image to be detected.

[0010] According to another aspect of the present disclosure, a storage medium is also provided, the storage medium including a stored program, wherein, when the program is executed, a processor performs any of the methods described above.

[0011] According to another aspect of the present disclosure, an adversarial defense device for driver state recognition is also provided, comprising: a model building module for building an adversarial defense model connected to a state recognition model, wherein the state recognition model is used to monitor the driver's driving state, and the adversarial defense model includes a first adversarial defense network; an object sample first generation module for collecting multiple first normal samples, using multiple different adversarial sample generation algorithms and based on the multiple first normal samples to generate multiple first adversarial samples corresponding to the first normal samples, wherein the adversarial attack type and / or adversarial attack intensity of each first adversarial sample is different; a sample pair determination module for determining multiple first mixed sample strategies, forming multiple first sample pairs according to the multiple first mixed sample strategies and based on the multiple first normal samples and the first adversarial samples corresponding to each first normal sample, wherein the first mixed sample strategies are used to instruct that each first adversarial sample be mixed according to a pre-set adversarial attack type and adversarial attack intensity; a model training module for inputting the multiple first sample pairs into the adversarial defense model and training the adversarial defense model; and a model application module for performing adversarial detection and descrambling processing on the image to be detected using the first adversarial defense network after the adversarial defense model has been trained.

[0012] According to another aspect of the present disclosure, an adversarial defense device for driver state recognition is also provided, comprising: a processor; and a memory connected to the processor, configured to provide the processor with instructions for processing the following steps: constructing an adversarial defense model connected to a state recognition model, wherein the state recognition model is used to monitor the driver's driving state, and the adversarial defense model includes a first adversarial defense network; collecting multiple first normal samples, and generating multiple first adversarial samples corresponding to each first normal sample using multiple different adversarial sample generation algorithms and based on the multiple first normal samples, wherein the adversarial attack type and / or adversarial attack intensity of each first adversarial sample is different; determining multiple first mixed sample strategies, and forming multiple first sample pairs according to the multiple first mixed sample strategies and based on the multiple first normal samples and the first adversarial samples corresponding to each first normal sample, wherein the first mixed sample strategies are used to instruct the mixing of each first adversarial sample according to a pre-set adversarial attack type and adversarial attack intensity; inputting the first multiple sample pairs into the adversarial defense model and training the adversarial defense model; and, after the adversarial defense model has been trained, using the first adversarial defense network to perform adversarial detection and descrambling processing on the image to be detected.

[0013] This application discloses an adversarial defense method, apparatus, and storage medium for driver state recognition. First, a processor constructs an adversarial defense model connected to a state recognition model. Then, the processor collects multiple first normal samples and, using various first adversarial sample generation algorithms, generates multiple first adversarial samples corresponding to the first normal samples. Further, the processor forms multiple first sample pairs based on the multiple first normal samples and the corresponding first adversarial samples. The processor then inputs these multiple first sample pairs into the adversarial defense model and trains the model. Finally, after the adversarial defense model is trained, the processor uses the first adversarial defense network to perform adversarial detection and descrambling on the image to be detected.

[0014] As described above, this application generates multiple adversarial samples corresponding to each of the multiple first normal samples. Based on these multiple first normal samples and the corresponding adversarial samples, multiple first sample pairs with different adversarial attack types and / or intensities are formed. Therefore, the adversarial defense model constructed in this application can, while carefully designed to disrupt the malicious gradients and pixel manipulation of adversarial samples, ensure that the image contours can be correctly recognized by the classifier. Since the tasks of identifying and descrambling the target image and the application of the adversarial defense model in image denoising have commonalities, training the adversarial defense model using multiple first sample pairs enhances its ability to identify and descramble target images with various adversarial types, thereby accurately identifying different types of adversarial attacks.

[0015] Furthermore, since the type of adversarial attack is often unknown beforehand, training multiple adversarial defense models specifically for different attack types is impractical in real-world applications. Therefore, the best strategy is to train an adversarial defense model capable of handling multiple different types of adversarial attacks. Thus, in this application, training the adversarial defense model using samples of different adversarial attack types and / or attack intensities enables the model to cope with different types of adversarial attacks, thereby improving its defensive transferability.

[0016] This solves the technical problems of weak defense mobility and low accuracy in identifying different types of adversarial attacks in some existing defense schemes. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this application, illustrate exemplary embodiments of this disclosure and are used to explain this disclosure, but do not constitute an undue limitation of this disclosure. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of a computing device for implementing the method described in Embodiment 1 of this disclosure;

[0019] Figure 2 This is a schematic diagram of an adversarial defense system for driver state recognition according to Embodiment 1 of this disclosure;

[0020] Figure 3 This is a flowchart illustrating the adversarial defense method for driver state recognition according to Embodiment 1 of this disclosure;

[0021] Figure 4This is a schematic diagram of the adversarial defense model according to Embodiment 1 of this disclosure;

[0022] Figure 5 This is a schematic diagram of each of the first adversarial samples according to Embodiment 1 of this disclosure;

[0023] Figure 6 This is a schematic diagram illustrating the adversarial detection and descrambling of the image to be detected using a first adversarial defense network according to Embodiment 1 of this disclosure;

[0024] Figure 7 This is a schematic diagram of an anti-defense device for driver status recognition according to Embodiment 2 of this disclosure; Figure 8 This is a schematic diagram of an anti-defense device for driver status recognition according to Embodiment 3 of this disclosure. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] Example 1

[0028] According to this embodiment, an embodiment of an adversarial defense method for driver state recognition is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] The method embodiments provided in this example can be executed on mobile terminals, computer terminals, servers, or similar computing devices. Figure 1 A hardware block diagram of a computing device for implementing adversarial defense against driver state recognition is shown. Figure 1 As shown, a computing device may include one or more processors (processors may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), memory for storing data, transmission devices for communication functions, and input / output interfaces. The memory, transmission devices, and input / output interfaces are connected to the processor via a bus. In addition, it may also include a display, keyboard, and cursor control device connected to the input / output interfaces. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, a computing device may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0030] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element in a computing device. As involved in the embodiments of this disclosure, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0031] The memory can be used to store software programs and modules of application software, such as the program instruction / data storage device corresponding to the adversarial defense method for driver state recognition in the embodiments of this disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the aforementioned adversarial defense method for driver state recognition in the application. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the computing device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0032] The transmission device is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the computing device's communications provider. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0033] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows users to interact with the user interface of the computing device.

[0034] It should be noted here that, in some optional embodiments, the above... Figure 1 The computing device shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned computing devices.

[0035] Figure 2 This is a schematic diagram of the adversarial defense system for driver state recognition as described in this embodiment. (Refer to...) Figure 2 As shown, the system includes, but is not limited to, installations in a smart car, and includes: an image acquisition device 100 and a processor 200. The image acquisition device 100 is connected to the processor 200, and can acquire real-time images of the driver inside the smart car (i.e., the image to be detected), and then send the image to be detected to the processor 200. The processor 200 sends the image to be detected to a pre-trained adversarial defense model, and uses a first adversarial defense network in the adversarial defense model to perform adversarial detection and descrambling processing on the image to be detected.

[0036] It should be noted that the processor 200 in the system can be adapted to the hardware structure described above.

[0037] Under the aforementioned operating environment, according to the first aspect of this embodiment, an adversarial defense method for driver state recognition is provided, the method comprising: Figure 2 The processor 200 shown is implemented. Figure 3 A flowchart illustrating the method is shown below. (Refer to...) Figure 3 As shown, the method includes:

[0038] S302: Construct an adversarial defense model connected to the state recognition model, wherein the state recognition model is used to monitor the driver's driving state, and the adversarial defense model includes a first adversarial defense network.

[0039] S304: Collect multiple first normal samples, use multiple different adversarial sample generation algorithms and based on the multiple first normal samples, generate first adversarial samples corresponding to each first normal sample, wherein the adversarial attack type and / or adversarial attack intensity of each first adversarial sample are different.

[0040] S306: Determine multiple first mixed sample strategies, and based on multiple first normal samples and first adversarial samples corresponding to each first normal sample, form multiple first sample pairs, wherein the first mixed sample strategy is used to instruct that each first adversarial sample be mixed according to a pre-set adversarial attack type and adversarial attack strength.

[0041] S308: Input multiple first sample pairs into the adversarial defense model and train the adversarial defense model; and

[0042] S310: After the adversarial defense model has been trained, the first adversarial defense network is used to perform adversarial detection and descrambling on the image to be detected.

[0043] Specifically, firstly, the processor 200 constructs an adversarial defense model connected to the state recognition model (S302). Figure 4 This is a schematic diagram of an adversarial defense model according to an embodiment of this application. (Reference) Figure 4 As shown, the adversarial defense model includes a second adversarial defense network and a first adversarial defense network. The second adversarial defense network includes a first generation module and a first discrimination module, while the first adversarial defense network includes a second discrimination module and a second generation module.

[0044] The second generation module converts the first adversarial sample into a descrambled sample, which is similar to but not identical to the first normal sample. The first generation module converts the first normal sample into a noisy sample, which is similar to but not identical to the first adversarial sample. The second discrimination module identifies the first normal sample and the descrambled sample, and determines whether the first normal sample and the descrambled sample are real images. The first discrimination module identifies the noisy sample and the first adversarial sample, and determines whether the noise sample and the first adversarial sample are real images. The above-mentioned contents will be described in detail later, so they will not be repeated here.

[0045] Table 1 shows the structure and parameters of the state recognition model.

[0046]

[0047] Referring to Table 1, the state recognition model mainly consists of five convolutional blocks, each containing a two-dimensional convolutional layer, a batch normalization layer, an activation function, and in some cases, a dropout layer. Generally, the structure and parameters of the adversarial defense model corresponding to the state recognition model are also as described above, i.e., a multi-class deep neural network model composed of several two-dimensional convolutional blocks.

[0048] To match the scale of the state recognition model, a CycleGAN model with a structure that is compatible with it is constructed. Table 2 shows the structure and parameters of the first and second generation modules.

[0049]

[0050] Table 3 shows the structure and parameters of the first and second discrimination modules.

[0051]

[0052] Referring to Tables 2 and 3, the first and second generation modules in the adversarial defense model each consist of four forward convolutional blocks and two deconvolutional blocks, interspersed with N residual blocks, the number of which is determined according to actual needs. The first and second discriminant modules in the adversarial defense model each consist of five convolutional blocks. Each convolutional block in the adversarial defense model contains a normalization layer and a LeakyReLU activation function.

[0053] It is worth noting that this application trains the first generation module, the first discriminator module, the second discriminator module, and the second generation module during the training of the adversarial defense model. However, in practical applications, only the second discriminator module and the second generation module are used.

[0054] Then the processor 200 crawls multiple first normal samples, uses a variety of different adversarial sample generation algorithms, and generates a first adversarial sample corresponding to the first normal sample based on the multiple first normal samples (S304). Figure 5 This is a schematic diagram of various first adversarial examples according to embodiments of this application. (Reference) Figure 5 As shown, in this application, the StateFarm driver state recognition dataset is used as the first normal sample, and the normal sample is subjected to adversarial processing by methods such as Fast GradientSign Method, Jacobian Saliency Map Attack, DeepFool CW Attack (Carlini & Wagner Attack), and adding Gaussian noise, thereby generating multiple first adversarial samples corresponding to the first normal sample.

[0055] It is worth noting that the adversarial attack types and / or adversarial attack strengths of each first adversarial sample are different. For example, first adversarial sample 1 is a sample generated by adversarially processing first normal sample 1 using a deep deception algorithm, and the adversarial attack strength ε of first adversarial sample 1 is 0.1.

[0056] The first adversarial sample 2 is a sample generated by adversarially processing the first normal sample 2 using the CW attack algorithm, and the adversarial attack strength of the first adversarial sample 2 is ε = 0.1.

[0057] The first adversarial sample 3 is a sample generated by using the deep deception algorithm to perform adversarial processing on the first normal sample 3, and the adversarial attack strength of the first adversarial sample 3 is ε = 0.2.

[0058] Further, the processor 200 determines multiple first mixing sample strategies, and based on the multiple first mixing sample strategies and multiple first normal samples and the first adversarial samples corresponding to each first normal sample, forms multiple first sample pairs (S306). Specifically, when multiple first normal samples and multiple first adversarial samples corresponding to each first normal sample are determined, the multiple first adversarial samples can be mixed in different proportions based on the determined first mixing sample strategies. The first mixing sample strategies are used to instruct that the various first adversarial samples be mixed according to a pre-set adversarial attack type and adversarial attack strength.

[0059] The first mixed sample strategy may include, for example, mixed sample strategies A to E. Mixed sample strategy A includes the same number of first adversarial samples corresponding to the deep deception algorithm, the saliency graph adversarial algorithm, the fast gradient sign attack algorithm, the CW attack algorithm, and the Gaussian noise algorithm. Furthermore, the adversarial attack strength of the first adversarial samples corresponding to the deep deception algorithm, the saliency graph adversarial algorithm, and the fast gradient sign attack algorithm is ε = 0.2. The adversarial attack strength of the first adversarial samples corresponding to the CW attack algorithm and the Gaussian noise algorithm is ε = 0.1.

[0060] Hybrid sample strategy B includes an equal number of first adversarial samples corresponding to the fast gradient sign attack algorithm and the first adversarial samples corresponding to the Gaussian noise algorithm. The adversarial attack strength of the first adversarial samples corresponding to the fast gradient sign attack algorithm and the adversarial attack strength of the first adversarial samples corresponding to the Gaussian noise algorithm are both ε = 0.1.

[0061] The hybrid sample strategy C includes an equal number of first adversarial samples corresponding to the fast gradient sign attack algorithm and the first adversarial samples corresponding to the Gaussian noise algorithm. The adversarial attack strength ε = 0.1 for the first adversarial samples corresponding to the fast gradient sign attack algorithm and ε = 0.2 for the first adversarial samples corresponding to the Gaussian noise algorithm.

[0062] The mixed-sample strategy D includes an equal number of first adversarial samples corresponding to the fast gradient sign attack algorithm and the first adversarial samples corresponding to the Gaussian noise algorithm. The adversarial attack strength ε = 0.1 for the first adversarial samples corresponding to the fast gradient sign attack algorithm, and ε = 0.4 for the first adversarial samples corresponding to the Gaussian noise algorithm.

[0063] The mixed sample strategy E includes 80% of the first adversarial samples corresponding to the Gaussian noise algorithm, 5% of the first adversarial samples corresponding to the fast gradient sign attack algorithm, 5% of the first adversarial samples corresponding to the deep deception algorithm, 5% of the first adversarial samples corresponding to the saliency graph adversarial algorithm, and 5% of the first adversarial samples corresponding to the fast gradient sign attack algorithm.

[0064] The first mixed-sample strategy also includes training the adversarial defense model using only the first adversarial samples of the corresponding type. For example, when dealing with the first adversarial samples corresponding to the fast gradient symbol attack algorithm, the adversarial defense model is trained using only the first adversarial samples corresponding to that type.

[0065] The first hybrid sampling strategy also includes directly processing sample results using the first adversarial defense network.

[0066] Then, processor 200 inputs multiple first sample pairs into the adversarial defense model and trains the adversarial defense model (S308). Specifically, firstly, processor 200 inputs the first adversarial sample from the first sample pair into the second generation module and generates descrambled samples. Then, processor 200 inputs the first normal sample from the first sample pair into the first generation module and generates noisy samples. Further, processor 200 inputs the first normal sample and the descrambled sample together into the second discrimination module and outputs the first detection result and the second detection result. Then, processor 200 inputs the first adversarial sample and the noisy sample together into the first discrimination module and outputs the first discrimination result and the second discrimination result. Finally, processor 200 trains the adversarial defense model based on the descrambled sample, the noisy sample, the first detection result, the second detection result, the first discrimination result, and the second discrimination result, using the loss function corresponding to the adversarial defense model. The above will be described in detail later, so it will not be repeated here.

[0067] Finally, after the processor 200 has completed training the adversarial defense model, it receives the image to be detected sent by the image acquisition device 100 and sends the image to be detected to the adversarial defense model, whereby the first adversarial defense network in the adversarial defense model performs adversarial detection and descrambling on the image to be detected (S310). Furthermore, after using the adversarial defense model to perform adversarial detection and descrambling on the image to be detected, the processed image to be detected is further sent to the state recognition model, and the state recognition model detects the driver's real-time state.

[0068] As described in the background section, adversarial attack defense schemes such as adversarial training, defensive distillation, and adversarial attack detection have been proposed to address the problem. However, these schemes all have certain shortcomings in the application of driver state recognition tasks. For example, some defense schemes have weak transferability and can only work against specific state recognition models. Once the state recognition model is changed, the construction and training of the defense network needs to be re-executed, which is cumbersome and the workload is related to the complexity of the replaced state recognition model. That is, there is a one-to-one correspondence between the defense scheme and the state recognition model. Furthermore, some defense schemes have low accuracy in identifying different types of adversarial attacks; in practical applications, the state recognition model cannot distinguish between real images and adversarial examples.

[0069] In view of this, this application provides an adversarial defense method for driver state recognition. Referring to the above description, this application generates multiple adversarial samples corresponding to each of the multiple first normal samples. Based on these multiple first normal samples and the corresponding adversarial samples, multiple first sample pairs with different adversarial attack types and / or intensities are formed. Therefore, the adversarial defense model constructed in this application can, while carefully designed to destroy malicious gradients and pixel tampering of adversarial samples, ensure that the image contour can be correctly recognized by the classifier. Since the tasks of identifying and descrambling the target image and the application of the adversarial defense model in image denoising have commonalities, when the adversarial defense model is trained using multiple first sample pairs, the adversarial defense model trained on each first sample pair has enhanced ability to identify and descramble target images with various adversarial types, thereby accurately identifying different types of adversarial attacks.

[0070] Furthermore, since the type of adversarial attack is often unknown beforehand, training multiple adversarial defense models specifically for different attack types is impractical in real-world applications. Therefore, the best strategy is to train an adversarial defense model capable of handling multiple different types of adversarial attacks. Thus, in this application, training the adversarial defense model using samples of different adversarial attack types and / or attack intensities enables the model to cope with different types of adversarial attacks, thereby improving its defensive transferability.

[0071] This solves the technical problems of weak defense mobility and low accuracy in identifying different types of adversarial attacks in some existing defense schemes.

[0072] Furthermore, regarding the design of training data, this application also considers the transferability of CycleGAN module functionality. Defensive function transferability refers to whether the CycleGAN second discriminator module and second generator module, designed for adversarial example defense against a specific state recognition model, can be applied to adversarial example defense against other state recognition models. Even if the malicious interference noise patterns generated by similar adversarial examples basically follow the same rules, it is impossible to blindly and arbitrarily assume that defending against adversarial examples against one state recognition model will automatically defend against other state recognition models.

[0073] To avoid overfitting issues such as CycleGAN degenerating into a specialized defense module only capable of targeting a specific model due to adversarial examples carrying potential high-level semantic information of a particular state recognition model, thus rendering it unapplicable to other target models, this application also selects and combines adversarial examples generated against common state recognition models to form the first adversarial examples of the CycleGAN's second discriminator module and second generator module. For example, adversarial examples targeting driver state recognition models based on common networks such as MobileNet and ResNet are applied as the first adversarial examples in this application to obtain broader defense transfer performance.

[0074] Optionally, determining multiple first mixed sample strategies and forming multiple first sample pairs based on multiple first normal samples and first adversarial samples corresponding to each first normal sample includes: selecting a second mixed sample strategy from the multiple first mixed sample strategies, wherein the second mixed sample strategy is used to indicate the strategy that best trains the adversarial defense model; and selecting multiple second adversarial samples from the multiple first adversarial samples based on the second mixed sample strategy, and forming multiple second sample pairs using the multiple second adversarial samples and second normal samples corresponding to each second adversarial sample. Further optionally, the operation of selecting multiple second adversarial samples from the multiple first adversarial samples based on the second mixed sample strategy, and forming multiple second sample pairs using the multiple second adversarial samples and second normal samples corresponding to each second adversarial sample includes: determining a second mixed sample strategy, wherein the second mixed sample strategy is used to indicate training the adversarial defense model using multiple third adversarial samples and multiple fourth adversarial samples of equal quantity from the multiple first adversarial samples, wherein the multiple second adversarial samples include multiple third adversarial samples and multiple fourth adversarial samples, and the adversarial attack strength of the third adversarial samples is 0.1, and the adversarial attack strength of the fourth adversarial samples is 0.2; and selecting multiple second adversarial samples from the multiple first adversarial samples based on the second mixed sample strategy. This strategy selects multiple third adversarial samples from multiple first adversarial samples, including first adversarial samples corresponding to the deep deception algorithm, first adversarial samples corresponding to the saliency map adversarial algorithm, and first adversarial samples corresponding to the fast gradient symbol attack algorithm; based on a second hybrid sample strategy, it selects multiple fourth adversarial samples from multiple first adversarial samples, including first adversarial samples corresponding to the CW attack algorithm and first adversarial samples corresponding to the Gaussian noise algorithm; and uses multiple third adversarial samples, multiple fourth adversarial samples, and second normal samples corresponding to each third adversarial sample and each fourth adversarial sample to form multiple second sample pairs.

[0075] Specifically, referring to the above description, the processor 200 first determines multiple first mixed sample strategies. These first mixed sample strategies instruct the mixing of first adversarial samples corresponding to each first normal sample according to pre-defined adversarial attack types and intensities. In this embodiment, the multiple first mixed sample strategies include mixed sample strategies A through E, training the adversarial defense model using only the corresponding type of first adversarial samples, and directly processing the sample results using the first adversarial defense network. Table 4 shows the effects of training the adversarial defense model corresponding to each first mixed sample strategy.

[0076]

[0077]

[0078] Then, the processor 200 selects a second mixed-sample strategy from multiple first mixed-sample strategies. The second mixed-sample strategy indicates the strategy among the multiple first mixed-sample strategies that best trains the adversarial defense model. Referring to Table 4, after training the adversarial defense model based on mixed-sample strategy A, the adversarial defense model exhibits better defense performance compared to models trained based on other mixed-sample strategies. That is, in this embodiment, mixed-sample strategy A is the second mixed-sample strategy.

[0079] For example, hybrid sample strategy A involves selecting an equal number of third adversarial samples and multiple fourth adversarial samples from a plurality of first adversarial samples to train the adversarial defense model. The adversarial attack strength of the third adversarial samples is 0.1, and the adversarial attack strength of the fourth adversarial samples is 0.2. The plurality of third adversarial samples includes first adversarial samples corresponding to the deep deception algorithm, first adversarial samples corresponding to the saliency map adversarial algorithm, and first adversarial samples corresponding to the fast gradient sign attack algorithm. The plurality of fourth adversarial samples includes first adversarial samples corresponding to the CW attack algorithm and first adversarial samples corresponding to the Gaussian noise algorithm.

[0080] Thus, when the processor 200 determines multiple third adversarial samples and multiple fourth adversarial samples based on the second mixed sample strategy, it will form a second sample pair 1 by combining each third adversarial sample with the corresponding second normal sample, and form a second sample pair 2 by combining each fourth adversarial sample with the corresponding second normal sample.

[0081] Finally, the processor 200 trains the adversarial defense model using the second sample pair 1 and the second sample pair 2.

[0082] Thus, the above operations achieve the technical effect of improving the accuracy of the trained adversarial defense model, ensuring that the trained adversarial defense model can cope with a variety of different types of adversarial attacks, and improving the defensive transferability of the adversarial defense model.

[0083] Optionally, the adversarial defense model further includes a second adversarial defense network, which includes a first generation module and a first discriminator module. The first adversarial defense network includes a second discriminator module and a second generation module. The operation of inputting multiple first sample pairs into the adversarial defense model and training the model includes: inputting a first adversarial sample from each first sample pair into the second generation module to generate a descrambled sample; inputting a first normal sample from each first sample pair into the first generation module to generate a noisy sample; inputting the first normal sample and the descrambled sample together into the second discriminator module and outputting a first detection result and a second detection result; inputting the first adversarial sample and the noisy sample together into the first discriminator module and outputting a first discriminator result and a second discriminator result; and training the adversarial defense model based on the descrambled sample, the noisy sample, the first detection result, the second detection result, the first discriminator result, and the second discriminator result, using a loss function corresponding to the adversarial defense model.

[0084] Specifically, refer to Figure 4 As shown, the adversarial defense model also includes a second adversarial defense network, which comprises a first generation module and a first discriminator module. During training of the adversarial defense model, the processor 200 first inputs the first adversarial sample from the first sample pair into the second generation module. After processing by the second generation module, descrambled samples are generated. These descrambled samples are similar to but not identical to the first normal samples. Simultaneously, the processor 200 inputs the first normal sample from the first sample pair into the first generation module. After processing by the first generation module, noisy samples are generated. These noisy samples are similar to but not identical to the first adversarial samples.

[0085] Further, the processor 200 inputs the first normal sample and the descrambled sample into the second discrimination module, respectively, and obtains a first detection result corresponding to the first normal sample and a second detection result corresponding to the descrambled sample. At the same time, the processor 200 inputs the first adversarial sample and the noise sample into the first discrimination module, respectively, and obtains a first discrimination result corresponding to the first adversarial sample and a second discrimination result corresponding to the noise sample.

[0086] The processor 200 then substitutes the scrambled sample, the noisy sample, the first detection result, the second detection result, the first discrimination result, and the second discrimination result back into the Loss formula to perform Loss calculation with the second adversarial defense network and the first adversarial defense network, and uses the Adam parameter to optimize backpropagation until the network converges and reaches game equilibrium.

[0087] The specific Loss functions of the first discrimination module in the second adversarial defense network and the second discrimination module in the defense adversarial network are as follows:

[0088] In this embodiment, the Loss function of the first and second discriminant modules uses the cross-entropy function to determine the distance between CycleGAN games, as shown in the following formula:

[0089]

[0090] Where Dis represents the first discrimination module or the second discrimination module. Gen represents the first generation module corresponding to the first discrimination module, or the second generation module corresponding to the second discrimination module.

[0091] In this embodiment, the Loss functions of the first generation module and the second generation module use a combination of the mean squared error function and the cross-entropy function, as shown in the following formula:

[0092] Loss Gen =Loss Gen1 +Loss Gen2 (Gen∈{G,F})

[0093] Among them, Loss Gen1 Used to measure the difference between the descrambled samples generated by the second generation module and the normal samples. Specifically:

[0094]

[0095] in, This indicates that the second generation module generates a descrambled sample based on the first adversarial sample, and x represents the first normal sample.

[0096] And among them, Loss Gen2 This involves using the cross-entropy function based on the game theory principles of GANs. Specifically:

[0097]

[0098] The Loss function of the generator network is designed in this way. It considers the similarity between the descrambled sample generated by the second generation module after processing the adversarial sample and the first normal sample from the perspective of pixel difference. It also utilizes the working principle of sub-network game and uses the discrimination result of the second discriminator as the similarity measure, thereby prompting the second generation module to descramble the adversarial sample.

[0099] Furthermore, compared to typical generative adversarial network (GAN) schemes, this embodiment integrates CycleGAN into the adversarial example defense process, which can utilize different components in CycleGAN to achieve adversarial example detection and descrambling without the need for additional training of a second adversarial example discrimination module.

[0100] Specifically, since the second generation module in the adversarial defense model of this embodiment takes an image as input, it needs to first extract certain features from the image using two-dimensional convolution, and then perform deconvolution to generate a new denoised image. Furthermore, since this invention aims to provide the state recognition model with the function of defending against various adversarial attacks, considering the requirements of lightweight applications in vehicle-mounted models, this embodiment designs the structure of the second generation module as a deep learning model composed of several forward depthwise separable convolutional blocks, deconvolutional blocks, and residual blocks. The second discrimination module consists of several depthwise separable convolutional blocks and residual blocks, and finally outputs multi-size binary classification results.

[0101] The number of depthwise separable convolutional blocks and residual blocks can be adjusted according to task complexity and vehicle load capacity. In the CycleGAN defense scheme, the biggest advantage of using depthwise separable convolutional layers compared to ordinary convolutional layers is the reduction in computation and parameters. Ordinary convolution processes input data using a standard convolutional kernel, while depthwise separable convolution divides this operation into two steps: pointwise convolution of the depthwise kernel. This significantly reduces the complexity and number of parameters, making the adversarial defense model more efficient in terms of processing speed and memory usage, and better suited for edge computing platforms.

[0102] At the same time, due to its smaller number of parameters, it can also better reduce the overfitting of adversarial defense models.

[0103] Furthermore, the use of residual layers in the CycleGAN defense scheme can significantly improve the training efficiency of the adversarial defense model and the quality of the descrambled images. By introducing skip connections, residual layers directly pass the input signal to subsequent layers, helping to avoid the gradient minimization and gradient explosion problems common in deep networks, thereby accelerating the convergence process and making the adversarial defense model easier to train, especially in deeper network structures.

[0104] Furthermore, residual connections can alleviate the problem of gradient minimization, allowing gradients to be propagated more smoothly to preceding layers and maintaining the stability of the training process. For the second generation module, residual layers help learn more complex and detailed image features, enhancing the expressive power of the descrambling network, ensuring that the descrambled image remains realistic, and minimizing detail loss. Simultaneously, the residual structure allows the adversarial defense model to learn not only direct mappings but also the differences between input and output, thereby better fitting the target distribution, reducing the risk of overfitting, and ensuring the generalization ability of the adversarial defense model in the descrambling task of adversarial examples (images to be detected).

[0105] Optionally, after the adversarial defense model has been trained, the operation of performing adversarial detection and descrambling on the image to be detected using the first adversarial defense network includes: inputting the image to be detected into the second discriminator module and using the second discriminator module to determine whether the image to be detected corresponds to the adversarial type of the adversarial sample; if the image to be detected corresponds to the adversarial type of the adversarial sample, inputting the image to be detected into the second generation module and using the second generation module to descramble the image to be detected; and if the image to be detected does not correspond to the adversarial type of the adversarial sample, inputting the image to be detected into the state recognition model.

[0106] Specifically, Figure 6 This is a schematic diagram illustrating adversarial detection and descrambling of an image to be detected using a first adversarial defense network, as described in an embodiment of this application. (Reference) Figure 6 As shown, after the adversarial defense model is trained, it receives the image to be detected sent by the image acquisition device 100 and inputs it into the second discrimination module. The second discrimination module then determines whether the image to be detected corresponds to any type of adversarial attack. For example, whether the image to be detected has been subjected to an adversarial attack using a deepfake attack algorithm.

[0107] If the second discrimination module determines that the image to be detected corresponds to any one of the multiple adversarial attack types, the image to be detected is input to the second generation module, which then performs scrambling on the image. After the second generation module has finished processing the image, the scrambling image is input to the state recognition model, which then identifies the driver's state in the image.

[0108] If the second discrimination module determines that the image to be detected does not correspond to any of the multiple adversarial attack types, the image to be detected is directly input into the state recognition model, and the state recognition model is used to identify the driver's state in the image to be detected.

[0109] Therefore, since the adversarial defense model can comprehensively consider the problem that the current intelligent vehicle driver state recognition is not robust enough and is vulnerable to adversarial attacks, and combined with the special game-theoretic properties of the adversarial defense model, CycleGAN can be applied to the defense of driver state recognition. Thus, it can improve the robustness of the state recognition model and reduce the risk of intelligent vehicle decision errors caused by malicious attacks or noise.

[0110] In addition, refer to Figure 1 As shown, according to a second aspect of this embodiment, a storage medium is provided. The storage medium includes a stored program, wherein, when the program is executed, a processor performs any of the methods described above.

[0111] Thus, according to this embodiment, the technical effects of improving the defensive mobility of the adversarial defense model and improving the recognition accuracy of the adversarial defense model against different types of adversarial attacks are achieved.

[0112] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0113] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0114] Example 2

[0115] Figure 7An anti-defense device 700 for driver state recognition according to Embodiment 1 is shown, which corresponds to the method described in Embodiment 1. Referring to Figure 7, the device 700 includes: a model building module 710 for building an adversarial defense model connected to a state recognition model, wherein the state recognition model is used to monitor the driver's driving state, and the adversarial defense model includes a first adversarial defense network; an object sample first generation module 720 for collecting multiple first normal samples, using multiple different adversarial sample generation algorithms and based on the multiple first normal samples to generate first adversarial samples corresponding to each first normal sample, wherein the adversarial attack type and / or adversarial attack intensity of each first adversarial sample is different; a sample pair determination module 730 for determining multiple first mixed sample strategies, forming multiple first sample pairs according to the multiple first mixed sample strategies and based on the multiple first normal samples and the first adversarial samples corresponding to each first normal sample, wherein the first mixed sample strategies are used to instruct that the first adversarial samples be mixed according to a pre-set adversarial attack type and adversarial attack intensity; a model training module 740 for inputting the multiple first sample pairs into the adversarial defense model and training the adversarial defense model; and a model application module 750 for performing adversarial detection and descrambling processing on the image to be detected using the first adversarial defense network after the adversarial defense model has been trained.

[0116] Optionally, the sample pair determination module 730 includes: a second mixed sample strategy selection module, used to select a second mixed sample strategy from a plurality of first mixed sample strategies, wherein the second mixed sample strategy is used to indicate the strategy that best performs in training the adversarial defense model; and a sample pair first generation module, used to select a plurality of second adversarial samples from a plurality of first adversarial samples based on the second mixed sample strategy, and to form a plurality of second sample pairs using the plurality of second adversarial samples and the second normal samples corresponding to each second adversarial sample.

[0117] Optionally, the first sample generation module includes: a second mixed sample policy determination module, used to determine a second mixed sample policy, wherein the second mixed sample policy is used to instruct the adversarial defense model to be trained using an equal number of third adversarial samples and a plurality of fourth adversarial samples from a plurality of first adversarial samples, wherein the plurality of second adversarial samples includes a plurality of third adversarial samples and a plurality of fourth adversarial samples, and the adversarial attack strength of the third adversarial samples is 0.1, and the adversarial attack strength of the fourth adversarial samples is 0.2; and a third adversarial sample selection module, used to select a plurality of third adversarial samples from the plurality of first adversarial samples based on the second mixed sample policy, wherein the third adversarial samples... It includes a first adversarial sample corresponding to the deep deception algorithm, a first adversarial sample corresponding to the saliency map adversarial algorithm, and a first adversarial sample corresponding to the fast gradient symbol attack algorithm; a fourth adversarial sample selection module, used to select multiple fourth adversarial samples from multiple first adversarial samples based on a second hybrid sample strategy, wherein the fourth adversarial samples include first adversarial samples corresponding to the CW attack algorithm and first adversarial samples corresponding to the Gaussian noise algorithm; and a sample pair generation submodule, used to form multiple second sample pairs using multiple third adversarial samples, multiple fourth adversarial samples, and second normal samples corresponding to each third adversarial sample and each fourth adversarial sample respectively.

[0118] Optionally, the adversarial defense model further includes a second adversarial defense network, and the second adversarial defense network includes a first generation module and a first discrimination module. The first adversarial defense network includes a second discrimination module and a second generation module. The model training module 740 includes: a first generation module for descrambling samples, used to input the first adversarial sample in the first sample pair into the second generation module and generate descrambling samples; a first generation module for noise samples, used to input the first normal sample in the first sample pair into the first generation module and generate noise samples; a detection result output module, used to input the first normal sample and the descrambling sample into the second discrimination module respectively and output the first detection result and the second detection result; a discrimination result output module, used to input the first adversarial sample and the noise sample into the first discrimination module respectively and output the first discrimination result and the second discrimination result; and a model training submodule, used to train the adversarial defense model based on the descrambling sample, the noise sample, the first detection result, the second detection result, the first discrimination result, and the second discrimination result, and using a loss function corresponding to the adversarial defense model.

[0119] Optionally, the model application module 750 includes: an adversarial attack type determination module, used to match any one of the adversarial attack types among the types to be detected; a descrambling processing module, used to input the image to be detected to the second generation module and use the second generation module to descramble the image to be detected when the image to be detected matches any one of the multiple adversarial attack types; and a direct input module, used to input the image to be detected to the state recognition model when the image to be detected does not match the adversarial type of the adversarial sample.

[0120] Thus, according to this embodiment, the technical effects of improving the defensive mobility of the adversarial defense model and improving the recognition accuracy of the adversarial defense model against different types of adversarial attacks are achieved.

[0121] Example 3

[0122] Figure 8 An anti-countermeasure device 800 for driver state recognition according to Embodiment 1 is shown, which corresponds to the method described in Embodiment 1. (See reference...) Figure 8 As shown, the device 800 includes: a processor 810; and a memory 820 connected to the processor 810, for providing the processor 810 with instructions to process the following steps: constructing an adversarial defense model connected to a state recognition model, wherein the state recognition model is used to monitor the driver's driving state, and the adversarial defense model includes a first adversarial defense network; collecting multiple first normal samples, and generating multiple first adversarial samples corresponding to each first normal sample using multiple different adversarial sample generation algorithms, wherein the adversarial attack type and / or adversarial attack intensity of each first adversarial sample is different; determining multiple first mixed sample strategies, and forming multiple first sample pairs according to the multiple first mixed sample strategies and based on the multiple first normal samples and the first adversarial samples corresponding to each first normal sample, wherein the first mixed sample strategies are used to instruct that the first adversarial samples be mixed according to a pre-set adversarial attack type and adversarial attack intensity; inputting the first multiple sample pairs into the adversarial defense model and training the adversarial defense model; and, after the adversarial defense model has been trained, using the first adversarial defense network to perform adversarial detection and descrambling processing on the image to be detected.

[0123] Optionally, determining multiple first mixed sample strategies and forming multiple first sample pairs based on the multiple first mixed sample strategies and multiple first normal samples and first adversarial samples corresponding to each first normal sample includes: selecting a second mixed sample strategy from the multiple first mixed sample strategies, wherein the second mixed sample strategy is used to indicate the strategy that best performs in training the adversarial defense model; and selecting multiple second adversarial samples from the multiple first adversarial samples based on the second mixed sample strategy, and forming multiple second sample pairs using the multiple second adversarial samples and second normal samples corresponding to each second adversarial sample.

[0124] Optionally, based on a second mixed-sample strategy, the operation of selecting multiple second adversarial samples from multiple first adversarial samples and forming multiple second sample pairs using the multiple second adversarial samples and the second normal samples corresponding to each second adversarial sample includes: determining a second mixed-sample strategy, wherein the second mixed-sample strategy is used to instruct the adversarial defense model to be trained using multiple third adversarial samples and multiple fourth adversarial samples of equal number from the multiple first adversarial samples, the multiple second adversarial samples including multiple third adversarial samples and multiple fourth adversarial samples, and the adversarial attack strength of the third adversarial samples is 0.1, and the adversarial attack strength of the fourth adversarial samples is 0.2; based on the second mixed-sample strategy... The process involves selecting multiple third adversarial samples from multiple first adversarial samples, including first adversarial samples corresponding to the deep deception algorithm, first adversarial samples corresponding to the saliency map adversarial algorithm, and first adversarial samples corresponding to the fast gradient symbol attack algorithm; based on a second hybrid sample strategy, selecting multiple fourth adversarial samples from multiple first adversarial samples, including first adversarial samples corresponding to the CW attack algorithm and first adversarial samples corresponding to the Gaussian noise algorithm; and using multiple third adversarial samples, multiple fourth adversarial samples, and second normal samples corresponding to each third adversarial sample and each fourth adversarial sample respectively, forming multiple second sample pairs.

[0125] Optionally, the adversarial defense model further includes a second adversarial defense network, which includes a first generation module and a first discriminator module. The first adversarial defense network includes a second discriminator module and a second generation module. The operation of inputting multiple first sample pairs into the adversarial defense model and training the model includes: inputting a first adversarial sample from each first sample pair into the second generation module to generate a descrambled sample; inputting a first normal sample from each first sample pair into the first generation module to generate a noisy sample; inputting the first normal sample and the descrambled sample together into the second discriminator module and outputting a first detection result and a second detection result; inputting the first adversarial sample and the noisy sample together into the first discriminator module and outputting a first discriminator result and a second discriminator result; and training the adversarial defense model based on the descrambled sample, the noisy sample, the first detection result, the second detection result, the first discriminator result, and the second discriminator result, using a loss function corresponding to the adversarial defense model.

[0126] Optionally, after the adversarial defense model has been trained, the operation of performing adversarial detection and descrambling on the image to be detected using the first adversarial defense network includes: inputting the image to be detected into the second discriminator module and using the second discriminator module to determine whether the image to be detected corresponds to the adversarial type of the adversarial sample; if the image to be detected corresponds to the adversarial type of the adversarial sample, inputting the image to be detected into the second generation module and using the second generation module to descramble the image to be detected; and if the image to be detected does not correspond to the adversarial type of the adversarial sample, inputting the image to be detected into the state recognition model.

[0127] Thus, according to this embodiment, the technical effects of improving the defensive mobility of the adversarial defense model and improving the recognition accuracy of the adversarial defense model against different types of adversarial attacks are achieved.

[0128] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0129] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0131] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0133] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0134] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An adversarial defense method for driver state recognition, characterized in that, include: Construct an adversarial defense model connected to a state recognition model, wherein the state recognition model is used to monitor the driver's driving state, and the adversarial defense model includes a first adversarial defense network; Collect multiple first normal samples, use multiple different adversarial sample generation algorithms and based on the multiple first normal samples, generate first adversarial samples corresponding to each first normal sample, wherein the adversarial attack type and / or adversarial attack intensity of each first adversarial sample are different. Multiple first mixed sample strategies are determined, and multiple first sample pairs are formed based on the multiple first mixed sample strategies and the multiple first normal samples and the first adversarial samples corresponding to each first normal sample. The first mixed sample strategies are used to instruct the mixing of the first adversarial samples according to a pre-set adversarial attack type and adversarial attack strength. The plurality of first sample pairs are input into the adversarial defense model, and the adversarial defense model is trained. as well as Once the adversarial defense model has been trained, the first adversarial defense network is used to perform adversarial detection and descrambling on the image to be detected.

2. The method according to claim 1, characterized in that, The operation of determining multiple first mixed sample strategies and forming multiple first sample pairs based on the multiple first mixed sample strategies and the multiple first normal samples and the first adversarial samples corresponding to each first normal sample includes: Among the plurality of first mixed-sample strategies, a second mixed-sample strategy is selected, wherein the second mixed-sample strategy is used to indicate the strategy that best trains the adversarial defense model; and Based on the second hybrid sample strategy, multiple second adversarial samples are selected from the multiple first adversarial samples, and multiple second sample pairs are formed by using the multiple second adversarial samples and the second normal samples corresponding to each second adversarial sample.

3. The method according to claim 2, characterized in that, Based on the second hybrid sample strategy, the operation of selecting multiple second adversarial samples from the plurality of first adversarial samples, and using the plurality of second adversarial samples and the second normal samples corresponding to each second adversarial sample to form multiple second sample pairs includes: Determine the second hybrid sample strategy, wherein the second hybrid sample strategy is used to instruct the adversarial defense model to be trained using an equal number of third adversarial samples and a plurality of fourth adversarial samples from the plurality of first adversarial samples, wherein the plurality of second adversarial samples include the plurality of third adversarial samples and the plurality of fourth adversarial samples, and the adversarial attack strength of the third adversarial samples is 0.1, and the adversarial attack strength of the fourth adversarial samples is 0.2; Based on the second hybrid sample strategy, the plurality of third adversarial samples are selected from the plurality of first adversarial samples, wherein the third adversarial samples include the first adversarial samples corresponding to the deep deception algorithm, the first adversarial samples corresponding to the saliency graph adversarial algorithm, and the first adversarial samples corresponding to the fast gradient symbol attack algorithm. Based on the second hybrid sample strategy, a plurality of fourth adversarial samples are selected from the plurality of first adversarial samples, wherein the fourth adversarial samples include first adversarial samples corresponding to the CW attack algorithm and first adversarial samples corresponding to the Gaussian noise algorithm; and The plurality of third adversarial samples, the plurality of fourth adversarial samples, and the second normal samples corresponding to each third adversarial sample and each fourth adversarial sample are used to form the plurality of second sample pairs.

4. The method according to claim 1, characterized in that, The adversarial defense model further includes a second adversarial defense network, and the second adversarial defense network includes a first generation module and a first discrimination module. The first adversarial defense network includes a second discrimination module and a second generation module. The operation of inputting the plurality of first sample pairs into the adversarial defense model and training the adversarial defense model includes: The first adversarial sample from the first sample pair is input into the second generation module to generate descrambled samples; The first normal sample from the first sample pair is input into the first generation module, and a noise sample is generated. The first normal sample and the descrambled sample are respectively input into the second discrimination module, and the first detection result and the second detection result are output. The first adversarial sample and the noise sample are respectively input into the first discrimination module, and a first discrimination result and a second discrimination result are output; and Based on the descrambled samples, the noise samples, the first detection result, the second detection result, the first discrimination result, and the second discrimination result, and using the loss function corresponding to the adversarial defense model, the adversarial defense model is trained.

5. The method according to claim 4, characterized in that, Once the adversarial defense model has been trained, the operation of performing adversarial detection and descrambling on the image to be detected using the first adversarial defense network includes: The image to be detected is input to the second discrimination module, and the second discrimination module is used to determine whether the image to be detected corresponds to any one of the multiple adversarial attack types; If the image to be detected corresponds to any one of the multiple adversarial attack types, the image to be detected is input to the second generation module, and the second generation module is used to perform descrambling processing on the image to be detected; and If the image to be detected does not correspond to the adversarial type of the adversarial sample, the image to be detected is input into the state recognition model.

6. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, the method described in any one of claims 1 to 5 is performed by a processor.

7. An anti-defense device for driver state recognition, characterized in that, include: A model building module is used to build an adversarial defense model connected to a state recognition model, wherein the state recognition model is used to monitor the driver's driving state, and the adversarial defense model includes a first adversarial defense network. The first sample generation module is used to collect multiple first normal samples, and generate first adversarial samples corresponding to each first normal sample based on the multiple first normal samples using a variety of different adversarial sample generation algorithms. The adversarial attack type and / or adversarial attack intensity of each first adversarial sample are different. The sample pair determination module is used to determine multiple first mixed sample strategies, and to form multiple first sample pairs based on the multiple first mixed sample strategies and the multiple first normal samples and the first adversarial samples corresponding to each first normal sample. The first mixed sample strategies are used to instruct the mixing of each first adversarial sample according to a preset adversarial attack type and adversarial attack intensity. The model training module is used to input the plurality of first sample pairs into the adversarial defense model and train the adversarial defense model; as well as The model application module is used to perform adversarial detection and descrambling on the image to be detected using the first adversarial defense network after the adversarial defense model has been trained.

8. The apparatus according to claim 7, characterized in that, The sample pair determination module includes: A second mixed-sample strategy selection module is used to select a second mixed-sample strategy from the plurality of first mixed-sample strategies, wherein the second mixed-sample strategy is used to indicate the strategy that best trains the adversarial defense model; and The first sample pair generation module is used to select multiple second adversarial samples from the multiple first adversarial samples based on the second hybrid sample strategy, and to form multiple second sample pairs using the multiple second adversarial samples and the second normal samples corresponding to each second adversarial sample.

9. The apparatus according to claim 8, characterized in that, The sample pair first generation module includes: The second mixed sample strategy determination module is used to determine the second mixed sample strategy, wherein the second mixed sample strategy is used to instruct the adversarial defense model to be trained using an equal number of third adversarial samples and a plurality of fourth adversarial samples from the plurality of first adversarial samples, wherein the plurality of second adversarial samples include the plurality of third adversarial samples and the plurality of fourth adversarial samples, and the adversarial attack strength of the third adversarial samples is 0.1, and the adversarial attack strength of the fourth adversarial samples is 0.

2. The second adversarial sample selection module is used to select the plurality of third adversarial samples from the plurality of first adversarial samples based on the second hybrid sample strategy, wherein the third adversarial samples include first adversarial samples corresponding to the deep deception algorithm, first adversarial samples corresponding to the saliency graph adversarial algorithm, and first adversarial samples corresponding to the fast gradient symbol attack algorithm. The third adversarial sample selection module is used to select the plurality of fourth adversarial samples from the plurality of first adversarial samples based on the second hybrid sample strategy, wherein the fourth adversarial samples include first adversarial samples corresponding to the CW attack algorithm and first adversarial samples corresponding to the Gaussian noise algorithm; and The sample pair generation submodule is used to form the plurality of second sample pairs by utilizing the plurality of third adversarial samples, the plurality of fourth adversarial samples, and second normal samples corresponding to each third adversarial sample and each fourth adversarial sample.

10. An adversarial defense device for driver state recognition, characterized in that, include: processor; as well as A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: Construct an adversarial defense model connected to a state recognition model, wherein the state recognition model is used to monitor the driver's driving state, and the adversarial defense model includes a first adversarial defense network; Collect multiple first normal samples, use multiple different adversarial sample generation algorithms and based on the multiple first normal samples, generate first adversarial samples corresponding to each first normal sample, wherein the adversarial attack type and / or adversarial attack intensity of each first adversarial sample are different. Multiple first mixed sample strategies are determined, and multiple first sample pairs are formed based on the multiple first mixed sample strategies and the multiple first normal samples and the first adversarial samples corresponding to each first normal sample. The first mixed sample strategies are used to instruct the mixing of the first adversarial samples according to a pre-set adversarial attack type and adversarial attack strength. The plurality of first sample pairs are input into the adversarial defense model, and the adversarial defense model is trained. as well as Once the adversarial defense model has been trained, the first adversarial defense network is used to perform adversarial detection and descrambling on the image to be detected.

Images