Target detector safety testing method and device, electronic device and storage medium

Abstract

The present disclosure relates to a method and device for testing the safety of a target detector, an electronic device and a storage medium. The method comprises: inputting latent space variables satisfying a multi-dimensional Gaussian distribution into a generating network of an initial state to obtain a generating image, then inputting the latent space variables and the generating image into an auxiliary network of an initial state to obtain a single-dimensional auxiliary variable, then pasting the generating image into the annotated areas of each sample image in a training set to obtain a first training image after pasting, and finally optimizing the generating network of the initial state according to the target detector, the first training image and the auxiliary variables to obtain a generating network of a target state, wherein the generating network of the target state can effectively detect the safety and reliability of the target detector, remind users to pay attention to the risks of using the target detector, and is conducive to the developers of the target detector to improve the target detector.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a method and device for testing the safety of an object detector, an electronic device, and a storage medium. Background Art

[0002] Object detection refers to using computers to determine the location of objects in a scene, and marking the area where the object is located by labeling, marking, coloring or highlighting. The object can be a person or an object. Object detection, as an automatic positioning algorithm on images, has important applications in the fields of intelligent camera monitoring, autonomous driving and intelligent robots.

[0003] The currently popular target detection algorithm can use a deep neural network (DNN) to determine the position and size of the marked area where each target is located. However, DNN has been found to be disturbed by some specific disturbances, resulting in misjudgment or missed judgment. This disturbance is called an adversarial sample, which exists not only in the digital world, but also in the physical world. Such adversarial samples can make the target invisible to the target detector. At present, the invisibility of target detectors in the physical world usually adopts the method of patch patterns, which can only work when the patch pattern is facing the camera. Summary of the invention

[0004] In view of this, the present disclosure proposes a target detector security testing method and device, an electronic device and a storage medium.

[0005] According to one aspect of the present disclosure, a method for security testing of a target detector is provided, the method comprising: inputting preset latent space variables into a generative network of an initial state to obtain a generated image, the latent space variables satisfying a multi-dimensional Gaussian distribution; inputting the latent space variables and the generated image into an auxiliary network of an initial state to obtain a single-dimensional auxiliary variable; pasting the generated image into annotated areas of each sample image in a training set to obtain a first training image after pasting, the sample image including annotated areas of one or more targets; optimizing the generative network of the initial state according to the target detector, the first training image and the auxiliary variables to obtain a generative network of a target state, the generative network of the target state being used to perform security testing on the target detector.

[0006] In a possible implementation, the method further includes: inputting the latent space variables into a generation network of the target state to obtain a first test image, wherein the first test image is used to perform a security test on the target detector.

[0007] In a possible implementation, the method further includes: sampling the latent space variables using a circular cutting method to obtain sampling variables; inputting the sampling variables into the target state generation network to obtain a sampling image; pasting the sampling image into the annotated areas of each sample image in a training set to obtain a second training image after pasting; and optimizing the latent space variables according to the target detector and the second training image to obtain an optimized latent space variable.

[0008] In a possible implementation, the method further includes: inputting the optimized latent space variables into a generation network of the target state to obtain a second test image, wherein the second test image is used to perform a security test on the target detector.

[0009] In a possible implementation, the generative network of the initial state is optimized according to the target detector, the first training image and the auxiliary variables to obtain an optimized generative network of the target state, including: based on a loss function, the generative network of the initial state and the auxiliary network of the initial state are optimized to obtain the generative network of the target state and the auxiliary network of the target state; wherein the loss function includes a function representing the detection result of the target detector, a function representing the image smoothness of the generated image, and a mutual information substitution function representing the size of the mutual information between the input and output of the generative network, and the detection result of the target detector includes the confidence of the target.

[0010] In a possible implementation, the generation network includes a plurality of convolutional layers with a padding number of zero.

[0011] In a possible implementation, the latent space variables are sampled using a circular cutting method to obtain sampling variables, including: connecting the upper and lower sides and the left and right sides of the latent space variables to form a three-dimensional circular diagram; and sampling the plane expansion diagram of the three-dimensional circular diagram to obtain the sampling variables.

[0012] According to another aspect of the present disclosure, a security testing device for a target detector is provided, comprising: a first generating module, used to input preset latent space variables into a generating network in an initial state to obtain a generated image, wherein the latent space variables satisfy a multi-dimensional Gaussian distribution; a mutual information auxiliary module, used to input the latent space variables and the generated image into an auxiliary network in an initial state to obtain a single-dimensional auxiliary variable; a first pasting module, used to paste the generated image into the annotated area of ​​each sample image in a training set to obtain a first training image after pasting, wherein the sample image includes annotated areas of one or more targets; a first optimization module, used to optimize the generating network in the initial state according to the target detector, the first training image and the auxiliary variables to obtain a generating network in a target state, wherein the generating network in the target state is used to perform security testing on the target detector.

[0013] In a possible implementation, the device also includes a first testing module: used to input the latent space variables into the generation network of the target state to obtain a first test image, and the first test image is used to perform a security test on the target detector.

[0014] In a possible implementation, the device also includes: a ring cutting module, which is used to sample the latent space variables by a ring cutting method to obtain sampling variables; a second generation module, which is used to input the sampling variables into the generation network of the target state to obtain a sampling image; a second pasting module, which is used to paste the sampling image into the annotated area of ​​each sample image in the training set to obtain a pasted second training image; and a second optimization module, which is used to optimize the latent space variables according to the target detector and the second training image to obtain optimized latent space variables.

[0015] In a possible implementation, the method further includes a second testing module: configured to input the optimized latent space variables into a generation network of the target state to obtain a second test image, wherein the second test image is used to perform a security test on the target detector.

[0016] In one possible implementation, the first optimization module is specifically used to: optimize the generative network of the initial state and the auxiliary network of the initial state based on a loss function to obtain the generative network of the target state and the auxiliary network of the target state; wherein the loss function includes a function representing the detection result of the target detector, a function representing the image smoothness of the generated image, and a mutual information substitution function representing the size of the mutual information between the input and output of the generative network, and the detection result of the target detector includes the confidence of the target.

[0017] In a possible implementation, the generation network includes a plurality of convolutional layers with a padding number of zero.

[0018] In a possible implementation, the annular cutting module is specifically used to: connect the upper and lower sides and the left and right sides of the latent space variables to form a three-dimensional circular diagram; and sample the plane expansion diagram of the three-dimensional circular diagram to obtain the sampled variables.

[0019] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0020] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0021] In the disclosed embodiment, latent space variables satisfying multi-dimensional Gaussian distribution are input into the generative network of the initial state to obtain a generated image, and then the latent space variables and the generated image are input into the auxiliary network of the initial state to obtain a single-dimensional auxiliary variable, and then the generated image is pasted into the annotated area of ​​each sample image in the training set to obtain the first training image after pasting, and finally, according to the target detector, the first training image and the auxiliary variables, the generative network of the initial state is optimized to obtain the generative network of the target state, which can effectively detect the safety and reliability of the target detector, remind users to pay attention to the risks of using the target detector, and help the developers of the target detector to improve the target detector.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0024] Figure 1 A flowchart showing a method for testing the safety of an object detector according to an embodiment of the present disclosure is shown;

[0025] Figure 2 A schematic diagram showing a method for testing the safety of an object detector according to an embodiment of the present disclosure;

[0026] Figure 3A schematic diagram showing a first test image according to an embodiment of the present disclosure;

[0027] Figure 4 A schematic diagram showing a safety testing method of an object detector based on ring-cut sampling according to an embodiment of the present disclosure is shown;

[0028] Figure 5 A schematic diagram of a ring cutting sampling method according to an embodiment of the present disclosure;

[0029] Figure 6 A schematic diagram showing a second test image according to an embodiment of the present disclosure;

[0030] Figure 7 A schematic diagram showing a safety testing method of a pedestrian detector according to an embodiment of the present disclosure;

[0031] Figure 8 A block diagram showing a safety testing device of an object detector according to an embodiment of the present disclosure;

[0032] Fig. 9 A block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0033] Fig.10 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0034] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0035] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0036] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0037] In the related art, the physical world basically uses patch patterns to construct counter-perturbations for the invisibility of target detectors. For example, for the invisibility of pedestrian detectors, one or more square patch patterns can be pasted on relatively flat parts such as the chest of clothes, so that these patch patterns interfere with the output results of pedestrian detectors after being received by the camera. These patterns can be optimized by minimizing an objective function, and the objective function is the maximum value of the objectness score of each bounding box in the output results of the pedestrian detector.

[0038] However, in the related art, there are the following problems regarding the security testing method of the target detector:

[0039] First, in the physical world, the stealth method for target detectors can only be implemented through an overall patch pattern. When the target rotates and blocks part of the patch pattern, its stealth effect will be greatly reduced. Therefore, the patch pattern will only work as a stealth when it faces the camera. In this case, the stealth method and type are greatly limited.

[0040] Second, in the related art, simple expansion of patch patterns, such as pasting multiple patch patterns, is still difficult to work, because different patches will only work as a whole, and it is difficult for parts of multiple patches to be combined together to have a stealth effect.

[0041] Third, although the relevant technology can be extended to optimize 3D models of different objects (such as clothing), its simulation process is very cumbersome and complicated, and requires patch patterns to be affixed at specific locations, which has certain limitations in universality and versatility.

[0042] In view of the problems that the patch pattern can only achieve invisibility within a specific angle range relative to the camera and that 3D modeling is relatively complex, the present invention proposes a security testing method for a target detector, which inputs latent space variables satisfying multi-dimensional Gaussian distribution into a generating network in an initial state to obtain a generated image, and then inputs the latent space variables and the generated image into an auxiliary network in an initial state to obtain a single-dimensional auxiliary variable, and then pastes the generated image into the annotated area of ​​each sample image in the training set to obtain the first training image after pasting, and finally, according to the target detector, the first training image and the auxiliary variable, optimizes the generating network in the initial state to obtain a generating network in the target state.

[0043] The target state generation network can be used to represent the distribution of scalable texture patterns with stealth effects, and make such distribution have translation-invariant characteristics at each position of the pattern. Therefore, each part of the first test image generated by the target state generation network has a stealth effect, and the target detector can be made invisible at various shooting angles. The stealth effect of the first test image can be used to effectively test the security and reliability of the target detector, remind users to pay attention to the risks of using the target detector, and help the developers of the target detector to improve the detector.

[0044] Figure 1 A flowchart of a method for testing the safety of an object detector according to an embodiment of the present disclosure is shown. Figure 1 As shown, the method includes:

[0045] In step S11, a preset latent space variable is input into the generative network in the initial state to obtain a generated image, wherein the latent space variable satisfies a multi-dimensional Gaussian distribution;

[0046] In step S12, the latent space variables and the generated image are input into the auxiliary network in the initial state to obtain a single-dimensional auxiliary variable;

[0047] In step S13, the generated image is pasted into the annotated area of ​​each sample image in the training set to obtain a first training image after pasting, wherein the sample image includes an annotated area of ​​one or more targets;

[0048] In step S14, the generation network of the initial state is optimized according to the target detector, the first training image and the auxiliary variables to obtain the generation network of the target state, and the generation network of the target state is used to perform a security test on the target detector.

[0049] In a possible implementation, the security testing method of the target detector may be executed by a terminal device or other processing device, wherein the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, etc. Other processing devices may be a server or a cloud server, etc. In some possible implementations, the security testing method may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0050] In a possible implementation, in step S11, a preset latent space variable may be generated first. Since the latent space variable obeys a multidimensional standard Gaussian distribution Multidimensional standard Gaussian distribution Random sampling generates multi-dimensional latent space variables; or artificially gives a multi-dimensional standard Gaussian distribution Latent space variables. The present disclosure does not limit the method for generating latent space variables.

[0051] After obtaining the latent space variables, the latent space variables can be input into the generative network in the initial state to obtain the generated image.

[0052] The generative network is a scalable generative network for converting a multidimensional standard Gaussian distribution into a distribution in an image space, and the generative network includes a plurality of convolutional layers with a padding number of 0. For example, the generative network may be a fully convolutional neural network (FCNN), and each network layer included in the network is a plurality of convolutional layers with a padding number of 0. For the convolution operation of each layer, when the convolution kernel slides to the edge of the input data to be processed at the layer, the portion of the data edge corresponding to the convolution kernel that exceeds the portion of the data edge may be padded with 0.

[0053] Since the same convolution kernel can support convolutions of different dimensions, the dimensions of the latent space variables can be arbitrary. By inputting latent space variables of different dimensions into the generative network, generated images of different sizes can be generated.

[0054] Assume that during the training process, the size of the latent space variable is 16×128×9×9, the generative network is a fully convolutional neural network FCNN consisting of 7 convolutional layers with padding number 0, the convolution kernel size corresponding to each convolutional layer can be 3×3, and the size of the generated image output by the network can be 324×324. In this way, using the latent space variables of the same size, the generative network also generates generated images of the same size.

[0055] During the test, due to the scalability of the generative network, the last two dimensions of the latent space variables used in the training process (i.e., the dimensions of the height H and width W represented by 9×9) can be changed, so that the size of the latent space variables input into the trained generative network can be 16×128×H×W. When the values ​​of H and W are large enough (for example, H×W is much larger than 9×9), a large generated image (much larger than 324×324) can be generated. In addition, the generated image can be arbitrarily cropped (for example, into the shape of clothes) to facilitate security testing.

[0056] Through this method, each local pattern (such as the local pattern at the edge) in the generated image output by the generative network can share the same convolution kernel and satisfy the same translation-invariant distribution. The generated images of the same size can be used during the training process, and the generated images of a sufficiently large spatial size can be selected during the testing process, and the generated images of the sufficiently large spatial size can be arbitrarily cropped.

[0057] It should be understood that the size of the latent space variables and the network structure of the generated network can be set based on experience, and the present disclosure does not limit the size of the latent space variables and the network structure of the generated network.

[0058] In a possible implementation, in step S12, the latent space variables and the generated image obtained in step S11 may be input into the auxiliary network in the initial state, and a single-dimensional auxiliary variable may be obtained through processing by the auxiliary network.

[0059] The specific method is to obey the multidimensional standard Gaussian distribution The first latent space variable and the corresponding generated image (i.e., the latent space variable is input into the generating network, and the generated image is generated) are input into the auxiliary network in the initial state to obtain the first auxiliary variable. The generated image corresponding to the second latent space variable sampled in the above example and the first latent space variable is input into the auxiliary network in the initial state to obtain the second auxiliary variable. The first auxiliary variable and the second auxiliary variable are transformed (for example, including transformation by a softplus function) and then subtracted to obtain a single-dimensional auxiliary variable.

[0060] Among them, the first latent space variable and the second latent space variable are two latent space variables, which can be obtained by respectively applying multidimensional standard Gaussian distribution Random sampling generation. The auxiliary network can be used to represent the size of the mutual information between the input and output of the generating network. When the generated image is generated by the corresponding latent space variable through the generating network, the output value of the auxiliary network is large; when the latent space variable is independent of the generated image, the value of the auxiliary network output is small. The greater the difference between the two outputs of the auxiliary network (i.e., the first auxiliary variable and the second auxiliary variable) after transformation, the larger the single-dimensional auxiliary variable is, and the greater the mutual information between the input and output of the generating network.

[0061] It should be understood that the auxiliary network may be a neural network including multiple convolutional layers, and the present disclosure may not limit the specific network structure of the auxiliary network.

[0062] In a possible implementation, in step S13, the generated image may be randomly transformed to simulate the situation in the physical world, for example, by changing the scaling, rotation angle, contrast, brightness, and superimposing Gaussian noise on the generated image. Then, the transformed generated image may be pasted into the annotated area of ​​each sample image in the training set to obtain a pasted first training image.

[0063] When the sample image includes annotated areas of multiple targets, the generated images can be transformed according to the annotated areas of the multiple targets, so that each transformed generated image matches the corresponding annotated area, and then each transformed generated image is pasted to the corresponding annotated area.

[0064] The sample images in the training set may be images captured by an image acquisition device (e.g., a camera), and may include one or more targets to be identified, such as human bodies, faces, vehicles, objects, etc. The present disclosure does not limit the method for acquiring sample images and the categories of targets in the sample images.

[0065] The sample images in the training set may include the annotation results of the target to be identified, that is, the sample images in the training set are annotated with the targets by labeling, marking, coloring or highlighting. For example, the sample images may be input into a target detector, and the target detector may perform target detection on the sample images to obtain the annotated areas of the sample images. The annotated areas of the sample images may also be obtained by manual marking. The present disclosure does not limit the method for obtaining the annotated areas in the sample images.

[0066] In a possible implementation, in step S14, the first training image may be first input into the target detector, so that the target detector performs target detection processing on the training image, and obtains the detection result of the target detector. The target detector is used to detect the target, and may include a target detector based on a deep neural network or a target detector based on a convolutional neural network. The present disclosure does not limit the type of the target detector.

[0067] The detection result of the target detector and the auxiliary variable obtained in step S12 are then input into the loss function. A loss function can be set to degrade the detection result of the target detector and make the generation effect of the generation network better and better.

[0068] Based on the loss function, the back propagation algorithm can be used to optimize the parameters of the generating network and the auxiliary network to obtain the generating network of the optimized target state and the auxiliary network of the optimized target state.

[0069] In the process of optimizing the parameters of the generating network and the auxiliary network through the loss function, the fast gradient sign method (Fast Gradient Sign Method, FGSM), the projected gradient descent method (Project Gradient Descent, PGD) or the adaptive momentum stochastic optimization method (Adaptive Momentum, Adam) can be combined. The present disclosure does not limit the specific optimization algorithm.

[0070] The target state generation network obtained in the end can be used to perform security tests on target detectors. For example, the image generated by the target state generation network can be used to perform security tests on target detectors.

[0071] In this way, the latent space variables that satisfy the multi-dimensional Gaussian distribution are input into the generative network of the initial state to obtain a generated image, and then the latent space variables and the generated image are input into the auxiliary network of the initial state to obtain a single-dimensional auxiliary variable. Then, the generated image is pasted into the annotated area of ​​each sample image in the training set to obtain the first training image after pasting. Finally, according to the target detector, the first training image and the auxiliary variables, the generative network of the initial state is optimized to obtain the generative network of the target state. The generative network of the target state can effectively detect the safety and reliability of the target detector, and remind users to pay attention to the risks of using the target detector.

[0072] The following is a detailed description of the security testing method of the target detector according to the embodiment of the present disclosure.

[0073] Figure 2 A schematic diagram showing a method for testing the safety of an object detector according to an embodiment of the present disclosure is shown as follows: Figure 2 As shown, a scalable generative network G can be used to parameterize the distribution of the stealth texture pattern τ. The generative network G can be iteratively optimized using the method of steps S11 to S14 until the generative network G of the target state is obtained.

[0074] Multidimensional standard Gaussian distribution Random sampling generates a multi-dimensional latent space variable z; or it can be artificially given to satisfy the multi-dimensional standard Gaussian distribution The latent space variable z can be a tensor of size B×C×H×W, where B represents the batch size of z, C represents the number of channels of z, and H and W represent the height and width of z, respectively. It should be understood that the present disclosure does not limit the size and acquisition method of the latent space variable z.

[0075] After obtaining the latent space variable z, the latent space variable z can be input into the generative network G in the initial state in step S11 to obtain the generated image Among them, the generated image It can be added as a part of the overall texture pattern τ to the sample image in the training set for training, and the generation network G is a scalable generation network, for example, including a fully convolutional neural network FCNN. In the case of satisfying scalability, the generation network G may include multiple convolutional layers, and the present disclosure does not limit the specific network structure of the generation network G.

[0076] In step S11, the latent space variable z and the generated image are obtained In step S12, the latent space variable z can be combined with the generated image An auxiliary auxiliary network T is input to obtain a single-dimensional auxiliary variable. It should be understood that the auxiliary network can be a neural network including multiple convolutional layers, and the present disclosure does not limit the specific structure of the auxiliary network.

[0077] Among them, in the whole optimization process, the generation effect of the generated network G can be judged according to the size of the auxiliary variables, thereby assisting in optimizing the network parameters of the generated network G.

[0078] In step S13, for each sample image x in the training set, the labeled area of ​​each target in the sample image x can be obtained by manually marking or detecting the mark with a target detector. Perform the transformation and then generate the transformed image Paste it into the labeled area of ​​each sample image x in the training set to obtain the first training image after pasting.

[0079] Among them, the generated image The transformation process can simulate the situation in the physical world, such as changing the scale, rotation angle, contrast, brightness, superimposing Gaussian noise, etc. In addition, the transformation process can also use random thin plate spline (TPS) deformation to simulate the deformation of objects (such as cloth) in the physical world, that is, parameterizing the deformed object by grid calibration method. It should be understood that the present disclosure does not limit the specific form of the transformation process.

[0080] The above series of pairs can be used to generate images The operation of transforming and adding the sample image x is denoted as M, and the first training image after pasting is represented as

[0081] In step S14, according to the target detector, the first training image and the auxiliary variables, optimize the generated network G of the initial state to obtain the generated network G of the target state.

[0082] In a possible implementation, step S14 includes: optimizing the generating network of the initial state and the auxiliary network of the initial state based on the loss function to obtain the generating network of the target state and the auxiliary network of the target state;

[0083] The loss function includes a function representing the detection result of the target detector, a function representing the image smoothness of the generated image, and a mutual information substitution function representing the size of the mutual information between the input and output of the generating network, and the detection result of the target detector includes the confidence of the target.

[0084] For example, Figure 2 The mathematical meaning of the method shown is as follows: A predefined energy function U can be used to represent the desired target stealth pattern: (i.e., generating an image) probability density function:

[0085]

[0086] In formula (1), is the partition function, where Represents the generated image The energy function, energy The lower the probability The larger the energy function The choice can make the generated pattern more invisible.

[0087] For example, the generated image can be Energy function It is expressed as:

[0088]

[0089] In formula (2), the energy function It includes two energy function terms, α and β are preset weighting constants. It should be understood that the energy function U may also include other energy terms representing the detection results of the target detector, and the present disclosure does not limit the specific form of the energy function and the specific value of the weighting constant.

[0090] In formula (2), Represents the energy function of the detection result of the target detector. Among them, the function f represents the first training image Input the target detector, and the target detector outputs the maximum value of the target existence confidence. represents the expectation of random transformations in different sample images x and M.

[0091] It should be understood that the function f is only an example of the target existence confidence of the target detector, and may also include one or more of the target existence confidence, the confidence of a specific category, and the predicted detection frame coordinates. The present disclosure does not limit the specific form of the function f.

[0092] In formula (1), Represents the generated image The energy function of image smoothness can be expressed as:

[0093]

[0094] In formula (3), Represents the generated image The pixel at the i-th row and j-th column of . Function Generate images by calculation Each pixel The two surrounding pixels The calculation method of adding the absolute values ​​of the differences is used to describe the smoothness of the image. The smaller the value, the more similar each pixel is to the two surrounding pixels, and the smoother the image (the more continuous the pattern).

[0095] It should be understood that the function representing image smoothness is a function used to measure the degree of mutation or noise in the image, and can be a function for determining the similarity between two or more pixels in adjacent or close pixels. The present disclosure does not limit the specific function that can represent image smoothness.

[0096] According to formulas (1) to (3), when the training set and target detector are determined, the generated image The probability density p adv is a certain probability distribution. However, due to the difficulty in calculating the partition function Z U , it is impossible to directly satisfy the probability density p adv The generated image is obtained by sampling the distribution of Therefore, the generative network G can be used to approximate the generated image The distribution of Expressed as Where z is a latent space variable that satisfies the multidimensional standard Gaussian distribution.

[0097] Since the generated image It can be directly generated by feedforward, by first sampling the latent space variable z, and then obtaining the generated image through the generative network G Since the multidimensional standard Gaussian distribution satisfied by the latent space variable z is very easy to sample, the generated image can be easily obtained The generated image obtained by the above method The distribution of can be expressed as:

[0098]

[0099] In formula (4), p z is a standard Gaussian distribution The probability density function (PDF) of , δ is the Dirac function, is the parameter of the generative network G, generating the image Distribution and the parameters of the generative network G Related.

[0100] In order to make the distribution defined by the generated network G in formula (4) Closer to the definition in formula (1) Distribution, the optimization goal can be determined as minimizing the KL distance (Kullback-Leibler Divergence) between the two, where the KL distance is used to measure the difference between two probability distributions in the same event space. and The KL distance can be expressed as:

[0101]

[0102] In formula (5), the first term yes The negative of the entropy, that is Since the generated image Determined by the latent space variable z, mutual information can be introduced to help calculate this entropy. It can be expressed as:

[0103]

[0104] In formula (6), Is to generate an image and the joint distribution of the latent space variable z. Note that

[0105] And the marginal distribution You can get:

[0106]

[0107] In formula (7), represents the conditional entropy. According to formula (7), the first term of formula (5) can be replaced by Furthermore, note that You can get:

[0108]

[0109] According to formula (8), is a constant.

[0110] For the second term of formula (5) Notice You can get:

[0111]

[0112] In formula (9), logZ U is a constant.

[0113] In summary, after the derivation of formulas (5) to (9), the minimization formula (5) is equivalent to:

[0114]

[0115] According to formula (10), the optimization objectives need to be maximized simultaneously and minimize Based on the Deep InfoMax (DIM) method, we maximize Equivalent to maximizing a mutual information substitution function of JS divergence (Jensen-Shannon Divergence, JSD) Formula (10) can be further expressed as:

[0116]

[0117] In formula (11), the mutual information substitution function It can be expressed as:

[0118]

[0119] In formula (12), the softplus function, sp(t)=log(1+e t ), T ω Represents an auxiliary auxiliary network that can simultaneously receive generated images and latent space variables z as input and produces a single-dimensional output.

[0120] Therefore, the parameters of the generative network G can be adjusted based on the loss function shown in formula (11): The parameters ω of the auxiliary network T are optimized to obtain the target state generation network G and the target state auxiliary network T. The loss function may include a function representing the detection result of the target detector Image representing the generated image Function of smoothness And the mutual information substitution function representing the judgment result of the auxiliary network

[0121] In this way, by minimizing the loss function, we can obtain the target state generation network G, which can make the generated image corresponding to any latent space z that obeys the standard Gaussian distribution Both have stealth effects.

[0122] Through the optimization method of steps S11 to S14 above, a target state generation network for performing safety testing on the target detector can be obtained. The following step S15 introduces the specific process of performing safety testing using the target state generation network.

[0123] In a possible implementation, the method further includes step S15: inputting the latent space variables into the generation network of the target state to obtain a first test image, wherein the first test image is used to perform a security test on the target detector.

[0124] For example, the network parameters of the generated network can be optimized by the method of steps S11 to S14 to obtain a generated network of a target state. The generated network of the target state can be used to represent the distribution of an expandable texture pattern with a stealth effect, and make the distribution have the property of translation invariance at each position of the pattern.

[0125] According to the input latent space variables, the target state generation network can generate a first test image. Figure 3 A schematic diagram showing a first test image according to an embodiment of the present disclosure is shown as follows: Figure 3 As shown, the first test image is an expandable texture pattern with a stealth effect, and each small part in the first test image has a stealth effect on the target detector. For example, when the target of the target detector is a pedestrian, using the first test image, there is no need to print patterns at specific locations on clothing, nor to optimize for different clothing, so that different clothing containing or covered with stealth patterns (i.e., the first test image) can achieve stealth to the target detector at all angles.

[0126] Among them, due to the scalability of the generative network, a stealth texture pattern (first test image) of any scale can be generated. Each local pattern of the texture pattern can be regarded as generated by the same generative network during training, and will also have a stealth effect. The stealth effect is local and continuous.

[0127] The stealth effect of the first test image can be used to perform a security test on the target detector. If the target detector passes the security test of the first test image, it means that the target detector has a relatively high security; if the target detector fails the security test of the first test image, it means that the target detector has a relatively low security.

[0128] In this way, each part of the first test image generated by the target state generation network has a stealth effect, which can achieve stealth for the target detector at various shooting angles. The stealth effect of the first test image can be used to effectively test the security and reliability of the target detector, reminding users to pay attention to the risks of using the target detector, which is conducive to the developers of the target detector to improve the detector.

[0129] Although the first test image can already perform a good security test on the target detector, in order to further improve the effectiveness and versatility of the disclosed security test method, the latent space variable z can be further optimized.

[0130] Figure 4 A schematic diagram showing a method for testing the safety of a target detector based on ring-cut sampling according to an embodiment of the present disclosure is shown as follows: Figure 4 As shown, in the first stage of optimization (steps S11 to S14), the target state generation network G is obtained, and the parameters of the target state generation network G can be fixed. In the second stage (steps S21 to S24), the latent space variable z can be regarded as a periodically repeatable unit z in the latent space. local , that is, z local The repeating unit of z that represents the periodic repetition is optimized by using the optimization method based on ring cutting sampling for the periodic repetitive latent space variable z local Unit optimization can optimize a z that can be repeated periodically in space local mode.

[0131] In a possible implementation, the method further includes:

[0132] In step S21, the latent space variables are sampled using a ring cutting method to obtain sampled variables;

[0133] In step S22, the sampling variable is input into the generation network of the target state to obtain a sampling image;

[0134] In step S23, the sampled image is pasted into the annotated area of ​​each sample image in the training set to obtain a pasted second training image;

[0135] In step S24, the latent space variables are optimized according to the target detector and the second training image to obtain optimized latent space variables.

[0136] For example, in step S21, the latent space variable z local Perform toroidal cropping sampling (TCS) to obtain the sampling variable Crop torus (z local );

[0137] In a possible implementation, step S21 includes connecting the upper and lower sides and the left and right sides of the latent space variables to form a three-dimensional donut chart; sampling the plane expansion diagram of the three-dimensional donut chart to obtain the sampled variables.

[0138] Latent space variable z local It can be a tensor of size B×C×H×W, where B represents z local The batch size of C represents z local The number of channels, H and W represent z local The height and width of the latent space variable z can be local In the last two spatial dimensions (H×W), operations of connecting the upper and lower sides and the left and right sides are performed to form a three-dimensional donut chart, and then the plane expansion diagram of the donut chart is sampled to obtain the sampling variables.

[0139] For example, Figure 5 A schematic diagram of the ring cutting sampling method according to an embodiment of the present disclosure is shown in FIG. Figure 5 As shown, for ease of understanding, the latent space variable z local The values ​​of the first two spatial dimensions (B×C) are set to 1, and the latent space variable z local Think of it as a rectangle with height H and width W.

[0140] You can connect the top and bottom sides of the rectangle ( Figure 5 The edge in the direction of the horizontal arrow) is obtained as a hollow cylinder ( Figure 5 The hollow cylinder in the middle); then, connect the left and right sides of the cylinder (i.e. Figure 5 The edge of the vertical arrow in the left rectangle in the middle) to obtain a three-dimensional ring ( Figure 5 right circle).

[0141] Crop torus The latent space variable z local It is regarded as a three-dimensional torus surface, and then random sampling is performed on the randomly unfolded three-dimensional torus surface, which is equivalent to random sampling on the three-dimensional torus surface.

[0142] In this way, since the surface of the ring has no edges, such sampling methods are not limited by the edges of the rectangular pattern. local The random cutting sampling method of ring cutting is equivalent to the latent space variable z arranged by multiple cycles. local Random cutting sampling is performed on the infinitely expandable variables. When further selecting the latent space representation corresponding to the generated image with a more degraded effect, the sampling efficiency can be greatly improved, and the generated image can also be periodically expandable.

[0143] It should be understood that the present disclosure does not apply to latent space variables z localThere is no restriction on the size of B×C×H×W, and the data of the first two spatial dimensions (B×C) can be used to perform circular cutting sampling on the last two spatial dimensions (H×W).

[0144] In step S22, the sampling variable Crop torus (z local ) Input the target state generation network G and get the sampled image τ TS =G(Crop torus (z local )).

[0145] The target state generation network G is obtained by optimizing the network parameters of the initial state generation network G through the first stage optimization method (steps S11 to S14). Get the target state generation network G.

[0146] In step S23, reference may be made to the first stage optimization method (i.e. Figure 2 In step S13 of the method shown in FIG. 1 , the sampled image τ is TS Paste it into the labeled area of ​​each sample image x in the training set to obtain the pasted second training image M(x, τ TS )=M(x,G(Crop torus (z local ))), I will not go into details here.

[0147] In step S24, according to the target detector and the second training image M(x, τ TS ), for latent space variables z local Optimize and get the optimized latent space variable z local .

[0148] The second stage loss function can be preset for the latent space variable z local Optimize and get the optimized latent space variable z local Among them, the loss function includes a function representing the detection result of the target detector and a function representing the sampled image τ TS function of the image smoothness.

[0149] The smaller the function value of the loss function, the more the sampled image τ will be pasted. TS The sample image x, that is, the second training image M(x, τ TS ), input to the target detector, the greater the probability of the detection result being wrong, and the sampling image τ TS The smoother the image is, the better. Given the sampled image τ TS =G(Crop torus (z local ), we can take reducing the function value of the loss function as the goal and iteratively calculate the latent space variable zlocal The values ​​of each unit element are adjusted to obtain the optimized latent space variable z local .

[0150] Therefore, during the entire training process, the loss function E can be minimized. x,M [f(M(x,τ TS ))]+αE[TV(τ TS )] for latent space variables z local For optimization, the optimization objective can be expressed as:

[0151]

[0152] In formula (13), the loss function can be expressed as the detection result function of the target detector: And the function of image smoothness The weighted sum of α is a preset weighting coefficient, which can be set according to experience. The present disclosure does not limit the specific value of α.

[0153] Among them, the function f represents the second training image M(x,τ TS )=M(x,G(Crop torus (z local ))) Input the target detector, and the target detector outputs the maximum value of the target existence confidence. The smaller the value of f, the smaller the maximum value of the target existence confidence output by the target detector, and the lower the accuracy of the target detector in detecting the target.

[0154] It should be understood that in formula (13), function f only takes the target existence confidence of the target detector as an example, and may also include one or more of the target existence confidence, the confidence of a specific category, and the predicted detection box coordinates. The present disclosure does not limit the specific form of function f.

[0155] It should be understood that the second term of formula (13) is the same as that of formula (11) in the first stage optimization method. The essence is the same, but the optimization objects of the two are different. The second term of formula (11) The optimization object is to generate the parameters of the network G The optimization object of formula (13) is the latent space variable z local , and is the optimal parameter of the generated network G in the first stage After that, the parameters of the generated network are When fixed, adjust the latent space variable z by minimizing formula (13) local The values ​​of each unit element are included to obtain the optimized latent space variable z local .

[0156] In this way, the loss function shown in formula (13) can be minimized and the latent space variables z can be optimized. local , and get the optimized latent space variable z local . Cain space variable z local It can be repeated periodically, which is beneficial for making the pattern generated by the corresponding generation network G also repeat periodically, further improving the effectiveness and versatility of the security testing method of the target detector.

[0157] In a possible implementation, the method further includes step S25: inputting the optimized latent space variables into the target state generation network to obtain a second test image, wherein the second test image is used to perform a security test on the target detector.

[0158] For example, the optimized latent space variable z obtained by the method of steps S21 to S24 is local , and the Cain space variable z local The target state generation network (i.e., the generation network obtained in steps S11 to S14) is input into the target state. The target state generation network G is based on the input latent space variable z local , a second test image can be generated.

[0159] Figure 6 A schematic diagram showing a second test image according to an embodiment of the present disclosure is shown as follows: Figure 6 As shown, the second test image is a periodically extendable texture pattern with a stealth effect.

[0160] Each small part in the second test image has a stealth effect on the target detector. For example, when the target of the target detector is a pedestrian, using the second test image, there is no need to print patterns at specific locations on the clothes, nor to optimize for different clothes. The stealth texture pattern (i.e., the periodically extended second test image) can be directly printed on the fabric, and the clothes made of these fabrics have the effect of being invisible to the target detector at all angles.

[0161] The invisibility effect of the second test image can be used to perform a security test on the target detector. If the target detector passes the security test performed by the second test image, it means that the target detector has a relatively high security; if the target detector fails the security test performed by the second test image, it means that the target detector has a relatively low security.

[0162] In this way, each part of the second test image generated by the target state generation network has a stealth effect, which can achieve stealth for the target detector at various shooting angles. The stealth effect of the second test image can be used to effectively test the security and reliability of the target detector. Figure 3), the second test image has a better test degradation effect, which can more effectively test the security of the target detector, remind users to pay attention to the risks of using the target detector, and is conducive to the developers of the target detector to improve the detector.

[0163] In summary, in practical applications, such as the safety test of pedestrian detectors, invisible texture patterns (first test images or second test images) can be directly printed on fabrics, and clothing made of these fabrics will have the effect of avoiding pedestrian detectors at all angles. Then, the pedestrian detector can be tested for safety based on this invisible property. If the pedestrian detector can detect pedestrians wearing invisible fabrics, it means that the safety of the pedestrian detector is relatively good; if the pedestrian detector cannot detect pedestrians wearing invisible fabrics, it means that the safety of the pedestrian detector needs to be improved.

[0164] It can be seen that the stronger the stealth effect of the stealth texture image, the more effective the safety of the pedestrian detector can be tested. Figure 7 A schematic diagram showing a safety test method for a pedestrian detector according to an embodiment of the present disclosure is shown as follows: Figure 7 As shown, for the shooting angles of different cameras of the pedestrian detector, including the case where part of the invisible cloth is blocked, the pedestrian detector does not detect the pedestrian wearing the invisible cloth. The security of the detector needs to be further enhanced.

[0165] The disclosed method has better versatility. It does not need to print the invisible texture pattern at a specific position on the clothing, nor does it need to be optimized for different clothing. Moreover, after the training is completed, the parameters of the generation network can be left unchanged, and a invisible pattern of any size with a translation-invariant distribution can be directly generated.

[0166] Therefore, in the disclosed embodiment, in steps S11 to S15, latent space variables that satisfy the multi-dimensional Gaussian distribution are input into the generative network of the initial state to obtain a generated image, and then the latent space variables and the generated image are input into the auxiliary network of the initial state to obtain a single-dimensional auxiliary variable, and then the generated image is pasted into the annotated area of ​​each sample image in the training set to obtain the first training image after pasting, and finally, according to the target detector, the first training image and the auxiliary variables, the generative network of the initial state is optimized to obtain the generative network of the target state, and the generative network of the target state can generate a first test image for effectively testing the safety and reliability of the target detector, reminding users to pay attention to the risks of using the target detector, which is conducive to the developers of the target detector to improve the target detector.

[0167] Furthermore, after obtaining the target state generation network in steps S11 to S15, the parameters of the target state generation network can be fixed in steps S21 to S25, and the latent space variables can be optimized based on the ring cutting sampling method to obtain the optimized latent space variables with periodic expansion. Finally, the optimized periodically repeatable latent space variables are input into the target state generation network to obtain the second test image. Compared with the first test image, the second test image has better versatility and can more effectively test the security and reliability of the target detector.

[0168] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0169] In addition, the present disclosure also provides a safety testing device, an electronic device, a computer-readable storage medium, and a program for a target detector, all of which can be used to implement any safety testing method for a target detector provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method section and will not be repeated here.

[0170] Figure 8 A block diagram of a safety testing device for an object detector according to an embodiment of the present disclosure is shown. Figure 8 As shown, the device comprises:

[0171] A first generating module 71 is used to input preset latent space variables into a generating network in an initial state to obtain a generated image, wherein the latent space variables satisfy a multi-dimensional Gaussian distribution;

[0172] A mutual information auxiliary module 72, used for inputting the latent space variables and the generated image into an auxiliary network in an initial state to obtain a single-dimensional auxiliary variable;

[0173] A first pasting module 73, used for pasting the generated image into the annotated area of ​​each sample image in the training set to obtain a first training image after pasting, wherein the sample image includes an annotated area of ​​one or more targets;

[0174] The first optimization module 74 is used to optimize the generation network of the initial state according to the target detector, the first training image and the auxiliary variables to obtain the generation network of the target state, and the generation network of the target state is used to perform security testing on the target detector.

[0175] In a possible implementation, the device also includes a first testing module: used to input the latent space variables into the generation network of the target state to obtain a first test image, and the first test image is used to perform a security test on the target detector.

[0176] In a possible implementation, the device also includes: a ring cutting module, which is used to sample the latent space variables by a ring cutting method to obtain sampling variables; a second generation module, which is used to input the sampling variables into the generation network of the target state to obtain a sampling image; a second pasting module, which is used to paste the sampling image into the annotated area of ​​each sample image in the training set to obtain a pasted second training image; and a second optimization module, which is used to optimize the latent space variables according to the target detector and the second training image to obtain optimized latent space variables.

[0177] In a possible implementation, the method further includes a second testing module: configured to input the optimized latent space variables into a generation network of the target state to obtain a second test image, wherein the second test image is used to perform a security test on the target detector.

[0178] In one possible implementation, the first optimization module 74 is specifically used to: optimize the generative network of the initial state and the auxiliary network of the initial state based on a loss function to obtain the generative network of the target state and the auxiliary network of the target state; wherein the loss function includes a function representing the detection result of the target detector, a function representing the image smoothness of the generated image, and a mutual information substitution function representing the size of the mutual information between the input and output of the generative network, and the detection result of the target detector includes the confidence of the target.

[0179] In a possible implementation, the generation network includes a plurality of convolutional layers with a padding number of zero.

[0180] In a possible implementation, the annular cutting module is specifically used to: connect the upper and lower sides and the left and right sides of the latent space variables to form a three-dimensional circular diagram; and sample the plane expansion diagram of the three-dimensional circular diagram to obtain the sampled variables.

[0181] In some embodiments, the security testing device of the target detector provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the method described in the above method embodiments. Its specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0182] The embodiment of the present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0183] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0184] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0185] Fig. 9 A block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0186] Reference Fig. 9 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0187] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0188] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0189] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0190] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0191] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0192] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0193] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0194] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G), a third generation mobile communication technology (3G), a fourth generation mobile communication technology (4G) or a fifth generation mobile communication technology (5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0195] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0196] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above method.

[0197] Fig.101 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Fig.10 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0198] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Microsoft's server operating system (Windows ServerTM), Apple's graphical user interface-based operating system (Mac OSXTM), a multi-user multi-process computer operating system (UnixTM), a free and open source Unix-like operating system (LinuxTM), an open source Unix-like operating system (FreeBSDTM), or the like.

[0199] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0200] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0201] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0202] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0203] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0204] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0205] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0206] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0207] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0208] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0209] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for testing the safety of a target detector, characterized in that: The method comprises: Inputting preset latent space variables into the generative network in the initial state to obtain a generated image, wherein the latent space variables satisfy a multi-dimensional Gaussian distribution; Inputting the latent space variables and the generated image into the auxiliary network in the initial state to obtain a single-dimensional auxiliary variable; Pasting the generated image into the annotated area of ​​each sample image in the training set to obtain a first training image after pasting, wherein the sample image includes an annotated area of ​​one or more targets; According to the target detector, the first training image and the auxiliary variable, the generation network of the initial state is optimized to obtain a generation network of the target state, wherein the generation network of the target state is used to perform a safety test on the target detector; Inputting the latent space variable into the target state generation network to obtain a first test image, wherein the first test image is used to perform a security test on the target detector; Wherein, the method further comprises: Performing a ring cutting method to sample the latent space variables to obtain sampling variables; Inputting the sampling variable into the generation network of the target state to obtain a sampling image; Pasting the sampled image into the annotated area of ​​each sample image in the training set to obtain a pasted second training image; Optimizing the latent space variables according to the target detector and the second training image to obtain optimized latent space variables; The optimized latent space variables are input into the target state generation network to obtain a second test image, where the second test image is used to perform a security test on the target detector.

2. The method according to claim 1, characterized in that Optimizing the generative network of the initial state according to the target detector, the first training image and the auxiliary variable to obtain an optimized generative network of the target state includes: Based on the loss function, optimizing the generating network of the initial state and the auxiliary network of the initial state to obtain the generating network of the target state and the auxiliary network of the target state; The loss function includes a function representing the detection result of the target detector, a function representing the image smoothness of the generated image, and a mutual information substitution function representing the mutual information size between the input and output of the generated network. The detection result of the target detector includes the confidence of the target.

3. The method according to claim 1, characterized in that The generation network includes a plurality of convolutional layers with a padding number of zero.

4. The method according to claim 1, characterized in that: The latent space variables are sampled by the ring cutting method to obtain sampling variables, including: Connecting the upper and lower sides and the left and right sides of the latent space variable to form a three-dimensional doughnut diagram; The plane expansion diagram of the three-dimensional doughnut chart is sampled to obtain the sampling variables.

5. A safety test device for a target detector, characterized in that: include: A first generation module is used to input preset latent space variables into a generation network in an initial state to obtain a generated image, wherein the latent space variables satisfy a multi-dimensional Gaussian distribution; A mutual information auxiliary module, used for inputting the latent space variables and the generated image into an auxiliary network in an initial state to obtain a single-dimensional auxiliary variable; A first pasting module, used for pasting the generated image into the annotated area of ​​each sample image in the training set to obtain a first training image after pasting, wherein the sample image includes an annotated area of ​​one or more targets; A first optimization module is used to optimize the generation network of the initial state according to the target detector, the first training image and the auxiliary variable to obtain a generation network of the target state, wherein the generation network of the target state is used to perform a safety test on the target detector; A first testing module: used for inputting the latent space variables into the generation network of the target state to obtain a first test image, wherein the first test image is used for performing a security test on the target detector; Wherein, the device further comprises: A ring cutting module, used for sampling the latent space variables by a ring cutting method to obtain sampled variables; A second generation module is used to input the sampling variable into the generation network of the target state to obtain a sampling image; A second pasting module, used for pasting the sampled image into the annotated area of ​​each sample image in the training set to obtain a pasted second training image; A second optimization module, configured to optimize the latent space variables according to the target detector and the second training image to obtain optimized latent space variables; The second testing module is used to input the optimized latent space variables into the generation network of the target state to obtain a second test image, and the second test image is used to perform a security test on the target detector.

6. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 4.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images