Robustness analysis methods, devices, electronic equipment and storage media

Abstract

This invention provides a viewpoint robustness analysis method, apparatus, electronic device, and storage medium. The viewpoint robustness analysis method includes: acquiring multiple two-dimensional images of an object to be identified; obtaining a three-dimensional representation of the object based on the multiple two-dimensional images; obtaining a rendered two-dimensional image under an initial adversarial viewpoint based on the three-dimensional representation; determining an optimized adversarial viewpoint of the object based on the rendered two-dimensional image under the initial adversarial viewpoint; and evaluating the viewpoint robustness of a visual perception model based on the optimized adversarial viewpoint. The viewpoint robustness analysis method of this invention improves the accuracy of analysis during the viewpoint robustness analysis of a visual perception model and reduces operating costs.

Description

Technical Field

[0001] This invention relates to the field of visual perception technology, and in particular to a method, apparatus, electronic device, and storage medium for viewpoint robustness analysis. Background Technology

[0002] Viewpoint robustness refers to the ability of a visual perception model to recognize objects from different viewpoints; that is, the ability of a visual perception model to recognize objects from different viewpoints.

[0003] According to relevant technologies, the current approach often involves constructing a dataset containing images taken from different perspectives, or by creating 3D models of objects and simulating their transformations under different perspectives, to evaluate the perspective robustness of visual perception models.

[0004] However, using the aforementioned method to perform viewpoint robustness analysis on visual perception models is costly and has low accuracy. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for viewpoint robustness analysis, which improves the accuracy of analysis in the process of viewpoint robustness analysis of visual perception models and reduces operating costs.

[0006] This invention provides a viewpoint robustness analysis method, which includes: acquiring multiple two-dimensional images of an object to be identified; obtaining a three-dimensional representation of the object to be identified based on the multiple two-dimensional images; obtaining a rendered two-dimensional image under an initial adversarial viewpoint based on the three-dimensional representation; determining an optimized adversarial viewpoint of the object to be identified based on the rendered two-dimensional image under the initial adversarial viewpoint; and evaluating the viewpoint robustness of a visual perception model based on the optimized adversarial viewpoint.

[0007] According to a perspective robustness analysis method provided by the present invention, obtaining a three-dimensional representation of the object to be identified based on multiple two-dimensional images specifically includes: obtaining a three-dimensional representation of the object to be identified through a neural radiation field based on multiple two-dimensional images.

[0008] According to a viewpoint robustness analysis method provided by the present invention, obtaining a rendered 2D image under an initial adversarial viewpoint based on the 3D representation specifically includes: determining the initial adversarial viewpoint; and obtaining a rendered 2D image under the initial adversarial viewpoint through a neural radiation field based on the 3D representation and the initial adversarial viewpoint.

[0009] According to a viewpoint robustness analysis method provided by the present invention, determining the optimized adversarial viewpoint of the object to be identified based on the rendered 2D image under the initial adversarial viewpoint specifically includes: obtaining a classification loss function based on the rendered 2D image under the initial adversarial viewpoint; optimizing the initial adversarial viewpoint based on the classification loss function to obtain an optimized initial adversarial viewpoint, so as to maximize the value of the classification loss function obtained based on the rendered 2D image under the optimized initial adversarial viewpoint, and using the optimized initial adversarial viewpoint as the optimized adversarial viewpoint of the object to be identified.

[0010] According to a perspective robustness analysis method provided by the present invention, the optimization of the initial adversarial perspective based on the classification loss function specifically includes: optimizing the initial adversarial perspective using a Gaussian distribution based on the classification loss function.

[0011] The present invention also provides a viewpoint robustness analysis device, comprising: a first module for acquiring multiple two-dimensional images of an object to be identified; a second module for obtaining a three-dimensional representation of the object to be identified based on the multiple two-dimensional images; a third module for obtaining a rendered two-dimensional image under an initial adversarial viewpoint based on the three-dimensional representation; and a fourth module for determining an optimized adversarial viewpoint of the object to be identified based on the rendered two-dimensional image under the initial adversarial viewpoint, and evaluating the viewpoint robustness of a visual perception model based on the optimized adversarial viewpoint.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the perspective robustness analysis method as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the perspective robustness analysis method as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the perspective robustness analysis method as described above.

[0015] The present invention provides a viewpoint robustness analysis method, apparatus, electronic device, and storage medium that obtains a three-dimensional representation of the object to be identified based on multiple two-dimensional images of the object, obtains a rendered two-dimensional image under an initial adversarial viewpoint based on the three-dimensional representation, and determines an optimized adversarial viewpoint of the object to be identified based on the rendered two-dimensional image under the initial adversarial viewpoint. This optimizes the adversarial viewpoint so that it can deceive the visual perception model with a higher success rate, thereby improving the accuracy of the viewpoint robustness analysis based on the optimized adversarial viewpoint and reducing the operational cost of evaluating the viewpoint robustness of the visual perception model. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the perspective robustness analysis method provided by the present invention;

[0018] Figure 2 This is a schematic diagram of the process of obtaining a rendered 2D image from an initial adversarial perspective based on 3D representation, provided by the present invention.

[0019] Figure 3 This is a flowchart illustrating the process of determining the optimized adversarial perspective of the object to be identified based on a rendered 2D image from an initial adversarial perspective, as provided by the present invention.

[0020] Figure 4 This is a schematic diagram illustrating an application scenario of the perspective robustness analysis method provided by the present invention;

[0021] Figure 5 This is a schematic diagram of the viewpoint robustness analysis device provided by the present invention;

[0022] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] To better study the viewpoint robustness of visual perception models, the viewpoint robustness analysis method provided in this invention finds the distribution of adversarial viewpoints through optimization. That is, any viewpoint sampled from the distribution can deceive the visual perception model, thereby improving the accuracy of the analysis of the viewpoint robustness of the visual perception model based on the optimized adversarial viewpoints and reducing the operational cost of evaluating the viewpoint robustness of the visual perception model.

[0025] The following will combine Figure 1 The process of the robustness analysis method for perspective provided by this invention will be described.

[0026] Figure 1 This is a flowchart illustrating the perspective robustness analysis method provided by the present invention.

[0027] In an exemplary embodiment of the present invention, combined with Figure 1 As can be seen, the perspective robustness analysis method may include steps 110 to 140, which will be described in detail below.

[0028] In step 110, multiple two-dimensional images of the object to be identified are acquired.

[0029] In one embodiment, multiple two-dimensional images of the object to be identified from different perspectives can be acquired. The object to be identified can be understood as an object used to evaluate the perspective robustness of the visual perception model, that is, to evaluate the ability of the visual perception model to identify the object from different perspectives.

[0030] In step 120, a three-dimensional representation of the object to be identified is obtained based on multiple two-dimensional images.

[0031] In one embodiment, a three-dimensional representation of the object to be identified in three-dimensional space can be obtained from multiple two-dimensional images of the object from different perspectives.

[0032] In another embodiment, obtaining a three-dimensional representation of the object to be identified based on multiple two-dimensional images can be achieved in the following way:

[0033] Based on multiple two-dimensional images, a three-dimensional representation of the object to be identified is obtained through neural radiation fields.

[0034] In one example, to study the perspective robustness of a model for physical world objects, a neural radiation field can be used to model the physical world objects (corresponding to the object to be identified) to obtain a three-dimensional representation of the object to be identified.

[0035] It should be noted that neural radiation fields can represent the color and density of objects in physical space through neural networks, and obtain the color of each ray of light through voxel rendering technology. Neural radiation fields can render realistic images from multiple perspectives.

[0036] In one embodiment, the neural radiation field can be modeled using a neural network to represent the spatial radiation field as F:(x,d)→(c,τ). Here, x represents the coordinates in space, d represents the observation direction, and its output includes RGB color c and voxel density τ. Let r(t)=o+td be a beam of light emitted from the camera origin o along direction d; then the pixel value rendered in this direction can be expressed as formula (1):

[0037]

[0038] in, t i t j These represent the positions t of the light rays, respectively. i and the position of light t j ;δ i =t i+1 -t i These are two adjacent sampling points (corresponding to ray positions t). i+1 and t i The distance between ); δ j =t j+1 -t j These are two adjacent sampling points (corresponding to ray positions t). j+1 and t j The distance between them.

[0039] By specifying different directions, an image can be rendered; therefore, neural radiation fields can generate realistic images from any viewpoint.

[0040] In another embodiment, a three-dimensional representation of the object to be identified can be obtained based on multiple two-dimensional images through other processing methods. For example, a high-precision 3D model of the object can be obtained using a 3D scanner, thus obtaining a three-dimensional representation of the object to be identified. In this embodiment, the specific method of obtaining a three-dimensional representation of the object to be identified based on multiple two-dimensional images is not limited.

[0041] In step 130, a rendered 2D image from the initial adversarial perspective is obtained based on the 3D representation.

[0042] In step 140, based on the rendered 2D image under the initial adversarial view, the optimized adversarial view of the object to be identified is determined, and the view robustness of the visual perception model is evaluated based on the optimized adversarial view.

[0043] In one embodiment, a rendered 2D image under an initial adversarial viewpoint can be obtained based on the obtained 3D representation of the object to be identified. Here, the initial adversarial viewpoint can be understood as the initialized viewpoint. Furthermore, the distribution of adversarial viewpoints can be found through optimization to obtain an optimized adversarial viewpoint for the object to be identified.

[0044] In one example, an optimized adversarial view of the object to be identified can be obtained based on a rendered 2D image from an initial adversarial view. This optimized adversarial view can deceive the visual perception model with a higher success rate, thereby improving the analytical accuracy of evaluating the view robustness of the visual perception model based on the optimized adversarial view and reducing the operational cost of evaluating the view robustness of the visual perception model.

[0045] The viewpoint robustness analysis method provided by this invention obtains a three-dimensional representation of the object to be identified based on multiple two-dimensional images of the object to be identified, obtains a rendered two-dimensional image under an initial adversarial viewpoint based on the three-dimensional representation, and determines an optimized adversarial viewpoint of the object to be identified based on the rendered two-dimensional image under the initial adversarial viewpoint. This optimizes the adversarial viewpoint so that it can deceive the visual perception model with a higher success rate, thereby improving the accuracy of the analysis of viewpoint robustness of the visual perception model based on the optimized adversarial viewpoint and reducing the operational cost of evaluating the viewpoint robustness of the visual perception model.

[0046] To further conclude the perspective robustness analysis method provided by this invention, the following will combine... Figure 2 Please provide an explanation.

[0047] Figure 2 This is a schematic diagram of the process of obtaining a rendered 2D image from an initial adversarial perspective based on 3D representation, as provided by the present invention.

[0048] In an exemplary embodiment of the present invention, combined with Figure 2 As can be seen, obtaining the rendered 2D image from the initial adversarial perspective based on the 3D representation can include steps 210 and 220, which will be described in detail below.

[0049] In step 210, the initial adversarial perspective is determined.

[0050] In step 220, based on the three-dimensional representation and the initial adversarial perspective, a rendered two-dimensional image under the initial adversarial perspective is obtained through the neural radiation field.

[0051] In one embodiment, an initial position of the camera can be given and then rotated and translated. After the camera is transformed, a new shooting perspective, i.e., the initial adversarial perspective, can be obtained. The rotation angle can be represented as (ψ, θ, φ), and the translation distance can be represented as (Δ). x ,Δ y ,Δ z ).

[0052] In one example, a 2D image (corresponding to a 2D image) at the initial adversarial viewpoint can be obtained through neural radiation field rendering, which can be represented as v = [ψ, θ, φ, Δ]. x ,Δ y,Δ z The rendered image (corresponding to a rendered 2D image) can be represented as R(v), where R represents the entire rendering process.

[0053] To prevent the degradation of adversarial perspective distribution, the perspective robustness analysis method provided in this invention also proposes to utilize entropy regularization terms to effectively enhance the diversity of adversarial perspectives.

[0054] Figure 3 This is a flowchart illustrating the process of determining the optimized adversarial viewpoint of the object to be identified based on a rendered 2D image from an initial adversarial perspective, as provided by this invention. The following will combine... Figure 3 This paper describes the process of determining the optimized adversarial perspective of the object to be identified from a rendered 2D image based on the initial adversarial perspective.

[0055] In an exemplary embodiment of the present invention, combined with Figure 3 As can be seen, the optimized adversarial view of the object to be identified, based on the rendered 2D image under the initial adversarial view, can include steps 310 and 320. Each step will be described below.

[0056] In step 310, a classification loss function is obtained based on the rendered 2D image from the initial adversarial perspective.

[0057] In step 320, the initial adversarial viewpoint is optimized based on the classification loss function to obtain an optimized initial adversarial viewpoint, so as to maximize the value of the classification loss function obtained by rendering the 2D image based on the optimized initial adversarial viewpoint, and the optimized initial adversarial viewpoint is used as the optimized adversarial viewpoint of the object to be identified.

[0058] In one embodiment, in order to learn the distribution of adversarial viewpoints, a classification loss function can be obtained based on the rendered 2D image under the initial adversarial viewpoint, and the initial adversarial viewpoint can be optimized based on the classification loss function to obtain an optimized initial adversarial viewpoint, so as to maximize the value of the classification loss function obtained based on the rendered 2D image under the optimized initial adversarial viewpoint, and the optimized initial adversarial viewpoint is used as the optimized adversarial viewpoint of the object to be identified.

[0059] In one embodiment, the optimization problem can be solved using the following formula (2):

[0060]

[0061] Where f represents the image classification model; y represents the true category of the object to be identified; R(v) represents the rendered 2D image under the initial adversarial viewpoint v; L represents the classification loss function (e.g., cross-entropy loss); λ represents the hyperparameter; H(p(v)) = -E p(v)[logp(v)] represents the entropy of the distribution, where p(v) represents the distribution under the initial adversarial perspective v.

[0062] In application, by continuously optimizing the initial adversarial perspective, an optimized initial adversarial perspective can be obtained, maximizing the classification loss function value obtained from rendering 2D images based on the optimized initial adversarial perspective. This optimized initial adversarial perspective can then serve as the optimized post-adversarial perspective for the object to be identified. The optimized post-adversarial perspective can deceive the visual perception model with a higher success rate, thereby improving the accuracy of the analysis of the visual perception model's viewpoint robustness based on the optimized post-adversarial perspective and reducing the operational cost of evaluating the visual perception model's viewpoint robustness.

[0063] It should be noted that, in this embodiment, the entropy regularization term can effectively enhance the diversity of adversarial perspectives. This can effectively prevent deviations in camera pose in the real world, reduce the discrepancy between the neural radiation field rendered image and the real image, and improve the attack effectiveness against unknown black-box models.

[0064] To optimize the distribution of adversarial examples more efficiently, the perspective robustness analysis method provided in this invention uses a diagonal Gaussian distribution to model the adversarial perspective, and proposes an algorithm that combines search gradient and reparameterization to solve for the parameters of the distribution.

[0065] In yet another exemplary embodiment of the present invention, the optimization of the initial adversarial perspective based on the classification loss function can be achieved in the following manner:

[0066] Based on the classification loss function, the initial adversarial perspective is optimized using a Gaussian distribution.

[0067] In one embodiment, the distribution of the viewpoint can be parameterized using a Gaussian distribution. In application, the viewpoint parameter v = [ψ, θ, φ, Δ]. x ,Δ y ,Δ z The range of ] is subject to certain limitations and can be denoted as [v] min ,v max To define a reasonable distribution within this interval, this embodiment can employ a parameter transformation method, as shown in formula (3):

[0068] v=a·tanh(u)+b,u~N(μ,σ 2 I) (3)

[0069] in, u~N(μ,σ 2 I) represents a Gaussian distribution.

[0070] Furthermore, the optimization problem shown in equation (2) can be transformed into the problem shown in equation (4):

[0071]

[0072] To solve the optimization problem shown in Equation (4), it is necessary to calculate the gradient of the classification loss function L with respect to the parameters μ and σ.

[0073] In one embodiment, the gradient can be calculated using a combination of gradient search and reparameterization. The gradient can be represented by equations (5) and (6):

[0074]

[0075]

[0076] in, This represents a standard Gaussian distribution.

[0077] Furthermore, based on the solved parameters μ and σ, the initial adversarial perspective can be optimized according to formula (3) to obtain the optimized initial adversarial perspective.

[0078] It should be noted that in this embodiment, a Gaussian distribution can be used to model the adversarial perspective distribution, but it is not limited to a Gaussian distribution. In another example, more representative distribution forms such as mixture distributions and probability diffusion models can also be used.

[0079] To further introduce the perspective robustness analysis method provided by this invention, the following will be combined with... Figure 4 Please provide an explanation.

[0080] Figure 4 This is a schematic diagram illustrating an application scenario of the robustness analysis method for perspective provided by the present invention.

[0081] In an exemplary embodiment of the present invention, combined with Figure 4 Given the initial position of the camera, we first rotate it by an angle (ψ, θ, φ). Then we translate it by a distance (Δφ). x ,Δ y ,Δ z After the camera changes, a new shooting perspective can be obtained and rendered. From this perspective, a 2D photograph rendered from the neural radiation field can be obtained, denoted as v = [ψ, θ, φ, Δ]. x ,Δ y ,Δ z The rendered 2D photograph (corresponding to the rendered 2D image) can be represented as R(v), where R represents the entire rendering process.

[0082] Furthermore, a classification loss function is obtained based on the rendered 2D image R(v) from the initial adversarial perspective. This classification loss function is then used to optimize the initial adversarial perspective, resulting in an optimized initial adversarial perspective. This optimized initial adversarial perspective maximizes the value of the classification loss function obtained from the rendered 2D image under the optimized initial adversarial perspective. This optimized initial adversarial perspective is then used as the optimized adversarial perspective for the object to be identified. The optimized adversarial perspective can deceive the visual perception model with a higher success rate, thereby improving the accuracy of the analysis of the visual perception model's viewpoint robustness based on the optimized adversarial perspective and reducing the operational cost of evaluating the viewpoint robustness of the visual perception model.

[0083] In another example, based on the perspective robustness analysis method and image classification models ResNet-50 and ViT-B16 provided by this invention, the attack success rate results are shown in Table 1.

[0084] Table 1 Attack success rate results under different optimized adversarial perspectives

[0085]

[0086]

[0087] As shown in Table 1, the perspective robustness analysis method provided by this invention can achieve a higher attack success rate, meaning that the optimized adversarial perspective can deceive the visual perception model with a higher success rate. Furthermore, the adversarial examples obtained based on this invention also have better attack effects on real-world images.

[0088] As described above, the viewpoint robustness analysis method provided by this invention obtains a three-dimensional representation of the object to be identified based on multiple two-dimensional images of the object, obtains a rendered two-dimensional image under an initial adversarial view based on the three-dimensional representation, and determines an optimized adversarial view of the object to be identified based on the rendered two-dimensional image under the initial adversarial view. This optimizes the adversarial view so that it can deceive the visual perception model with a higher success rate, thereby improving the accuracy of the analysis of viewpoint robustness of the visual perception model based on the optimized adversarial view and reducing the operational cost of evaluating the viewpoint robustness of the visual perception model.

[0089] Based on the same concept, the present invention also provides a perspective robustness analysis device.

[0090] The viewpoint robustness analysis apparatus provided by the present invention will be described below. The viewpoint robustness analysis apparatus described below can be referred to in correspondence with the viewpoint robustness analysis method described above.

[0091] Figure 5 This is a schematic diagram of the perspective robustness analysis device provided by the present invention.

[0092] In an exemplary embodiment of the present invention, combined with Figure 5 As can be seen, the viewpoint robustness analysis device may include the first module 510 to the fourth module 540, and each module will be described below.

[0093] The first module 510 can be configured to acquire multiple two-dimensional images of the object to be identified;

[0094] The second module 520 can be configured to obtain a three-dimensional representation of the object to be identified based on multiple two-dimensional images;

[0095] The third module 530 can be configured to obtain a rendered 2D image from the initial adversarial perspective based on the 3D representation.

[0096] The fourth module 540 can be configured to render a 2D image based on the initial adversarial viewpoint, determine the optimized adversarial viewpoint of the object to be identified, and evaluate the viewpoint robustness of the visual perception model based on the optimized adversarial viewpoint.

[0097] In an exemplary embodiment of the present invention, the second module 520 may obtain a three-dimensional representation of the object to be identified based on multiple two-dimensional images in the following manner: based on multiple two-dimensional images, a three-dimensional representation of the object to be identified is obtained through a neural radiation field.

[0098] In an exemplary embodiment of the present invention, the third module 530 may obtain a rendered two-dimensional image from the initial adversarial perspective based on the three-dimensional representation in the following manner: determining the initial adversarial perspective; and obtaining the rendered two-dimensional image from the initial adversarial perspective through a neural radiation field based on the three-dimensional representation and the initial adversarial perspective.

[0099] In an exemplary embodiment of the present invention, the fourth module 540 may determine the optimized adversarial perspective of the object to be identified based on the rendered 2D image under the initial adversarial perspective in the following manner: obtaining a classification loss function based on the rendered 2D image under the initial adversarial perspective; optimizing the initial adversarial perspective based on the classification loss function to obtain an optimized initial adversarial perspective, so as to maximize the value of the classification loss function obtained based on the rendered 2D image under the optimized initial adversarial perspective, and using the optimized initial adversarial perspective as the optimized adversarial perspective of the object to be identified.

[0100] In an exemplary embodiment of the present invention, the fourth module 540 may optimize the initial adversarial perspective based on the classification loss function in the following manner:

[0101] Based on the classification loss function, the initial adversarial perspective is optimized using a Gaussian distribution.

[0102] Figure 6An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640. The processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a viewpoint robustness analysis method. This method includes: acquiring multiple two-dimensional images of the object to be identified; obtaining a three-dimensional representation of the object based on the multiple two-dimensional images; obtaining a rendered two-dimensional image under an initial adversarial viewpoint based on the three-dimensional representation; determining an optimized adversarial viewpoint of the object based on the rendered two-dimensional image under the initial adversarial viewpoint; and evaluating the viewpoint robustness of the visual perception model based on the optimized adversarial viewpoint.

[0103] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the viewpoint robustness analysis method provided by the above methods. The method includes: acquiring multiple two-dimensional images of an object to be identified; obtaining a three-dimensional representation of the object to be identified based on the multiple two-dimensional images; obtaining a rendered two-dimensional image under an initial adversarial viewpoint based on the three-dimensional representation; determining an optimized adversarial viewpoint of the object to be identified based on the rendered two-dimensional image under the initial adversarial viewpoint; and evaluating the viewpoint robustness of the visual perception model based on the optimized adversarial viewpoint.

[0105] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the viewpoint robustness analysis method provided by the above methods. The method includes: acquiring multiple two-dimensional images of an object to be identified; obtaining a three-dimensional representation of the object to be identified based on the multiple two-dimensional images; obtaining a rendered two-dimensional image under an initial adversarial viewpoint based on the three-dimensional representation; determining an optimized adversarial viewpoint of the object to be identified based on the rendered two-dimensional image under the initial adversarial viewpoint; and evaluating the viewpoint robustness of the visual perception model based on the optimized adversarial viewpoint.

[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0107] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0108] It is further understood that although the operations are described in a specific order in the accompanying drawings in the embodiments of the present invention, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all the operations shown to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A perspective robustness analysis method, characterized in that, The perspective robustness analysis method includes: Acquire multiple 2D images of the object to be identified; Based on multiple two-dimensional images, a three-dimensional representation of the object to be identified is obtained; Based on the three-dimensional representation, a rendered two-dimensional image from the initial adversarial perspective is obtained; Based on the rendered 2D image under the initial adversarial viewpoint, an optimized adversarial viewpoint for the object to be identified is determined, and the viewpoint robustness of the visual perception model is evaluated based on the optimized adversarial viewpoint. Specifically, determining the optimized adversarial viewpoint for the object to be identified based on the rendered 2D image under the initial adversarial viewpoint includes: Based on the rendered 2D image from the initial adversarial perspective, a classification loss function is obtained; Based on the classification loss function, the initial adversarial viewpoint is optimized to obtain an optimized initial adversarial viewpoint, which maximizes the value of the classification loss function obtained from the rendered 2D image under the optimized initial adversarial viewpoint. This optimized initial adversarial viewpoint is then used as the optimized adversarial viewpoint for the object to be identified. The optimization problem is solved using the following formula: Where f represents the image classification model; y represents the true category of the object to be identified; R(v) represents the rendered 2D image under the initial adversarial viewpoint v; L represents the classification loss function; and λ represents the hyperparameter. Let represent the entropy of the distribution, where p(v) represents the distribution under the initial adversarial perspective v.

2. The perspective robustness analysis method according to claim 1, characterized in that, The process of obtaining a three-dimensional representation of the object to be identified based on multiple two-dimensional images specifically includes: Based on multiple two-dimensional images, a three-dimensional representation of the object to be identified is obtained through a neural radiation field.

3. The perspective robustness analysis method according to claim 1, characterized in that, The process of obtaining the rendered 2D image from the initial adversarial perspective based on the 3D representation specifically includes: Determine the initial adversarial perspective; Based on the three-dimensional representation and the initial adversarial perspective, a rendered two-dimensional image under the initial adversarial perspective is obtained through a neural radiation field.

4. The perspective robustness analysis method according to claim 1, characterized in that, The optimization of the initial adversarial perspective based on the classification loss function specifically includes: Based on the classification loss function, the initial adversarial perspective is optimized using a Gaussian distribution.

5. A perspective robustness analysis device, characterized in that, The view robustness analysis apparatus is used to implement the view robustness analysis method according to any one of claims 1 to 4, and the apparatus includes: The first module is used to acquire multiple two-dimensional images of the object to be identified; The second module is used to obtain a three-dimensional representation of the object to be identified based on multiple two-dimensional images; The third module is used to obtain a rendered 2D image from the initial adversarial perspective based on the 3D representation. The fourth module is used to determine the optimized adversarial view of the object to be identified based on the rendered 2D image under the initial adversarial view, and to evaluate the view robustness of the visual perception model based on the optimized adversarial view.

6. The perspective robustness analysis device according to claim 5, characterized in that, The second module obtains a three-dimensional representation of the object to be identified based on multiple two-dimensional images in the following manner: Based on multiple two-dimensional images, a three-dimensional representation of the object to be identified is obtained through a neural radiation field.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the perspective robustness analysis method as described in any one of claims 1 to 4.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the perspective robustness analysis method as described in any one of claims 1 to 4.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the perspective robustness analysis method as described in any one of claims 1 to 4.

Images