A multi-view target sample and scene image generation method

Abstract

The application discloses a multi-view target sample and scene image generation method and belongs to the technical field of image processing. Mainly through the neural network, a target space neural radiation field is constructed; based on the space neural radiation field, a target 2D image under different views is obtained through volume rendering; then, a mixed Gaussian distribution is taken as a view distribution prior, a difficult view in the target 2D image is learned based on a distribution minimum-maximum optimization problem; finally, the difficult view is fused with a preset scene image. The application can effectively improve the recognition ability of the difficult view, and through the recognized different target weak views, the target view diversity of the image sample set is significantly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of digital image processing, and more particularly to a method for generating multi-view target samples and scene images. Background Art

[0002] In the fields of artificial intelligence and computer vision, image generation technology is becoming a key technology.

[0003] In practical applications, collecting sufficient and comprehensive image samples is often difficult, time-consuming, and labor-intensive, and can also lead to data missing or imbalance. Generating diverse scene images can expand the dataset and enrich the data distribution, thereby improving the generalization capabilities of downstream models and enabling them to perform better across diverse tasks and application scenarios.

[0004] Currently, image generation models, such as generative adversarial networks (GANs) and variational autoencoders (VAEs), often find it difficult to fully demonstrate sufficient diversity when generating images, especially in terms of target perspective coverage. This results in the generated image samples being unable to fully cover the various target perspectives that may appear in real scenes, causing the trained recognition model to exhibit obvious recognition defects under certain specific perspectives, thereby limiting the model's generalization ability and practical application effect. Summary of the Invention

[0005] In view of this, the present invention provides a method for generating multi-perspective target samples and scene images, which mainly searches for and determines difficult perspectives with weaker recognition performance based on the analysis of the difficulty of target recognition at different perspectives by a perception model; then, the targets at these perspectives are rendered and naturally integrated into the background image to generate a target scene image, thereby effectively improving the model's target recognition accuracy and generalization ability at difficult perspectives.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for generating multi-view target samples and scene images, comprising:

[0008] Constructing the target space neural radiation field through neural network;

[0009] Based on the spatial neural radiation field, volume rendering is used to obtain 2D images of the target at different viewing angles;

[0010] Using a mixed Gaussian distribution as the distribution prior for perspective, we learn difficult perspectives in the target 2D image based on a minimax optimization problem based on the distribution.

[0011] Merge difficult perspectives with preset scene images.

[0012] Preferably, the neural network is represented as:

[0013]

[0014] Where F is approximated by a multilayer perceptron, represents the 3D position of the target in space, represents the viewing direction of unit norm, Represents RGB color, Represents voxel density.

[0015] Preferably, the neural network is trained with the goal of minimizing the total square error between rendered pixel values ​​and true pixel values; and position encoding and layered volume sampling strategies are used to improve performance.

[0016] Position encoding is to map the input three-dimensional space coordinates and view direction vectors using sine and cosine functions respectively, and to enhance the neural network's ability to express high-frequency details by constructing a high-dimensional periodic representation.

[0017] The hierarchical volume sampling strategy involves obtaining a set of initial sampling points uniformly along the direction of the light; using a coarse network to estimate the voxel density of the initial sampling points, and performing importance sampling based on the distribution of voxel density to generate new sampling points focused on high-contribution areas; the initial sampling points and the new sampling points are merged and fed into a fine network to calculate the final color output.

[0018] Preferably, obtaining 2D images of the target at different viewing angles through volume rendering based on the spatial neural radiation field includes:

[0019] Will Denote as the camera ray emitted from the camera origin o through the given pixel t on the image plane, d represents the direction vector from the origin o to the given pixel t; the color of the given pixel is expressed as:

[0020]

[0021] In the formula, the integration point were randomly selected through stratified sampling. It represents the color representation of a given pixel t under light q, M represents the number of sampling points, and m represents the index of the sampling point sequence. is the transmittance from the camera origin to the mth sampling point, indicating the proportion that is not absorbed, Represents the opacity function of the voxel in the volume rendering process, which is usually determined by the product of the volume density and the sampling interval, reflecting the ability of the point to hinder the propagation of light. , Indicates The voxel density at , Indicates The RGB color at is the distance between two adjacent points, Indicates the sampling point with sequence number j in [1:m-1], is the distance between two adjacent points.

[0022] Preferably, the distribution-based minimax optimization problem is expressed as:

[0023]

[0024] Where N represents the number of targets, i represents the target index, and p(v i ) represents the potential distribution of the i-th object, represents the probability distribution p(v i ) under the expected value, represents the cross entropy loss, f W represents the target classifier with parameter w, Represents the viewing angle v i The corresponding rendered image, y i Represents the viewing angle v i The corresponding real image, represents the hyperparameter coefficient, Denotes the distribution p(v i ) to avoid degradation problems and help capture more diverse perspectives, and .

[0025] Preferably, learning difficult perspectives in a target 2D image includes: obtaining a natural gradient of the classification loss for a query of the target classifier, and directly computing the true gradient for the entropy regularization term,

[0026] Preferably, a Monte Carlo method is used to approximate the expectation in the gradient calculation, and the distribution parameters of each Gaussian component are optimized using iterative gradient ascent.

[0027] Preferably, the mixed Gaussian distribution is used as the distribution prior of the perspective, including:

[0028] The object potential distribution p(v) is parameterized by a mixture of K Gaussian components, and the random variable transformation method is used to ensure that the support interval of p(v) is contained in [v min , v max ] range, the formula is expressed as:

[0029]

[0030] Where v represents the perspective transformation parameter, , is the rotation angle, is the translation distance, a and b represent constant terms, , , vmax , v min respectively represent the maximum and minimum values of the transformation parameters, u represents an intermediate variable, follows a Gaussian distribution, tanh(u) represents a tangent function for converting u to a finite range, represents a p(v) distribution parameter, , represents the weight of the kth Gaussian component, represents the mean of the kth Gaussian distribution, represents the standard deviation of the kth Gaussian distribution, K represents the number of mixed Gaussian components, represents a normal distribution, represents the variance size, is a standard normal distribution, represents 6 dimensions, representing the space corresponding to 6 view variables respectively.

[0031] Preferably, a latent one-hot vector is introduced to determine the Gaussian component to which the sampled view angle belongs; wherein,

[0032]

[0033] In the formula, represents an element in the one-hot vector.

[0034] At this time, the internal maximization part can be rewritten as:

[0035]

[0036] Preferably, u is reparameterized as follows:

[0037]

[0038] In the formula, represents a standard Gaussian normal distribution, r is a sampled value following a standard normal distribution, is the power of , is the power of

[0039] , K Gaussian components are calculated together to obtain a sample u by reparameterization.

[0040] Further, the target under the difficult view angle is selected and rendered by NeRF, and then fused with the corresponding scene, so as to achieve the generation of multi-view target and scene images, and provide sufficient data support for subsequent visual task training and performance optimization.

[0041] Compared with the prior art, the advantages of the present application include:

[0042] 1. By establishing a difficult view angle distribution, a plurality of different target weak view angles can be identified to achieve effective target view angle diversity.

[0043] 2. The rendered target and background under the difficult view angle are reasonably fused to form a multi-view target scene image, thereby effectively improving the target view angle diversity of the image sample set. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0045] Figure 1 The flow chart of the multi-view target sample and scene image generation method of the present application;

[0046] Figure 2 The schematic diagram of the difficult view angle learning process of the present application;

[0047] Figure 3 The schematic diagram of the view angle transformation process of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0049] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the scope of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0050] The embodiments of the present application disclose a multi-view target sample and scene image generation method to expand and perfect the image sample set.

[0051] The present application first uses a neural radiance field (NeRF) to model objects in the physical world to solve the problem of 3D representation of objects in the physical world. In the case of obtaining a trained NeRF representation of an object, the purpose of the present application is to estimate difficult target perspectives so that the images rendered at the perspective can mislead the target visual recognition model, thereby achieving the purpose of perspective difficulty evaluation.

[0052] Specifically, the present application captures a variety of difficult perspectives by optimizing the distribution of difficult perspectives, and finds various difficult perspectives that can deceive the model to produce errors.

[0053] Finally, the target various difficult perspectives are rendered and reasonably integrated into the background image to finally form a multi-perspective target scene, providing data support for subsequent visual tasks.

[0054] In order to achieve the above-mentioned purpose of the application, the present application proposes a multi-perspective target sample and scene image generation method, the steps of which include:

[0055] constructing a target space neural radiance field through a neural network;

[0056] obtaining target 2D images at different perspectives based on volume rendering of the space neural radiance field;

[0057] using a mixture Gaussian distribution as the distribution prior of the perspective, learning the difficult perspectives in the target 2D images based on the distribution-based minimax optimization problem;

[0058] fusing the difficult perspectives with a preset scene image.

[0059] Specific reference Figure 1 , Figure 1 is a flowchart of the multi-perspective target sample and scene image generation method;

[0060] In one embodiment,

[0061] constructing a target space neural radiance field through a neural network; and using a neural radiance field (NeRF) to model target objects in the physical world to solve the problem of 3D representation of the target objects in the physical world.

[0062] In the present embodiment, the neural network is represented as:

[0063]

[0064] In the formula, F is a multi-layer perceptron (MLP) for approximation, which is a 3D position in space and an observation direction with a unit norm As input, its output includes RGB color and voxel density .

[0065] To train the radiance field network F, NeRF minimizes the total squared error between the rendered pixel values and the true pixel values in a set of images with known camera poses; and adopts position encoding and hierarchical volume sampling strategy to improve performance.

[0066] Position encoding is to map the input three-dimensional space coordinates and view direction vectors respectively by sine and cosine functions, and to construct high-dimensional periodic representation to enhance the expression ability of neural network to high-frequency details;

[0067] For example, for an input scalar p, its position encoding can be represented as:

[0068]

[0069] Where L represents the number of frequencies used. Concatenating the encoded dimensions as the input of the network can improve the model's ability to analyze spatial positions.

[0070] The hierarchical volume sampling strategy first uniformly selects a set of initial sampling points along the light direction. Then, with the help of a coarse network with relatively simple structure and fast calculation, the voxel density of these initial sampling points is estimated. The coarse network, with its fast processing capability, performs importance sampling based on the voxel density distribution to determine the areas that contribute more to the final result, and then generates new sampling points focusing on these high-contribution areas. Finally, the initial sampling points and the new sampling points are combined and input into a fine network with complex structure and strong expression ability. The fine network uses its rich parameters and multi-layer structure to comprehensively consider various factors and finely process the sampling points, thereby calculating the final accurate and detailed color output.

[0071] Through the above strategy, NeRF can more effectively utilize sampling resources, capture key contribution areas in light propagation, and implicitly learn the three-dimensional structure of target objects under the multi-view consistency constraint between calibrated images, thereby facilitating the realization of high-quality new view rendering. Figure 1

[0072] In one embodiment,

[0073] Based on the spatial neural radiance field, a 2D image of the target at different viewing angles is obtained by volume rendering; the present application can calculate the color of each pixel by volume rendering, and render the volume radiance field into a 2D image from a specific viewing angle.

[0074] Firstly, the ​Denoted as a camera ray emitted from the camera origin o through a given pixel t on the image plane, d represents the direction vector from the origin o to the given pixel t; the ray passes through the entire dataset and records the attribute information of the voxels encountered along the way. When the ray passes through multiple voxels, their cumulative contribution is calculated based on the transparency and color of the voxels in order to correctly handle the front-back occlusion relationship.

[0075] In this embodiment, the color of a given pixel is represented as:

[0076]

[0077] In the formula, the integration point were randomly selected through stratified sampling. It represents the color representation of a given pixel t under light q, M represents the number of sampling points, and m represents the index of the sampling point sequence. is the transmittance from the camera origin to the mth sampling point, indicating the proportion that is not absorbed, Represents the opacity function of the voxel in the volume rendering process, which is usually determined by the product of the volume density and the sampling interval, reflecting the ability of the point to hinder the propagation of light. , Indicates The voxel density at , Indicates The RGB color at is the distance between two adjacent points, Indicates the sampling point with sequence number j in [1:m-1], is the distance between two adjacent points.

[0078] By specifying different light directions at a certain viewing angle, this application can render an image at that viewing angle pixel by pixel. In other words, the neural radiation field can generate a realistic image at any viewing angle.

[0079] In one embodiment,

[0080] Based on the mini-max problem of distribution, the mixed Gaussian distribution is used as the distribution prior of the perspective to learn the difficult perspectives in the target 2D image; for specific reference Figure 2 ,Difficult perspective means that the image rendered under this perspective ,can mislead the target visual recognition model for generating a new target ,image.

[0081] This application learns the potential distribution p(v i ) to characterize diverse perspectives, and adopt Gaussian Mixture Distribution as the distribution prior of perspectives to capture more comprehensive difficult perspectives, where v represents the transformation parameter and i represents the i-th target.

[0082] To learn these difficult view distributions, the prediction performance of the target classifier on these difficult target views is enhanced through an outer minimization procedure, which can be formulated as a distribution-based min-max optimization problem:

[0083]

[0084] where N denotes the number of targets, i denotes the target index, p(v i ) denotes the latent distribution of the i-th object transformation parameter, denotes the expectation under the probability distribution p(v i ), denotes the cross-entropy loss, f W denotes the target classifier with parameters w, denotes the rendered image corresponding to the view v i , i denotes the real image corresponding to the view v i , denotes the Lagrange multiplier, denotes the entropy of the distribution p(v i ) to avoid degeneration and help capture more diverse views, and .

[0085] Further, the inner maximization aims to learn the distribution of difficult views under entropy regularization, while the outer minimization aims to optimize the target classifier parameters by minimizing the expected loss on the most difficult view distribution in the worst case, thus effectively reducing the reality gap between real objects and their neural representations. This distribution can cover a variety of difficult views and can significantly improve generalization performance.

[0086] In an exemplary embodiment, the target classifier f in visual recognition can make the same prediction when given two views of the same object, as follows:

[0087]

[0088] where, and are two images taken from arbitrary views v1, v2, achieving view invariance by learning difficult views in the worst case.

[0089] In this embodiment, v denotes the transformation parameters of the object view, as shown in Figure 3 For a predefined coordinate system xyz and an initialized camera pose, first perform a 3D rotation of the camera around the z-y-x axes in the order of Tait-Bryan angles (also known as yaw, pitch, and roll angles), with the rotation angles denoted as Then, the camera position is translated along the three axes by a distance noted After the camera transformation, a new view of the object is obtained. The transformation parameters of the object view are noted as a set of 6-dimensional vectors The image rendered at this parameter is noted where represents the rendering process of the neural radiance field.

[0090] To further optimize the above technical solution, a view transformation parameterization representation is used, and an additional multi-view training data set is introduced to learn a view-invariant representation therefrom.

[0091] When a data set including N objects and the corresponding true labels are given, it is assumed that each object has a set of multi-view images available.

[0092] Using these images, an Instant-NGP algorithm is used to train a NeRF model for each object to obtain a neural renderer that can synthesize new images from any view of the object.

[0093] Although the present application introduces an efficient Instant-NGP for obtaining a NeRF representation of an object, the interior maximization still requires a large number of rendering steps to obtain the gradient and undergoes multiple gradient descent steps to achieve convergence. In order to accelerate the training, a random update strategy for interior maximization. First, perform a complete interior optimization, and use a pre-trained image classifier to generate a preliminary difficult view distribution for all objects. In each fine-tuning step thereafter, only the distribution parameters of a randomly selected object in each class are updated, while the distribution parameters of other objects remain unchanged. It should be noted that all objects can be fully optimized under multiple fine-tuning rounds.

[0094] In a preferred embodiment,

[0095] To solve the interior maximization problem, the distribution of difficult views can be parameterized, i.e., learning a Gaussian mixture distribution of difficult views to cover more local maxima in the adversarial training that can be generalized.

[0096] For simplicity, since the recognition algorithm of all object objects is the same, the present application omits the subscript i, then parameterizes the distribution p(v) by a mixture of K Gaussian components, and uses a random variable transformation method to ensure that the support interval of p(v) is contained in the range [v min , v max ] as follows:

[0097]

[0098] where v denotes the view angle transformation parameter, , is the rotation angle, is the translation distance, a, b denote constant terms, , , v max , v min denote the maximum and minimum values of the transformation parameter respectively, u denotes an intermediate variable following a Gaussian distribution, v is obtained by a proper standardization transformation on u, tanh(u) denotes the hyperbolic tangent function used to transform u to a finite range, denotes the p(v) distribution parameter, , denotes the weight of the k-th Gaussian component, denotes the mean of the k-th Gaussian distribution, denotes the standard deviation of the k-th Gaussian distribution, K denotes the number of Gaussian components, denotes the normal distribution, denotes the variance size, is the standard normal distribution, denotes the 6 dimensions, representing the 6 view angle variables corresponding to the space respectively.

[0099] Since the current probability density function is in the form of summation, which makes it difficult to calculate the gradient. Therefore, the present application introduces a latent one-hot vector to determine which Gaussian component the sampled view angle belongs to, and follows a multinomial distribution, whose probability is determined by , and its probability distribution is denoted as .

[0100] After introducing the latent variable , the can be denoted as a multiplication form, which is rewritten as , and .

[0101] This helps to calculate the gradient with respect to the distribution parameters .

[0102] By using the above transformation, the inner maximization part in the distribution-based min-max optimization problem can be rewritten as:

[0103]

[0104] Further, the present application adopts a gradient-based method to optimize the distribution parameters To backpropagate the gradient from the random samples to the distribution parameters, the reparameterization trick is continued to be adopted. Specifically, u is reparameterized as:

[0105]

[0106] By reparameterization, the gradient of the loss function in the above equation with respect to the distribution parameters can be computed.

[0107] In one specific embodiment, the present application adopts the natural evolution strategy (NES), i.e., only using the natural gradient of the classification loss obtained by querying access to the target classifier; for the entropy regularization term, its real gradient can be directly calculated.

[0108] Therefore, the gradients of the rewritten objective function with respect to the distribution parameters , and can be derived as follows:

[0109]

[0110]

[0111] In the equation, represents the expected value of r subject to the standard normal distribution, represents the kth element in the one-hot vector, represents the cross-entropy loss, represents the classification loss calculated after the final cross-entropy formula.

[0112] Further, the present embodiment uses the Monte Carlo method to approximate the expectation in the gradient calculation, and uses iterative gradient ascent to optimize the distribution parameters of each Gaussian component. After each iteration, the is normalized to satisfy .

[0113] In one embodiment,

[0114] The difficult view is fused with the preset scene image.

[0115] After obtaining the set of multiple peak parameters , the optimal difficult view distribution set and the mean of the distribution are obtained. It can be understood that the optimal difficult view set, as shown in Figure 1 , contains various situations, including common natural views, and the target is obtained by sampling and rendering in the optimal difficult view.

[0116] Further, define the background image , Indicates the size of the image and the number of channels, the source of the background image is diverse, which can be a naturally shot image, or an image generated by a generated model, etc.

[0117] Put the difficult perspective target image into the background image Y, the synthesized image will usually produce the phenomenon of visual target and background incoordination, and the method of image synthesis and harmonization needs to be added. The image harmonization model is denoted as H, and the harmonization process is as follows:

[0118]

[0119] More specifically, the following three ways can be used for image harmonization processing as follows:

[0120] 1) Alpha channel mixing is to weight the target image and the background image, and the transparency (Alpha channel) determines the fusion degree of the target image and the background image. The fusion formula is as follows:

[0121]

[0122] Preferably, wherein, is the pixel value of the synthesized image, is the transparency channel, and are the color values of the background image and the target image at the pixel .

[0123] 2) A Poisson equation can also be used to ensure smooth transition of the image on the boundary and texture. The target area in the background is , the boundary is , and the pixel in the target area is To let X be the target image and be the Laplacian operator, the formula is as follows:

[0124]

[0125] That is, the Laplace filter result of X is consistent with the Laplace filter result of , and the boundary value of is consistent with the boundary value of Y. Solving the Poisson equation can obtain the final synthesized image .

[0126] 3) A pre-trained diffusion model such as PHDiffusion can also be used for harmonization processing, and the target and background images are input for image synthesis.

[0127] Finally, the target perspective diversity of the image is effectively enriched, and more realistic scene image samples are ultimately generated to provide data support for other tasks.

[0128] This paper uses NeRF to digitally represent real-world 3D objects and performs optimization in the search space of view parameters to capture the Gaussian distribution of viewpoints at which object recognition is difficult, thereby enabling an assessment of the view difficulty of the visual model. Furthermore, NeRF is used to select and render objects at difficult viewpoints and fuse them with the corresponding scene, achieving the goal of generating multi-view object and scene images, providing sufficient data support for subsequent training and performance optimization of visual tasks.

[0129] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0130] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating multi-view target samples and scene images, characterized in that: Constructing the target space neural radiation field through neural network; Based on the spatial neural radiation field, volume rendering is used to obtain 2D images of the target at different viewing angles; Using a mixed Gaussian distribution as the distribution prior for perspective, we learn difficult perspectives in the target 2D image based on a minimax optimization problem based on the distribution. The distribution prior from the perspective of mixed Gaussian distribution includes: The object potential distribution p(v) is parameterized by a mixture of K Gaussian components, and the random variable transformation method is used to ensure that the support interval of p(v) is contained in [v min , v max ] range, the formula is expressed as: Where v represents the perspective transformation parameter, v = [ψ, θ, φ, Δ x , Δ y , Δ z ], (ψ, θ, φ) is the rotation angle, (Δ x , Δ y , Δ z ) is the translation distance, a and b are constant terms, a=(v max -v min ), b=(v max +v min ), v max 、v min They represent the maximum and minimum values ​​of the transformation parameters, u represents the intermediate variable, which follows the Gaussian distribution, tanh(u) represents the tangent function, which is used to transform u to a finite range, Ψ represents the p(v) distribution parameter, represents the weight of the k-th Gaussian component, represents the mean of the k-th Gaussian distribution, represents the standard deviation of the kth Gaussian distribution, K represents the number of mixed Gaussian components, represents a normal distribution, Indicates the size of the variance, I is the standard normal distribution, Represents the space corresponding to the perspective variable parameters; The distribution-based minimax optimization problem is expressed as: Where N represents the number of targets, i represents the target index, and p(v i ) represents the potential distribution of the i-th object, represents the probability distribution p(v i ) under the expected value, represents the cross entropy loss, f W represents the target classifier with parameter w, Represents the viewing angle v i The corresponding rendered image, y i Represents the viewing angle v i The corresponding real image, λ represents the hyperparameter coefficient, Denotes the distribution p(v i )’s entropy; Merge difficult perspectives with preset scene images.

2. The generation method according to claim 1, characterized in that The neural network is represented as: F:(x,d)→(c,τ) Where F is approximated by a multilayer perceptron, represents the 3D position of the target in space, represents the observation direction of unit norm, c∈[0,1] 3 Represents RGB color, represents the voxel density, represents three-dimensional space, represents the set of positive real numbers, [0, 1] 3 Represents three components, each of which ranges from 0 to 1.

3. The generation method according to claim 1, characterized in that The neural network is trained with the goal of minimizing the total squared error between rendered pixel values ​​and true pixel values; and position encoding and layered volume sampling strategies are used to improve performance.

4. The generation method according to claim 1, characterized in that Based on the spatial neural radiation field, volume rendering is used to obtain 2D images of the target from different perspectives, including: Let q(t)=o+td be the camera ray emitted from the camera origin o through the given pixel t on the image plane, d represents the direction vector from the origin o to the given pixel t; the color of the given pixel is represented as: Where, It represents the color representation of a given pixel t under light q, M represents the number of sampling points, m represents the index of the sampling point sequence, T(t m ) is the transmittance from the camera origin to the mth sampling point, indicating the proportion of non-absorption, α(·) represents the opacity function of the voxel during volume rendering, τ(t m ) indicates that in r(t m ), c(t m ) indicates that in q(t m ) at RGB color, δ m =t m+1 -t m is the distance between two adjacent points, t j represents the sampling point with sequence number j in [1:m-1], δ j =t j+1 -t j is the distance between two adjacent points.

5. The generation method according to claim 1, characterized in that Learning difficult views of the target 2D image includes obtaining the natural gradient of the classification loss for queries to the target classifier and directly computing the true gradient for the entropy regularization term.

6. The generation method according to claim 1, characterized in that The Monte Carlo method is used to approximate the expectation in the gradient calculation, and the distribution parameters of each Gaussian component are optimized using iterative gradient ascent.

7. The generation method according to claim 1, characterized in that The potential one-hot vector Γ is introduced to determine the Gaussian component to which the sampling perspective belongs; Γ=[γ1,...,γ K ] Where, γ K Represents an element in a one-hot vector.

8. The generation method according to claim 7, characterized in that Reparameterize u as follows: Where, represents the standard Gaussian normal distribution, r represents the sampling value that obeys the standard normal distribution, is μ k γ k Power, is σ k γ k Power.

Images