A geometrically-aware human body radiation modeling method and system for live broadcast scenes

Abstract

The present invention discloses a method and system for geometrically perceiving human body radiation modeling for live broadcast scenes, the method comprising: obtaining human body geometry embedding according to a sample image and a parameterized human body model; determining a sample point corresponding to the sample image in a target space, and obtaining a first pixel alignment feature according to the sample point and the sample image; obtaining a second pixel alignment feature according to the first pixel alignment feature, human body geometry embedding, and a pixel alignment feature attention module, and obtaining human body geometry features according to the first pixel alignment feature and a geometry feature attention module; connecting the human body geometry embedding, the second pixel alignment feature, and the human body geometry features to obtain training samples, training a fully connected neural network, and obtaining a neural radiation field model; inputting a sparse image to be reconstructed into the neural radiation field model to generate multiple human body new posture images. The present invention can generate high-fidelity new postures and new views of 3D human avatars, and can be widely used in the field of three-dimensional reconstruction technology.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional reconstruction technology, and in particular to a method and system for geometrically perceiving human body radiation modeling for live broadcast scenes. Background Art

[0002] Virtual character live broadcast is a new live broadcast method that combines virtuality and reality. It uses advanced AI technology to enable virtual characters to interact and perform in real time, bringing a new audio-visual experience to the audience. It is widely used in e-commerce live broadcast, entertainment live broadcast, online education, virtual press conferences and other fields. In virtual character live broadcast, NeRF technology can be used to generate highly realistic virtual character models. Human NeRF focuses on using the Skinned Multi-Person Linear (SMPL) model as additional geometric prior information to reconstruct 3D human avatars from a small amount of videos or images. However, existing human NeRF technology faces challenges in accurately representing human geometry and clothing texture details. In particular, when synthesizing novel pose images that are different from the input view, the deviation caused by the spatial transformation process and the image self-occlusion problem will cause large areas of black artifacts in the reconstructed 3D human avatar. Summary of the invention

[0003] To solve the above technical problems, the purpose of the present invention is to provide a geometrically-aware human radiation modeling method and system for live broadcast scenes, which can generate new postures and new views of high-fidelity 3D human avatars.

[0004] To achieve the above purpose, an embodiment of the present application proposes a geometrically-aware human body radiation modeling method for a live broadcast scene, comprising the following steps:

[0005] Acquire a sample image, and then obtain a human body geometric embedding according to the sample image and a preset parameterized human body model;

[0006] Determine a sample point corresponding to the sample image in the target space, and then obtain a first pixel alignment feature according to the sample point and the sample image;

[0007] Obtain a second pixel alignment feature according to the first pixel alignment feature, the human body geometry embedding, and a preset pixel alignment feature attention module, and obtain a human body geometry feature according to the first pixel alignment feature and a preset geometry feature attention module;

[0008] Connecting the human body geometry embedding, the second pixel alignment feature, and the human body geometry feature to obtain a training sample, and then training a fully connected neural network according to the training sample to obtain a neural radiation field model;

[0009] A sparse image to be reconstructed is obtained, and the sparse image to be reconstructed is input into the neural radiation field model to generate a new posture image of the human body and a new perspective image of the human body.

[0010] In some embodiments, the step of obtaining a human body geometric embedding according to the sample image and a preset parameterized human body model specifically includes:

[0011] Obtaining a first query point according to the sample image;

[0012] Calculating the closest surface from the first query point to the parameterized human body model, and calculating a normal vector corresponding to the closest surface;

[0013] Calculating the local coordinates and the directed distance of the first query point on the nearest surface, and then obtaining the local position corresponding to the first query point according to the local coordinates and the directed distance;

[0014] A mapping of the first query point on the nearest surface is obtained, and then the human body geometric embedding is obtained according to the normal vector, the local position and the mapping.

[0015] In some embodiments, the step of obtaining a first pixel alignment feature according to the sample point and the sample image specifically includes:

[0016] Transforming the sample point from the target space to a standard space to obtain a second query point, and then transforming the second query point from the standard space to an observation space to obtain a third query point;

[0017] Projecting the third query point onto the sample image to obtain a two-dimensional coordinate point corresponding to the third query point;

[0018] Extracting features from the sample image using a convolutional neural network to obtain a feature map;

[0019] The first pixel alignment feature is obtained according to the two-dimensional coordinate point and the feature map.

[0020] In some embodiments, the human body geometry embedding includes a first human body geometry embedding in a specification space and a second human body geometry embedding in an observation space, and the step of obtaining the second pixel alignment feature according to the first pixel alignment feature, the human body geometry embedding, and a preset pixel alignment feature attention module specifically includes:

[0021] Obtaining a camera direction, and then using the second human body geometric embedding as a first query agent, using the first pixel alignment feature, the camera direction, and the first human body geometric embedding as a first key matrix, and using the first pixel alignment feature and the second human body geometric embedding as a first value matrix;

[0022] A first attention weight is obtained according to the similarity between the first query agent and the first key matrix, and the first value matrix is ​​weighted according to the first attention weight to obtain the second pixel alignment feature.

[0023] In some embodiments, the step of obtaining the human body geometric features according to the first pixel alignment features and a preset geometric feature attention module specifically includes:

[0024] Setting a vertex and calculating a surface normal of the vertex in the observation space;

[0025] using the surface normal as a second query proxy, the camera direction and the first pixel-aligned feature as a second key matrix, and the first pixel-aligned feature as a second value matrix;

[0026] A second attention weight is obtained according to the similarity between the second query agent and the second key matrix, and then the second value matrix is ​​weighted according to the second attention weight to obtain the human body geometric features.

[0027] In some embodiments, the fully connected neural network includes a density multilayer perceptron and a color multilayer perceptron, and the step of training the fully connected neural network according to the training samples to obtain the neural radiation field model specifically includes:

[0028] Input the training samples into the fully connected neural network, obtain the predicted volume density corresponding to each sample point through the density multilayer perceptron, and obtain the predicted color corresponding to each sample point through the color multilayer perceptron;

[0029] Obtaining a predicted color value corresponding to each light ray by orthogonal integral calculation according to the predicted volume density and the predicted color;

[0030] Calculating a square loss according to the predicted color value and the true color value, and then obtaining a loss function according to the square loss, a preset perceptual loss, and a preset mask loss;

[0031] The fully connected neural network is updated and trained according to the loss function to obtain the neural radiation field model.

[0032] In some embodiments, the predicted color value is determined by the following formula:

[0033]

[0034] Where C(r) represents the predicted color value of light r, t n Indicates the preset near limit, t frepresents the preset far limit, σ(t) represents the predicted volume density within the t limit, and t n ≤t≤t f , σ(s) represents the predicted volume density within the s limit, and t n ≤s≤t, c(t,d) represents the predicted color along the light direction d within the limit of t.

[0035] To achieve the above purpose, another aspect of the embodiment of the present application proposes a geometrically-aware human body radiation modeling system for a live broadcast scene, comprising:

[0036] A human body geometry embedding acquisition module, used to acquire a sample image, and then obtain a human body geometry embedding according to the sample image and a preset parameterized human body model;

[0037] A pixel alignment feature acquisition module, used to determine a sample point corresponding to the sample image in the target space, and then obtain a first pixel alignment feature according to the sample point and the sample image;

[0038] a geometry-aware attention module, configured to obtain a second pixel-aligned feature according to the first pixel-aligned feature, the human body geometry embedding, and a preset pixel-aligned feature attention module, and to obtain a human body geometry feature according to the first pixel-aligned feature and a preset geometry feature attention module;

[0039] A model training module, used for connecting the human body geometry embedding, the second pixel alignment feature and the human body geometry feature to obtain a training sample, and then training a fully connected neural network according to the training sample to obtain a neural radiation field model;

[0040] The posture and viewpoint reconstruction module is used to obtain the sparse image to be reconstructed, input the sparse image to be reconstructed into the neural radiation field model, and generate a new posture image of the human body and a new viewpoint image of the human body.

[0041] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the geometrically-aware human body radiation modeling method for live broadcast scenes as described above is implemented.

[0042] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium used for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the geometrically-aware human radiation modeling method for live broadcast scenes as described above.

[0043] The beneficial effects of the present invention are as follows: the geometrically-aware human body radiation modeling method and system for live broadcast scenes of the present invention, on the one hand, accurately anchors the features to the represented point positions through the acquired human body geometry embedding, provides accurate geometric prior information for the learning of the neural radiation field model, can solve the problem of inaccurate feature correspondence caused by spatial transformation, and reduce spatial transformation deviation; on the other hand, the geometrically-aware attention mechanism composed of a pixel-aligned feature attention module and a geometric feature attention module accurately anchors the features to the spatial position, and fuses the pixel features and the geometric features, can effectively reduce shape artifacts and detail blur, solve the self-occlusion problem in sparse view input, and thus synthesize high-fidelity 3D human avatar new postures and new views from sparse views. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 A flow chart of the steps of a geometrically-aware human body radiation modeling method for a live broadcast scene provided by an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of centroid mapping provided by an embodiment of the present invention;

[0047] Figure 3 A framework overview diagram of a geometrically-aware human body radiation modeling method for a live broadcast scene provided by an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of qualitative results of the human body radiation modeling method provided by the embodiment of the present invention and the existing method on new view synthesis;

[0049] Figure 5 A schematic diagram of qualitative results of the human body radiation modeling method provided by the embodiment of the present invention and the existing method in synthesizing new postures;

[0050] Figure 6 A schematic diagram of qualitative results of ablation experiments on the ZJU-MoCap dataset provided by an embodiment of the present invention;

[0051] Figure 7 A schematic diagram of the structure of a geometrically-aware human body radiation modeling system for live broadcasting scenes provided by an embodiment of the present invention;

[0052] Figure 8 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0054] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0055] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0056] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0057] NeRF (Neural Radiance Fields) is a computer vision technology used to generate high-quality 3D reconstruction models. It uses neural networks to implicitly express 3D scenes, and optimizes NeRF through a series of known perspectives to represent continuous scenes, and finally renders a new perspective of the scene. This technology is particularly effective in processing reflective scenes, improving the efficiency of collecting and processing real-life 3D data.

[0058] The SMPL (Skinned Multi-Person Linear) model is a parametric human body model that can simulate the bulges and depressions of human muscles during limb movement, thereby avoiding surface distortion of the human body during movement. The SMPL model can accurately depict the morphology of human muscle stretching and contraction movements, providing high-quality geometric prior information for 3D human body reconstruction.

[0059] Neural Body is a method for generating free-view images from sparse multi-view video data using neural networks. It combines the SMPL model and light field theory to restore the appearance of human actors and generate high-quality view synthesis.

[0060] MPS-NeRF uses sparse views as input, fuses pixel-aligned features of the input views through a transformer, and maps query points to the observation space to achieve rendering of new human poses.

[0061] AniNeRF (Animatable Neural Radiance Fields) aims to use neural radiance fields to model dynamic human bodies. This technology combines the animatable properties of Neural Radiance Fields (NeRF) to generate highly realistic and animatable 3D human models.

[0062] Neural Novel Actor is a generalized animatable neural representation used to simulate the movements of human actors. It can capture and generate dynamic performances of human actors, thereby achieving more realistic and flexible animation effects.

[0063] KeypointNeRF is an image-based 3D volumetric avatar generation method that uses relative spatial encoding of keypoints for generalization.

[0064] Perceptual loss (VGG loss) is a loss function used in neural networks that measures the difference between generated images and target images based on the feature representation of a pre-trained VGG network (such as VGG19). This loss function can focus on the high-level features of the image rather than just the pixel-level details.

[0065] Density MLP (Density Multilayer Perceptron) is mainly used to predict the volume density of each point in space. Volume density determines the contribution of a point to the final image during the rendering process. Through training, density MLP can learn the mapping relationship from input features (such as spatial coordinates, viewing direction, etc.) to volume density, thereby generating high-quality rendering results.

[0066] The color MLP (color multi-layer perceptron) is responsible for predicting the color information of each point in space. The color MLP predicts the color of the corresponding point based on the input features (such as volume density, spatial coordinates, viewing direction, etc.), thereby ensuring that the rendered image is also accurate and realistic in color.

[0067] Virtual character live broadcast is a new live broadcast method that combines virtuality and reality. It uses advanced AI technology to enable virtual characters to interact and perform in real time, bringing a new audio-visual experience to the audience. It is widely used in e-commerce live broadcast, entertainment live broadcast, online education, virtual press conferences and other fields. In virtual character live broadcast, NeRF technology can be used to generate highly realistic virtual character models. Human NeRF focuses on using the Skinned Multi-Person Linear (SMPL) model as additional geometric prior information to reconstruct 3D human avatars from a small number of videos or images. In order to achieve the synthesis of new pose images, many NeRF methods use linear mixed skinning to allow human neural radiation fields to transform different pose spaces. However, the use of approximate skin weights may lead to inaccurate positions of query points during spatial transformation, resulting in poor correspondence of retrieved features. This leads to obvious blurring and lack of clothing wrinkle details in the synthesized image. Secondly, when the input image is sparse, NeRF may encounter feature inconsistencies caused by object self-occlusion and non-surface features, resulting in rendering artifacts.

[0068] To this end, an embodiment of the present invention proposes a geometrically-aware human body radiation modeling method for live broadcast scenes, which accurately anchors features to the represented point positions through the acquired human body geometry embedding, provides accurate geometric prior information for the learning of the neural radiation field model, and can solve the problem of inaccurate feature correspondence caused by spatial transformation and reduce spatial transformation deviation; on the other hand, the geometrically-aware attention mechanism composed of a pixel-aligned feature attention module and a geometric feature attention module accurately anchors features to spatial positions, and fuses pixel features and geometric features, which can effectively reduce shape artifacts and detail blur, solve the self-occlusion problem in sparse view input, and synthesize high-fidelity 3D human avatar new postures and new views from sparse views. The human body radiation modeling method can be applied to e-commerce live broadcast, entertainment live broadcast, online education, virtual press conferences and other fields, but is not limited to them.

[0069] Reference Figure 1 , Figure 1 A flowchart of a method for geometrically perceiving human body radiation modeling for a live broadcast scene provided by an embodiment of the present invention. An embodiment of the present invention provides a method for geometrically perceiving human body radiation modeling for a live broadcast scene, the method comprising steps S101 to S105:

[0070] S101, obtaining a sample image, and then obtaining a human body geometric embedding according to the sample image and a preset parameterized human body model;

[0071] Specifically, although human pose and spatial deformation can establish correspondences between points in three spaces and allow synthesis of human images with new poses, the inaccuracy of the approximate skin weights can lead to inaccurate correspondences of points. To solve this problem, the embodiment of the present invention proposes to impose accurate human geometry priors on the network. The SMPL model can provide an estimate of the shape of the naked human body and acts as a rough approximation in the embodiment of the present invention.

[0072] Among them, MPS-NeRF rendered images are used as sample images. This subset involves 30 randomly selected subjects, each with 30 scans in different poses, capturing a variety of images of different subjects and poses from 4 camera views. All SMPL model parameters are obtained from the dataset itself.

[0073] As an optional implementation, the step of obtaining a human body geometry embedding according to the sample image and the preset parameterized human body model can be specifically divided into the following steps S1011 to S1014:

[0074] S1011, obtaining a first query point according to the sample image;

[0075] S1012, calculating the closest surface from the first query point to the parameterized human body model, and calculating the normal vector corresponding to the closest surface;

[0076] S1013, calculating the local coordinates and the directed distance of the first query point on the nearest surface, and then obtaining the local position corresponding to the first query point according to the local coordinates and the directed distance;

[0077] S1014, obtaining a mapping of the first query point on the nearest surface, and then obtaining a human body geometric embedding according to the normal vector, the local position and the mapping.

[0078] Specifically, in order to obtain the basic human body geometry prior, the first query point x is calculated i The nearest surface F to the SMPL model l .like Figure 2 The figure shows a schematic diagram of centroid mapping. Based on the centroid mapping relationship, (u, v, s) is used to represent the first query point x i The local position of the surface F l The local coordinates on the surface, s represents the nearest surface F l The human body geometry is embedded in g i The design is as follows:

[0079] g i =G(xi ,M)=[u;v;s;n;x' i ]

[0080] Where n represents the nearest surface F l The normal vector, x' i represents the first query point x i At the nearest surface F l Mapping on , M represents the SMPL model. Using g i The information obtained can impose meaningful constraints on the neural radiation field model, effectively anchoring the points at exact locations on the human body, thereby reducing the effects caused by spatial transformation bias.

[0081] S102, determining a sample point corresponding to the sample image in the target space, and then obtaining a first pixel alignment feature according to the sample point and the sample image;

[0082] As an optional implementation, the step of obtaining the first pixel alignment feature according to the sample points and the sample image can be specifically divided into the following steps S1021 to S1024:

[0083] S1021, transforming the sample point from the target space to the standard space to obtain a second query point, and then transforming the second query point from the standard space to the observation space to obtain a third query point;

[0084] S1022, projecting the third query point onto the sample image to obtain a two-dimensional coordinate point corresponding to the third query point;

[0085] S1023, extracting features from the sample image through a convolutional neural network to obtain a feature map;

[0086] S1024. Obtain a first pixel alignment feature according to the two-dimensional coordinate point and the feature map.

[0087] Specifically, Figure 3 The figure shows an overview of the framework of the geometric perception human radiation modeling method for live broadcast scenes, where the blue, green, red, orange and yellow blocks in the figure represent the pixel alignment features f p , human body geometric characteristics υ , human body geometry embedding g i , vertex surface normal vector n and ray direction d. Given a sample point in target space The sample point is transformed into Convert from target space to canonical space to get the second query point Then, the second query point Deform to observation space and get the third query point And the third query point Project the input sample image back and obtain the first pixel alignment feature through the feature map extracted by the convolutional neural network

[0088] Canonical space refers to a standard volume space or range. Data can only be processed by a specific system or model (such as a neural network) if it falls within this range. Data outside the range may be unwarranted, meaningless, or invalid. This normalization helps ensure the consistency and processability of data.

[0089] The observation space is a coordinate system centered on the camera or observer, where the camera is usually located at the origin. Because the points in the observation space are still in three-dimensional space, they need to be mapped to the two-dimensional image plane for further processing. After projecting the third query point onto the sample image, a two-dimensional coordinate is obtained, which corresponds to a pixel position on the sample image.

[0090] S103, obtaining a second pixel alignment feature according to the first pixel alignment feature, human body geometry embedding, and a preset pixel alignment feature attention module, and obtaining a human body geometry feature according to the first pixel alignment feature and a preset geometry feature attention module;

[0091] As an optional implementation, the human body geometry embedding includes a first human body geometry embedding in a specification space and a second human body geometry embedding in an observation space. The step of obtaining the second pixel alignment feature according to the first pixel alignment feature, the human body geometry embedding and a preset pixel alignment feature attention module can be specifically divided into the following steps S1031 and S1032:

[0092] S1031, obtaining a camera direction, and then using the second human body geometry embedding as a first query agent, using the first pixel alignment feature, the camera direction, and the first human body geometry embedding as a first key matrix, and using the first pixel alignment feature and the second human body geometry embedding as a first value matrix;

[0093] S1032. Obtain a first attention weight according to the similarity between the first query agent and the first key matrix, and then weight the first value matrix according to the first attention weight to obtain a second pixel alignment feature.

[0094] Specifically, although the first pixel-aligned feature combined with the human body geometry embedding can provide accurate and clear texture details, the inherent self-occlusion problem caused by the sparse view input leads to some artifacts in the synthesized image. To solve this problem, the embodiment of the present invention proposes a pixel-aligned feature attention module. First, the second human body geometry embedding in the observation space is used to As a query proxy, the first pixel is aligned to the feature Camera Direction and the first human geometry embedding computed in canonical space As the key matrix. Given a query point in the canonical space The pixel alignment feature attention module of the embodiment of the present invention is designed as follows:

[0095] A p =Att p [Q P ,K p ,V p ]

[0096]

[0097]

[0098]

[0099] in, Represents a pixel-aligned feature linear layer that generates query, key, and value matrices. This configuration enables features to be accurately anchored in spatial locations and allows the fully connected neural network to learn which features are more important for solving the feature mismatch problem caused by self-occlusion.

[0100] As an optional implementation, the step of obtaining the human body geometric features according to the first pixel alignment feature and the preset geometric feature attention module can be specifically divided into the following steps S1033 to S1035:

[0101] S1033, setting vertices, and calculating surface normals of the vertices in the observation space;

[0102] S1034, using the surface normal as a second query proxy, using the camera direction and the first pixel alignment feature as a second key matrix, and using the first pixel alignment feature as a second value matrix;

[0103] S1035. Obtain a second attention weight according to the similarity between the second query agent and the second key matrix, and then weight the second value matrix according to the second attention weight to obtain human body geometric features.

[0104] Specifically, using only the second pixel-aligned features as network input cannot effectively represent the human body, resulting in inaccurate posture. This is because the lack of guidance from the semantic and geometric information of body parts leads to poor performance in certain postures. To solve this problem, the embodiment of the present invention projects the 6890 vertices on the SMPL model back to the input sample image, obtains the pixel-aligned features and integrates them through the geometric feature attention module. Given a vertex v l, the geometric feature attention module of the embodiment of the present invention is designed as follows:

[0105] A υ =Att υ [Q υ ,K υ ,V υ ]

[0106]

[0107]

[0108]

[0109] in, Vertex v in observation space l The surface normal of Represents a geometric feature linear layer that generates query, key, and value matrices. Subsequently, a sparse 3D feature volume is obtained through voxelization and passed to a sparse 3D convolution using SparseConvNet to expand the features to a meaningful region. Given a query point in the observation space Human body geometric features can be obtained through trilinear interpolation In this setting, the surface normal The closer to the camera direction The geometric characteristics of the human body The greater the weight.

[0110] S104, connecting the human body geometry embedding, the second pixel alignment feature and the human body geometry feature to obtain a training sample, and then training a fully connected neural network according to the training sample to obtain a neural radiation field model;

[0111] Specifically, the first human body geometry embedding, the second pixel alignment features, and the human body geometry features calculated in the canonical space are connected to obtain the final input of the neural radiation field model, and finally the volume density σ and color c of the queried point are output.

[0112] As an optional implementation, the fully connected neural network includes a density multilayer perceptron and a color multilayer perceptron. The step of training the fully connected neural network according to the training samples to obtain the neural radiation field model can be specifically divided into the following steps S1041 to S1044:

[0113] S1041, input the training sample into a fully connected neural network, obtain the predicted volume density corresponding to each sample point through a density multilayer perceptron, and obtain the predicted color corresponding to each sample point through a color multilayer perceptron;

[0114] S1042, obtaining a predicted color value corresponding to each light ray by orthogonal integral calculation according to the predicted volume density and the predicted color;

[0115] As a further optional implementation, the predicted color value is determined by the following formula:

[0116]

[0117] Where C(r) represents the predicted color value of light r, t n Indicates the preset near limit, t f represents the preset far limit, σ(t) represents the predicted volume density within the t limit, and t n ≤t≤t f , σ(s) represents the predicted volume density within the s limit, and t n ≤s≤t, c(t,d) represents the predicted color along the light direction d within the limit of t.

[0118] Specifically, in order to obtain the color of a two-dimensional image, the volume density and color of points along the light are rendered as pixel colors using classic volume rendering techniques. Given a viewpoint, the volume density and color are accumulated along the corresponding light, and the predicted color of each light is calculated by orthogonal integration r(t)=o+td, where r represents the light, o represents the camera starting point, d represents the light direction, and t represents the sampling interval.

[0119] S1043, calculating the square loss according to the predicted color value and the true color value, and then obtaining a loss function according to the square loss, the preset perceptual loss, and the preset mask loss;

[0120] S1044. Update and train the fully connected neural network according to the loss function to obtain a neural radiation field model.

[0121] Specifically, the embodiment of the present invention uses color loss to supervise the learning of the entire neural radiation field model. Given a real image, the loss is defined as the L2 loss (i.e., square loss) between the predicted color value and the real color value. Calculated by the following formula:

[0122]

[0123] Among them, Ω represents the light set of pixels on the target image, C * (r) represents the true color value of light r. The embodiment of the present invention also uses VGG loss in the training of the neural radiation field module. (i.e., perceptual loss) and mask loss To better capture high-frequency details. The loss function is defined as follows:

[0124]

[0125] Here, λ represents the loss weight, and in the embodiment of the present invention, λ1=λ2=0.5 is set.

[0126] S105, obtaining a sparse image to be reconstructed, inputting the sparse image to be reconstructed into a neural radiation field model, and generating a new posture image of the human body and a new perspective image of the human body.

[0127] Specifically, by inputting the sparse image to be reconstructed into the trained neural radiation field model, new human body posture images and new human body perspective images for live broadcast scenes can be obtained, which can solve the self-occlusion problem in sparse perspective input, improve the body details of the 3D human avatar and reduce blur, bringing a more realistic audio-visual experience to the audience.

[0128] The above describes the geometric perception human radiation modeling method for live broadcast scenes according to an embodiment of the present invention. It can be recognized that, compared with the prior art, the embodiment of the present invention, on the one hand, accurately anchors the features to the represented point positions through the acquired human body geometry embedding, provides accurate geometric prior information for the learning of the neural radiation field model, can solve the problem of inaccurate feature correspondence caused by spatial transformation, and reduce spatial transformation deviation; on the other hand, the geometric perception attention mechanism composed of the pixel alignment feature attention module and the geometric feature attention module accurately anchors the features to the spatial position, and fuses the pixel features and the geometric features, which can effectively reduce shape artifacts and detail blur, solve the self-occlusion problem in the sparse view input, and synthesize high-fidelity 3D human avatar new postures and new views from the sparse view, and can generate highly realistic virtual 3D human avatars in live broadcast scenes.

[0129] In order to further verify the accuracy of the embodiment of the present invention, the effect of the embodiment of the present invention is further described below in conjunction with experiments.

[0130] The experiment of the embodiment of the present invention is implemented in the PyTorch framework, using the Adam optimizer and setting the learning rate to 10-4. In addition, V=3 camera views are used for training and testing, and an hourglass network similar to KeypointNeRF is used as the backbone to extract pixel-aligned image features. Pixel-aligned image features f pThe dimensions are H / 8×W / 8 and H / 2×W / 2. A patch of H'×W' is rendered (as in KeypointNeRF) and 64 points are sampled along each ray, divided into fine and coarse sampling, where the fine sampling resamples 64 points. In addition, most query points in space are located in areas where there are no human bodies. In order to save resources and increase speed, a 3D bounding box based on the SMPL model is used for point sampling. The neural radiation field model is then trained on a single 3090 GPU, sampling 1024 rays per batch for 200k iterations, and the training process takes about 3 days.

[0131] In order to evaluate the image quality of the human body radiation modeling method proposed in the embodiment of the present invention and compare it with previous work, including the quality of new perspectives and new poses, the ZJU-MoCa dataset and the Thuman dataset are used to compare with previous work. The ZJU-MoCap dataset contains 9 subjects and 21 camera views, trained on 6 subjects, and tested on the remaining 3 subjects. The Thuman dataset includes 202 human 3D scans. And using the evaluation indicators from the previous NeRF method: Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM), the embodiment of the present invention also follows other human body NeRF methods and reports indicators within the bounding box mask.

[0132] Experiment 1: Comparison of new view synthesis results

[0133] For fair comparison, all methods are evaluated under the KeypointNeRF setting. The quantitative results of new view synthesis are shown in Table 1 below. It can be seen from Table 1 that the human radiation modeling method proposed in the embodiment of the present invention (i.e., the ours column) is comparable to TransHuman in terms of PSNR and SSIM indicators, and outperforms other existing methods. Figure 4 The figure shows a performance comparison of the human body radiation modeling method proposed in the embodiment of the present invention and the existing method in new view synthesis, which visually compares the performance of the human body radiation modeling method proposed in the embodiment of the present invention and the existing method in new view synthesis. Figure 4 As can be seen, KeypointNeRF has difficulty handling unseen poses due to self-occlusion and suboptimal body shape; GP-NeRF exhibits image blur and artifacts; TransHuman performs well in image quality and metrics, but faces difficulties in shape and color accuracy, especially in occluded areas or heads. Despite the existence of positional deviations, the human body radiation modeling method proposed in an embodiment of the present invention synthesizes images with accurate clothing and body details, similar to TransHuman, due to the guidance of human body geometry priors. The geometry-guided attention mechanism reduces color inaccuracy and artifacts, ensuring shape accuracy in self-occlusion.

[0134]

[0135] Table 1

[0136] Experiment 2: Comparison of new pose synthesis results

[0137] For new pose synthesis, the neural radiation field model (NeRF model) trained by the embodiment of the present invention is compared with the existing methods MPS-NeRF, Animatable NeRF, Neural NoveActor (NNA) and Neural Body on the ZJU-MoCap dataset and the Thuman dataset, and the quantitative results are shown in Table 2 below. It can be seen from Table 2 that the human body radiation modeling method proposed in the embodiment of the present invention is quantitatively equivalent to NNA (due to the lack of NNA training settings and codes, the comparison relies on the results reported in existing papers), and maintains superior results in the synthesis of new perspectives and new poses. Figure 5 The figure shows the performance comparison of the human body radiation modeling method proposed in the embodiment of the present invention and the existing method in new posture synthesis. Figure 5 As can be seen in the figure, Neural Body has difficulties in processing unseen poses, resulting in artifacts in clothing and low image quality; AniNeRF has serious artifacts in new perspective images; MPS-NeRF produces robust results, but is affected by the bias introduced by spatial transformation and self-occlusion, resulting in blurred details. In contrast, the human body radiation modeling method proposed in the embodiment of the present invention significantly enhances the human body geometry and details by utilizing accurate human body geometry prior information and an effective attention mechanism, surpassing other methods.

[0138] In addition, it can be seen from Table 2 that the results of the Thuman dataset are less clear than those of the ZJU-MoCap dataset because multiple 3D scans are processed into 2D images with monochrome clothing. Neural Body, AniNeRF, and MPS-NeRF still show blurriness in the details of clothing wrinkles, while the human body radiation modeling method proposed in the embodiment of the present invention performs well in synthesizing finer clothing wrinkles.

[0139]

[0140] Table 2

[0141] Experiment 3: Ablation experiment

[0142] In order to verify the effectiveness of human body geometry embedding, pixel alignment features, human body geometry features, and the entire human body geometry perception attention mechanism in training the neural radiation field model of the embodiment of the present invention, ablation experiments are performed on the ZJU-MoCap dataset. The quantitative results of the ablation experiments on the ZJU-MoCap dataset are shown in Table 3 below. Figure 6 The figure shows the qualitative results of the ablation experiment on the ZJU-MoCap dataset, where human body geometry embedding is represented as "w / o geoembedding.", pixel alignment features are represented as "w / o pixel att.", human body geometry features are represented as "w / o geoatt.", the whole human body geometry perception attention mechanism is represented as "w / o att.", and the neural radiation field model of the embodiment of the present invention is represented as "our full madel". In addition, all modules are turned off, and only the pixel alignment features are used as the input of the fully connected neural network, which is represented as the base model "Base madel".

[0143]

[0144] Table 3

[0145] Influence of human body geometry embedding: In order to study the effectiveness of human body geometry embedding, the pixel alignment features are directly used as the input of the network and attention mechanism without using any geometric prior information. As can be seen from Table 3, its various performance indicators are significantly reduced compared with the neural radiation field model of the embodiment of the present invention, which shows the effectiveness of setting human body geometry embedding in the embodiment of the present invention. Figure 6 As can be seen in Figure 3, without body geometry embedding, the synthesized image exhibits poor detail performance and blurs on clothing and wrinkle details.

[0146] Influence of human geometry perception attention mechanism: In order to study the effectiveness of human geometry perception attention mechanism, the pixel alignment feature is directly combined with human geometry embedding to train the NeRF model without using the human geometry perception attention mechanism designed in the embodiment of the present invention. As can be seen from Table 3, the performance of the model is significantly reduced compared with the neural radiation field model in the embodiment of the present invention. Figure 6 The visualization results under "w / o att." show that due to the self-occlusion of the head and leg areas, the synthesized image has obvious geometric black artifacts and color darkening, and the human body geometry-aware attention mechanism of the embodiment of the present invention effectively solves these problems and improves the performance.

[0147] In addition, ablation experiments were conducted on pixel-aligned feature attention and geometric feature attention. From Table 3 and Figure 6The results of "w / opixel att." and "w / o geo att." in the figure show that without pixel-aligned feature attention, the detailed texture of the image appears blurred, and without geometric feature attention, certain artifacts appear in the synthesized image and lack the geometric shape of the human body. The neural radiance field model of the embodiment of the present invention demonstrates the effectiveness of our dual-module attention mechanism.

[0148] Reference Figure 7 The embodiment of the present invention further provides a geometrically-aware human body radiation modeling system for a live broadcast scene, comprising:

[0149] A human body geometry embedding acquisition module is used to acquire a sample image, and then obtain a human body geometry embedding according to the sample image and a preset parameterized human body model;

[0150] A pixel alignment feature acquisition module, used to determine the sample point corresponding to the sample image in the target space, and then obtain the first pixel alignment feature according to the sample point and the sample image;

[0151] A geometry-aware attention module, configured to obtain a second pixel-aligned feature based on the first pixel-aligned feature, a human body geometry embedding, and a preset pixel-aligned feature attention module, and to obtain a human body geometry feature based on the first pixel-aligned feature and a preset geometry feature attention module;

[0152] A model training module is used to connect the human body geometry embedding, the second pixel alignment feature and the human body geometry feature to obtain a training sample, and then train a fully connected neural network based on the training sample to obtain a neural radiation field model;

[0153] The posture and viewpoint reconstruction module is used to obtain the sparse image to be reconstructed, input the sparse image to be reconstructed into the neural radiation field model, and generate a new posture image of the human body and a new viewpoint image of the human body.

[0154] The contents of the above-mentioned embodiments of the geometrically-aware human body radiation modeling method for live broadcast scenes are all applicable to the embodiments of the geometrically-aware human body radiation modeling system for live broadcast scenes. The functions specifically implemented by the embodiments of the geometrically-aware human body radiation modeling system for live broadcast scenes are the same as those of the above-mentioned embodiments of the geometrically-aware human body radiation modeling method for live broadcast scenes, and the beneficial effects achieved are also the same as those achieved by the above-mentioned embodiments of the geometrically-aware human body radiation modeling method for live broadcast scenes.

[0155] The embodiment of the present invention further provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the program is executed by the processor, the above-mentioned geometric perception human body radiation modeling method for live broadcasting scenes is realized. The electronic device may be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0156] like Figure 8 FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention, referring to FIG. Figure 8 , an embodiment of the present invention provides an electronic device, including:

[0157] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0158] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the geometric perception human body radiation modeling method for live broadcast scenes in the embodiments of the present invention;

[0159] Input / output interface 1003, used to implement information input and output;

[0160] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0161] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0162] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0163] An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned geometrically perceived human radiation modeling method for live broadcast scenes.

[0164] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0165] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0166] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0167] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0168] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0169] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0170] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0171] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0172] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A geometrically perceived human body radiation modeling method for live broadcast scenes, characterized in that: The following steps are involved: Acquire a sample image, and then obtain a human body geometric embedding according to the sample image and a preset parameterized human body model; Determine a sample point corresponding to the sample image in the target space, and then obtain a first pixel alignment feature according to the sample point and the sample image; Obtain a second pixel alignment feature according to the first pixel alignment feature, the human body geometry embedding, and a preset pixel alignment feature attention module, and obtain a human body geometry feature according to the first pixel alignment feature and a preset geometry feature attention module; Connecting the human body geometry embedding, the second pixel alignment feature, and the human body geometry feature to obtain a training sample, and then training a fully connected neural network according to the training sample to obtain a neural radiation field model; A sparse image to be reconstructed is obtained, and the sparse image to be reconstructed is input into the neural radiation field model to generate a new posture image of the human body and a new perspective image of the human body.

2. According to claim 1, a geometric perception human body radiation modeling method for live broadcast scene is characterized in that: The step of obtaining human body geometry embedding according to the sample image and the preset parameterized human body model specifically includes: Obtaining a first query point according to the sample image; Calculating the closest surface from the first query point to the parameterized human body model, and calculating a normal vector corresponding to the closest surface; Calculating the local coordinates and the directed distance of the first query point on the nearest surface, and then obtaining the local position corresponding to the first query point according to the local coordinates and the directed distance; A mapping of the first query point on the nearest surface is obtained, and then the human body geometric embedding is obtained according to the normal vector, the local position and the mapping.

3. According to claim 1, a geometric perception human body radiation modeling method for live broadcast scene is characterized in that: The step of obtaining a first pixel alignment feature according to the sample point and the sample image specifically includes: Transforming the sample point from the target space to a standard space to obtain a second query point, and then transforming the second query point from the standard space to an observation space to obtain a third query point; Projecting the third query point onto the sample image to obtain a two-dimensional coordinate point corresponding to the third query point; Extracting features from the sample image using a convolutional neural network to obtain a feature map; The first pixel alignment feature is obtained according to the two-dimensional coordinate point and the feature map.

4. According to claim 1, a geometric perception human body radiation modeling method for live broadcast scene is characterized in that: The human body geometry embedding includes a first human body geometry embedding in a specification space and a second human body geometry embedding in an observation space, and the step of obtaining the second pixel alignment feature according to the first pixel alignment feature, the human body geometry embedding and a preset pixel alignment feature attention module specifically includes: Obtaining a camera direction, and then using the second human body geometric embedding as a first query agent, using the first pixel alignment feature, the camera direction, and the first human body geometric embedding as a first key matrix, and using the first pixel alignment feature and the second human body geometric embedding as a first value matrix; A first attention weight is obtained according to the similarity between the first query agent and the first key matrix, and the first value matrix is ​​weighted according to the first attention weight to obtain the second pixel alignment feature.

5. According to claim 4, a geometric perception human body radiation modeling method for live broadcast scene is characterized in that: The step of obtaining the human body geometric features according to the first pixel alignment features and the preset geometric feature attention module specifically includes: Setting vertices and calculating surface normals of the vertices in the observation space; using the surface normal as a second query proxy, the camera direction and the first pixel-aligned feature as a second key matrix, and the first pixel-aligned feature as a second value matrix; A second attention weight is obtained according to the similarity between the second query agent and the second key matrix, and then the second value matrix is ​​weighted according to the second attention weight to obtain the human body geometric features.

6. The geometric perception human body radiation modeling method for live broadcast scene according to claim 1, characterized in that: The fully connected neural network includes a density multilayer perceptron and a color multilayer perceptron. The step of training the fully connected neural network according to the training samples to obtain a neural radiation field model specifically includes: Input the training samples into the fully connected neural network, obtain the predicted volume density corresponding to each sample point through the density multilayer perceptron, and obtain the predicted color corresponding to each sample point through the color multilayer perceptron; Obtaining a predicted color value corresponding to each light ray by orthogonal integral calculation according to the predicted volume density and the predicted color; Calculating a square loss according to the predicted color value and the true color value, and then obtaining a loss function according to the square loss, a preset perceptual loss, and a preset mask loss; The fully connected neural network is updated and trained according to the loss function to obtain the neural radiation field model.

7. The geometric perception human body radiation modeling method for live broadcast scene according to claim 6, characterized in that: The predicted color value is determined by the following formula: Where C(r) represents the predicted color value of light r, t n Indicates the preset near limit, t f represents the preset far limit, σ(t) represents the predicted volume density within the t limit, and t n ≤t≤t f , σ(s) represents the predicted volume density within the s limit, and t n ≤s≤t, c(t,d) represents the predicted color along the light direction d within the limit of t.

8. A geometric perception human radiation modeling system for live broadcast scenes, characterized in that: include: A human body geometry embedding acquisition module, used to acquire a sample image, and then obtain a human body geometry embedding according to the sample image and a preset parameterized human body model; A pixel alignment feature acquisition module, used to determine a sample point corresponding to the sample image in the target space, and then obtain a first pixel alignment feature according to the sample point and the sample image; a geometry-aware attention module, configured to obtain a second pixel-aligned feature based on the first pixel-aligned feature, the human body geometry embedding, and a preset pixel-aligned feature attention module, and to obtain a human body geometry feature based on the first pixel-aligned feature and a preset geometry feature attention module; A model training module, used for connecting the human body geometry embedding, the second pixel alignment feature and the human body geometry feature to obtain a training sample, and then training a fully connected neural network according to the training sample to obtain a neural radiation field model; The posture and viewpoint reconstruction module is used to obtain the sparse image to be reconstructed, input the sparse image to be reconstructed into the neural radiation field model, and generate a new posture image of the human body and a new viewpoint image of the human body.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the geometrically-aware human body radiation modeling method for live broadcast scenes as described in any one of claims 1 to 7 are realized.

10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the geometrically-aware human radiation modeling method for live broadcast scenes as described in any one of claims 1 to 7.

Images