Virtual imaging method, system, computing device and storage medium for virtual photography

Abstract

The present application provides a virtual imaging method, system, computing device and storage medium for virtual shooting, which relates to the field of virtual imaging technology. Among them, a multi-wavelength light source is used to synchronously collect the reflection characteristic data of the object surface, and various shadow parameters are generated through the image feature extraction network; the dynamic posture data and binocular visual depth information are integrated to construct the dynamic space coordinate mapping of the virtual and real scenes; the mirror reflectivity and deformation correction parameters are combined to calculate the reflection intensity distribution of the virtual object surface in real time through the light propagation simulation module, and dynamically adjust the relative position of the virtual object and the real light source; a continuous and smooth shadow boundary transition effect is generated based on the material attenuation model; the shadow effect is calibrated in time and space by a dynamic posture sequence to achieve frame-level synchronous rendering of the virtual and real light source positions and shadow changes. The technical solution provided by the present application solves the light and shadow distortion problem of virtual objects in real scenes through optical parameter modeling and dynamic path optimization.

Description

Technical Field

[0001] The present application relates to the field of virtual imaging technology, and in particular to a virtual imaging method, system, computing device, and storage medium for virtual photography. Background Art

[0002] In virtual filming scenarios like film and television special effects and live streaming, the realism of virtual-reality fusion relies heavily on the consistency of light and shadow between virtual objects and the real scene. With the increasing demand for dynamic filming, existing technologies must meet three core requirements: real-time capture of the dynamic changes in the characteristics of real-world light sources and surface reflections; high-precision matching of the deformation and attenuation of virtual object shadows with the spatial relationship of real-world light sources; and smooth transitions between shadow boundaries on complex materials such as metal and transparent materials to avoid a sense of disconnection between virtual and real light and shadow.

[0003] The current mainstream approach is static light mapping technology based on a monocular RGB camera. This technology uses a single white light source to illuminate the object, capturing surface color information with the monocular RGB camera. This technology, combined with pre-trained light estimation models such as the SH light coefficient prediction network, estimates the ambient light distribution. Offline modeling tools are then used to generate static shadow maps for virtual objects, which are then placed into the scene using Euler angle coordinate transformations. In dynamic scenes, optical flow is used to track the object's motion, and linear interpolation is used to adjust shadow positions.

[0004] The solution is limited by the combination of a single white light source and a monocular RGB camera, and is unable to analyze multi-wavelength reflection characteristics, resulting in a mismatch between the highlight area of the virtual object and the spectral characteristics of the real scene, and metal or fabric materials are prone to color distortion. It relies on static shadow maps and optical flow methods to track dynamic postures. When the object moves quickly, the shadow offset error can reach 3-5 pixels due to model update delays, which requires manual frame-by-frame correction. In addition, the pre-trained lighting model only supports diffuse reflection approximation of basic materials, lacks modeling of the subsurface scattering effect of translucent materials, and relies on Gaussian blur processing for shadow boundaries, resulting in a harsh separation in the virtual-real transition area, greatly increasing the cost of subsequent repair. Summary of the Invention

[0005] The present application provides a virtual imaging method, system, computing device and storage medium for virtual shooting, which are used to solve the problems of delayed real-time matching of virtual and real light sources and abrupt transition of shadows of complex materials in the prior art.

[0006] In a first aspect, the present application provides a virtual imaging method for virtual photography, comprising:

[0007] Synchronously collecting and photographing reflection characteristic data of an object's surface using a multi-wavelength light source, inputting the reflection characteristic data into a pre-trained image feature extraction network to generate shadow feature parameters, wherein the shadow feature parameters include the specular reflectance and diffuse reflectance of the object's surface;

[0008] Collecting dynamic posture data of the photographed object and fusing the dynamic posture data with depth information obtained through a binocular vision positioning system to generate a spatial coordinate mapping relationship between virtual and real scenes;

[0009] Inputting the deformation correction parameters in the mapping relationship between the mirror reflectivity and the spatial coordinates into a light propagation path simulation module to calculate the reflection intensity distribution of each area on the surface of the virtual object, and adjusting the relative position relationship between the virtual object and the light source in the real scene in real time according to the dynamic posture data;

[0010] Inputting the reflection intensity distribution and the adjusted relative position relationship into a light propagation model including material attenuation characteristics to generate a shadow boundary transition effect with continuous deformation matching;

[0011] Based on the dynamic posture change sequence in the spatial coordinate mapping relationship, the shadow boundary transition effect is aligned with the light source position of the real scene in time and space to complete the synchronous light and shadow rendering of the virtual and real scene.

[0012] Optionally, inputting the deformation correction parameter in the mapping relationship between the mirror reflectivity and the spatial coordinates into a light propagation path simulation module to calculate the reflection intensity distribution of each area on the surface of the virtual object includes:

[0013] Splitting the mirror reflectivity into multiple local reflectivity sets according to the spatial positions of the vertices on the surface of the virtual object, and performing light propagation direction offset correction on each local reflectivity set according to the spatial deformation parameters corresponding to the vertices in the spatial coordinate mapping relationship;

[0014] Based on the corrected light propagation direction offset, the initial value of the mirror reflection intensity of each vertex at the preset light incident angle is calculated, and the spatial deformation parameters of adjacent vertices in the spatial coordinate mapping relationship are interpolated in a directional continuity manner to generate the reflection intensity distribution of the continuous area on the surface of the virtual object.

[0015] Optionally, adjusting the relative position relationship between the virtual object and the light source in the real scene in real time according to the dynamic posture data includes:

[0016] Converting the displacement variation in the dynamic posture data into a position variation of a virtual object coordinate system relative to a real scene light source coordinate system;

[0017] Taking the initial position of the light source in the real scene as a reference point, the position change is reversely superimposed along the moving direction vector to generate a compensated position offset of the light source relative to the virtual object;

[0018] Calculating a real-time projection distance of the light source in the moving direction of the virtual object according to the compensation position offset and the dynamic reference position of the vertex on the surface of the virtual object, and performing dynamic tilt correction on the illumination angle of the light source in the virtual scene based on the projection distance;

[0019] The tilt-corrected illumination angle is superimposed on the compensated position offset to obtain a relative position update parameter between the light source and the virtual object. Based on the update frequency of the dynamic posture data, the relative position update parameter is incrementally iterated to complete the relative position relationship adjustment between the light source and the virtual object.

[0020] Optionally, fusing the dynamic posture data with depth information obtained by a binocular vision positioning system to generate a virtual and real scene space coordinate mapping relationship includes:

[0021] Establishing a three-dimensional coordinate system with the center of mass of the photographed object as the origin based on the baseline parameters of the binocular vision positioning system; converting the joint rotation angle and displacement in the dynamic posture data into a three-dimensional vector in the three-dimensional coordinate system;

[0022] The three-dimensional vector is compositely superimposed with the three-dimensional coordinates of the joint feature points in the depth information obtained by the binocular vision positioning system to generate composite spatial data containing dynamic deformation parameters;

[0023] Calculating the spatial offset of each vertex on the surface of the virtual object based on the deformation parameters in the composite spatial data; generating a motion trajectory of the vertices on the surface of the virtual object as the dynamic posture changes based on the spatial offset and the object contour boundary in the depth information obtained by the binocular vision positioning system;

[0024] A dynamic mapping relationship between the vertex coordinates of the virtual object and the real scene coordinates is established based on the motion trajectory.

[0025] Optionally, the reflection intensity distribution and the adjusted relative position relationship are input into a light propagation model including material attenuation characteristics to generate a shadow boundary transition effect with continuous deformation matching, including:

[0026] Calculate the direct illumination contribution value of each vertex under the illumination of the light source based on the initial value of the specular reflection intensity of each vertex on the surface of the virtual object in the reflection intensity distribution and the real-time projection distance of the light source in the adjusted relative position relationship;

[0027] According to the material absorption coefficient preset in the material attenuation characteristic, the direct illumination contribution value is attenuated layer by layer along the light propagation path from the light source to the surface of the virtual object to obtain the attenuated illumination intensity;

[0028] Based on the deformation compensation coefficient in the spatial coordinate mapping relationship, the illumination intensity is dynamically deformed and matched to make the illumination intensity change between adjacent vertices conform to the continuous deformation trend of the surface of the virtual object;

[0029] Superimposing the corrected light intensity with the diffuse reflectance coefficient of the surface of the virtual object to generate the final reflection intensity of each vertex under the illumination of the real scene light source;

[0030] According to the spatial distribution of the final reflection intensity, the reflection intensity change gradient between adjacent vertices is calculated, and based on the change gradient, a smooth transition interpolation is performed on the boundary of the shadow area to generate a shadow boundary transition effect with continuous deformation matching.

[0031] Optionally, based on the dynamic posture change sequence in the spatial coordinate mapping relationship, the shadow boundary transition effect is spatially and temporally aligned with the light source position of the real scene to complete the light and shadow synchronization rendering of the virtual and real scene, including:

[0032] Calculating the movement direction and displacement speed of each vertex on the surface of the virtual object in the three-dimensional space coordinate system according to the posture difference of consecutive time nodes in the dynamic posture change sequence in the spatial coordinate mapping relationship;

[0033] Extracting the light source movement trajectory of the real scene, and establishing a direction difference parameter between the movement direction in the light source movement trajectory and the movement direction of the virtual object vertex;

[0034] Matching the brightness and darkness change rate of each pixel area in the shadow boundary transition effect with the direction difference parameter to obtain the shadow boundary deformation caused by the light source movement;

[0035] According to the time interval of the dynamic posture change sequence, the displacement speed of the virtual object vertex and the light source movement trajectory are subjected to time-series synchronous interpolation calculation to generate a correlation ratio between the shadow shape variable and the light source movement speed;

[0036] Based on the associated ratio, the light and dark distribution areas of the shadow boundary transition effect are dynamically scaled and adjusted so that the curvature of the virtual shadow boundary changes synchronously with the movement trajectory of the light source; the adjusted shadow boundary transition effect is superimposed and fused with the outline of the illumination area of the real scene light source, so that the deformation process of the virtual shadow boundary and the movement process of the real light source form a dynamic following relationship.

[0037] Optionally, performing directional continuity interpolation on spatial deformation parameters of adjacent vertices in the spatial coordinate mapping relationship to generate a reflection intensity distribution of a continuous area on the surface of the virtual object includes:

[0038] Calculating the deformation gradient change in the direction of the vertex connection line based on the difference in spatial deformation parameters between adjacent vertices in the spatial coordinate mapping relationship;

[0039] Based on the deformation gradient change, multiple intermediate interpolation points are set on the line connecting adjacent vertices, and according to the distance ratio between each intermediate interpolation point and the adjacent vertex, the spatial deformation parameter interpolation result corresponding to each intermediate interpolation point is calculated;

[0040] Performing weighted fusion on the interpolation result of the spatial deformation parameter and the initial value of the mirror reflection intensity of the corresponding vertex to obtain the reflection intensity value of the intermediate interpolation point;

[0041] According to the reflection intensity values of all intermediate interpolation points, the reflection intensity distribution of the continuous area on the surface of the virtual object is constructed.

[0042] In a second aspect, the present application provides a virtual imaging system for virtual photography, comprising:

[0043] An acquisition module is used to synchronously acquire reflection characteristic data of the object surface using a multi-wavelength light source, input the reflection characteristic data into a pre-trained image feature extraction network, and generate shadow feature parameters including the specular reflectivity and diffuse reflectance coefficient of the object surface;

[0044] A generation module is used to build a spatial posture monitoring array based on multiple motion sensors, collect dynamic posture data of the captured object in real time, and fuse the dynamic posture data with the depth information obtained by the binocular vision positioning system to generate a spatial coordinate mapping relationship between the virtual and real scenes;

[0045] a calculation module, configured to input the deformation correction parameters in the mapping relationship between the mirror reflectivity and the spatial coordinates into a light propagation path simulation module to calculate the reflection intensity distribution of each area on the surface of the virtual object, and to adjust the relative position relationship between the virtual object and the light source in the real scene in real time according to the dynamic posture data;

[0046] The generation module is further configured to input the reflection intensity distribution and the relative position relationship into a light propagation model including material attenuation characteristics to generate a shadow boundary transition effect with continuous deformation matching, wherein the deformation correction parameter is synchronously updated with the shadow boundary transition effect through the real-time posture change data of the spatial posture monitoring array;

[0047] The alignment module is used to align the shadow boundary transition effect with the light source position of the real scene in time and space based on the dynamic posture change sequence in the spatial coordinate mapping relationship, so as to complete the synchronous light and shadow rendering of the virtual and real scene.

[0048] In a third aspect, an embodiment of the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the virtual imaging method applied to virtual shooting as described in the first aspect above.

[0049] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program. When the computer program is executed by a computer, the virtual imaging method for virtual shooting as described in the first aspect is implemented.

[0050] In the embodiment of the present application, the reflection characteristic data of the object surface is synchronously collected through a multi-wavelength light source, and the image feature extraction network is combined to accurately separate the mirror reflectivity and diffuse reflectance coefficient, effectively solving the spectral distortion problem of complex materials; the dynamic posture data and binocular vision depth information are integrated to construct a high-precision virtual-real space coordinate mapping relationship, significantly reducing the shadow offset error in dynamic scenes; based on the light propagation path simulation module, the reflection intensity distribution of the virtual object surface is calculated in real time, and the material attenuation model is combined to generate a continuous and smooth shadow boundary transition effect, eliminating the sense of separation of the virtual-real fusion boundary; finally, through the spatiotemporal alignment algorithm of the dynamic posture sequence, the real-time synchronous rendering of the virtual shadow and the real light source is realized, achieving a film-level virtual-real light and shadow fusion effect.

[0051] Furthermore, the specular reflectivity is split into local reflectivity sets based on the vertices of the virtual object's surface. The light propagation direction offset is corrected using spatial deformation parameters, and a continuous reflection intensity distribution is generated through interpolation of the deformation parameters of adjacent vertices. Vertex-level reflectivity splitting and deformation correction improves the accuracy of light direction calculations in highlight areas of the virtual object's surface, avoiding light spot anomalies caused by dynamic surface deformation. Combined with interpolation of adjacent vertex deformation parameters, the continuity of the reflection intensity distribution on complex surfaces is ensured, eliminating the blocky light spots found in traditional global models and achieving highly realistic light and shadow simulation on dynamic surfaces.

[0052] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 A flowchart of a virtual imaging method for virtual shooting provided by the present application is shown;

[0055] Figure 2 A schematic structural diagram of a virtual imaging system for virtual photography provided by the present application is shown;

[0056] Figure 3 A schematic structural diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0058] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0059] Researchers have found that existing virtual photography technologies have core issues in dynamic light and shadow fusion, such as inaccurate capture of multi-wavelength reflection characteristics, delayed real-time matching of virtual and real light sources, and abrupt shadow transitions of complex materials. Based on this, this application provides a virtual imaging method for virtual photography. This method achieves high-fidelity synchronization of light and shadow deformation and attenuation effects of virtual objects in real scenes through multi-spectral reflection characteristic separation, dynamic spatial mapping, and material attenuation modeling. This solution is suitable for scenarios such as film and television special effects production and AR / VR real-time interactive live broadcasts.

[0060] The entire research and development process reflects the precise capture of the surface reflection characteristics of objects through multi-wavelength light sources, the construction of high-precision virtual-real space mapping by combining dynamic posture and binocular vision data, the real-time calculation of the reflection intensity distribution of virtual objects and the synchronization of light source positions, the generation of natural shadow transitions based on the material attenuation model, and the realization of dynamic matching of virtual and real light and shadow.

[0061] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0062] Figure 1 The embodiment of the present application provides a flowchart of a virtual imaging method applied to virtual shooting, such as Figure 1 As shown, the method includes:

[0063] 101. Synchronously collecting and photographing reflection characteristic data of an object surface using a multi-wavelength light source, inputting the reflection characteristic data into a pre-trained image feature extraction network to generate shadow feature parameters, wherein the shadow feature parameters include the specular reflectivity and diffuse reflectance of the object surface;

[0064] Synchronous multi-wavelength light source acquisition involves simultaneously illuminating an object's surface with an array of light sources encompassing multiple color spectra, then time-sharing the different wavelengths and recording the intensity of the light reflected from the object. Shadow feature parameters are quantitative indicators of the optical properties of an object's surface extracted from the reflection data using algorithms. These include specular reflectance and diffuse reflectance. Specular reflectance describes the surface's ability to directly reflect light, while diffuse reflectance describes the surface's ability to diffusely reflect light.

[0065] In this embodiment, a programmable light source controller first drives a multi-wavelength LED light source array, cyclically switching between red, green, and blue light sources according to a preset timing to illuminate the surface of an object. This synchronously triggers a high-frame-rate industrial camera to capture reflection images of the object surface at each wavelength, forming a multispectral reflectance characteristic dataset. This multispectral characteristic data is then input into a pre-trained convolutional neural network (CNN). The convolutional neural network extracts reflection characteristics at different wavelengths through convolutional layers, and then regresses these reflection characteristics through fully connected layers to output shadow characteristic parameters including specular reflectance and diffuse reflectance.

[0066] In a virtual filming scenario, a robot character made of a metal and plastic composite material is filmed using a multi-wavelength light source array, emitting light of different wavelengths to illuminate the robot's surface. An industrial camera simultaneously captures the strong specular reflection data of the robot's metal components under blue light and the uniform diffuse reflection data of its plastic components under red light, forming a multispectral image. A pre-trained convolutional neural network analyzes this multispectral image and outputs shadow feature parameters, including the high specular reflectivity of metal areas and the low specular reflectivity of plastic areas, providing basic data for subsequent light and shadow simulation.

[0067] 102. Collect dynamic posture data of the photographed object, and fuse the dynamic posture data with depth information obtained by the binocular vision positioning system to generate a spatial coordinate mapping relationship between the virtual and real scenes;

[0068] Dynamic posture data refers to the motion state of an object, captured in real time by sensors, including its position, rotation angle, and trajectory. Spatial coordinate mapping is a mathematical transformation model that precisely aligns the three-dimensional coordinate system of a virtual object with the coordinate system of the real scene, ensuring the consistency of the virtual object's spatial position in the real scene.

[0069] In one embodiment of the present application, an inertial sensor is deployed on the surface of an object to collect the object's motion posture data in real time. Simultaneously, a binocular camera captures the scene and uses a stereo matching algorithm to fuse the dynamic posture data with depth information acquired through a binocular vision positioning system to generate a depth map. Three-dimensional point cloud data from the depth map is then extracted. The dynamic posture data and the three-dimensional point cloud data are input into an extended Kalman filter, where they are timestamped and fused into six-degree-of-freedom posture parameters. Ultimately, a transformation matrix is constructed between the virtual coordinate system and the real scene coordinate system, resulting in a spatial coordinate mapping relationship between the virtual and real scenes.

[0070] Continuing with the above example, when the robot performs rapid rotation, inertial sensors installed at its joints collect real-time posture change data, including the robot's position offset and rotation angle. A binocular vision system simultaneously captures the scene and generates a depth map. A fusion algorithm aligns the dynamic posture data with the 3D point cloud data from the depth map, creating a precise spatial mapping between the robot's virtual model and the real-world ground surface. This ensures spatial consistency between the virtual object and the real environment during rotation.

[0071] 103. Inputting the deformation correction parameter in the mapping relationship between the mirror reflectivity and the spatial coordinates into a light propagation path simulation module to calculate the reflection intensity distribution of each area on the surface of the virtual object, and adjusting the relative position relationship between the virtual object and the light source in the real scene in real time according to the dynamic posture data;

[0072] Deformation correction parameters refer to surface vertex displacement compensation values calculated based on the dynamic deformation of an object, used to correct the light propagation path. Reflection intensity distribution refers to the distribution of reflected light intensity values across the surface of a virtual object at different light incidence angles.

[0073] In this embodiment, the specular reflectivity is divided into multiple local sets based on the mesh vertices of the virtual object's surface. The direction of the light propagation path for each vertex in each of these local sets is offset and corrected using the deformation correction parameters in the spatial coordinate mapping relationship. Based on the corrected light propagation path directions, Monte Carlo integration is used to calculate the initial specular reflection intensity of each vertex at a preset light source angle, and a continuous surface reflection intensity distribution is generated using a bilinear interpolation algorithm. Simultaneously, based on the dynamic posture data, the relative positional relationship between the virtual object and the light source in the real scene is updated in real time using a quaternion rotation matrix.

[0074] Continuing with the above example, when the robot arm bends, the system calculates the displacement of surface vertices based on deformation correction parameters and adjusts the light path in the robot's metal elbow area accordingly. A Monte Carlo integration algorithm calculates the specular reflection intensity of each surface vertex based on the corrected light path direction and generates a continuous surface reflection distribution map through interpolation. Simultaneously, the virtual spotlight's illumination angle is dynamically adjusted based on the real-time rotation data of the robot's head, synchronizing the light source direction with the robot's movements.

[0075] 104. Input the reflection intensity distribution and the adjusted relative position relationship into a light propagation model including material attenuation characteristics to generate a shadow boundary transition effect with continuous deformation matching;

[0076] Material attenuation refers to the varying degrees of light attenuation across different materials, such as the high reflective attenuation of metal versus the low reflective attenuation of cloth. Continuous deformation matching refers to the ability of shadow boundaries to smoothly transition as the surface of an object dynamically deforms.

[0077] In an embodiment of the present application, the Fresnel equation and Lambert's cosine law are called to calculate the light attenuation coefficient based on the difference in material types, the reflection intensity distribution and the adjusted relative position relationship are input into the light propagation model containing material attenuation characteristics, and the level set algorithm is used to generate the initial shadow boundary, and the aliasing is eliminated through the Laplace smoothing filter, and finally the output is a shadow boundary transition effect with continuous deformation matching.

[0078] Continuing with the above example, considering the material differences between the robot's metal torso and plastic arms, and combining the shadow attenuation effect calculated from the light's incident angle, the reflection intensity distribution and the adjusted relative position relationship are input into a light propagation model that uses the high attenuation coefficient for metal and the low attenuation coefficient for plastic. Based on the light propagation model with the input relative position relationship, a level set algorithm is used to generate an initial shadow boundary. A smoothing filter is then applied to create a natural gradient in the transition area of this initial shadow boundary, ultimately creating a soft shadow boundary transition effect at the junction of the metal and plastic, avoiding the visual disruption caused by traditional hard-edge shadows.

[0079] 105. Based on the dynamic posture change sequence in the spatial coordinate mapping relationship, the shadow boundary transition effect is spatially and temporally aligned with the light source position of the real scene to complete the synchronous light and shadow rendering of the virtual and real scene.

[0080] A dynamic pose change sequence refers to the continuous motion trajectory data of an object arranged in chronological order. Spatiotemporal alignment refers to the process of synchronizing virtual shadow effects with real scene light sources in both time and space.

[0081] In this embodiment, the dynamic pose change sequence in the spatial coordinate mapping relationship is segmented into discrete pose sequences based on timestamps. Based on these discrete pose sequences, the shadow boundary transition effect is mapped to the real scene coordinate system frame by frame according to the corresponding timestamps using a preset rigid body transformation matrix. Combined with the physical coordinates of the real scene light source position, the position offset of the virtual shadow is optimized using the least squares method, ultimately achieving pixel-level alignment of virtual and real light and shadow.

[0082] Continuing with the above example, as the robot runs quickly, the system analyzes its dynamic pose sequence frame by frame and maps the virtual shadow effect to the ground coordinate system of the real scene based on the timestamp. An optimization algorithm matches the coordinates of the mapped virtual shadow effect to the position of the real light source in the scene, and corrects the position offset of the virtual shadow in real time. This ensures that the shadow cast by the robot stretches or compresses naturally with the movement during high-speed movement, fully synchronizing with the real light source, and achieving synchronized light and shadow rendering of the virtual and real scene.

[0083] This solution uses multi-wavelength light sources to precisely separate the reflective properties of object surfaces, combines dynamic pose and depth data to construct a high-precision spatial mapping, calculates the reflective distribution of virtual object surfaces in real time, synchronizes the light source position, generates natural shadow transitions based on a material attenuation model, and ultimately achieves frame-level matching of virtual and real light and shadow through spatiotemporal alignment. This method solves the problems of highlight distortion, shadow offset, and virtual-real boundary separation in traditional solutions, making it suitable for high-precision dynamic light and shadow fusion scenarios such as movie special effects and AR live broadcasts.

[0084] In some embodiments, inputting the deformation correction parameter in the mapping relationship between the mirror reflectivity and the spatial coordinates into a light propagation path simulation module to calculate the reflection intensity distribution of each area on the surface of the virtual object includes:

[0085] 201. Split the mirror reflectivity into multiple local reflectivity sets according to the spatial positions of the vertices on the surface of the virtual object, and perform light propagation direction offset correction on each local reflectivity set according to the spatial deformation parameters corresponding to the vertices in the spatial coordinate mapping relationship;

[0086] Local reflectivity sets are subregions of a virtual object's surface divided by mesh vertices. Each subregion contains a set of vertices and their corresponding specular reflectivity data. Spatial deformation parameters are vertex displacements calculated based on the object's dynamic posture changes, used to describe the degree of local surface deformation. Light propagation direction offset correction adjusts the incident direction of light on the object's surface based on the deformation parameters to compensate for optical path deviations caused by deformation.

[0087] In an embodiment of the present application, the mirror reflectivity is first split into multiple local reflectivity sets based on the vertex density of the virtual object surface, and each set corresponds to a sub-area. For example, the mesh vertices in the elbow area of the robot arm are divided into independent sets. Then, based on the deformation parameters corresponding to the vertex in the spatial coordinate mapping relationship, the displacement vector of each vertex is extracted. The light propagation direction offset is corrected for each local reflectivity set by the normal vector correction method in the ray tracing algorithm, and the displacement vector is projected onto the original normal vector direction of the vertex to calculate the deformed light propagation direction offset. The incident direction of the light is adjusted according to the light propagation direction offset to generate a corrected light propagation direction offset. For example, if the vertex displacement causes the normal vector to deflect by 5 degrees, the incident direction of the light is synchronously deflected by 5 degrees to match the deformed surface geometry.

[0088] 202. Based on the corrected light propagation direction offset, calculate the initial value of the mirror reflection intensity of each vertex at a preset light incident angle, and perform directional continuity interpolation on the spatial deformation parameters of adjacent vertices in the spatial coordinate mapping relationship to generate a reflection intensity distribution of a continuous area on the surface of the virtual object.

[0089] The initial specular intensity value is the initial value of the vertex reflected light intensity calculated based on the corrected light direction. Directional continuity interpolation uses the deformation parameters of adjacent vertices to generate a smooth transition of the reflection intensity distribution through interpolation.

[0090] In an embodiment of the present application, based on the corrected offset of the light propagation direction, a Monte Carlo integration algorithm is used to calculate multiple light path samples for each vertex, and the average value of the reflected light intensity is statistically calculated as the initial value of the mirror reflection intensity at a preset light incident angle. For example, the robot elbow vertex is sampled 1024 times, and the average reflection intensity at a 60-degree incident angle of the spotlight is calculated. Subsequently, the displacement gradient and the normal vector change rate in the spatial deformation parameters of adjacent vertices in the spatial coordinate mapping relationship are extracted. The displacement gradient and the normal vector change rate are input into the bilinear interpolation algorithm, and the initial value of the mirror reflection intensity is weightedly fused along the edge of the grid. According to the corresponding displacement gradient weight distribution ratio, a smooth transition value of the intermediate area is generated, and finally a continuous and non-mutated reflection intensity distribution of the continuous area on the surface of the virtual object is formed.

[0091] Here's a specific example:

[0092] When virtually shooting a robot arm bending scene, the robot's elbow surface mesh vertices are first split into multiple local reflectivity sets. Based on the vertex displacement data generated by the robot's bending action, the normal vector offset of each vertex is calculated and the incident direction of the light is corrected to ensure that the highlight reflection direction of the metal surface to the virtual spotlight accurately matches the deformed geometric shape; then, based on the corrected light direction, the initial value of the specular reflection intensity of each vertex is calculated through Monte Carlo integration, and bilinear interpolation is performed using the displacement gradient parameters of adjacent vertices to generate a continuous highlight gradient distribution from the elbow to the forearm, eliminating the blocky light spots caused by traditional discrete sampling, and ultimately making the metal surface present a natural and smooth reflection effect during dynamic bending.

[0093] This solution dynamically adapts to the impact of surface deformation on light propagation paths through local reflectivity splitting and deformation parameter-driven light direction correction. This solves the problem of highlight direction deviation in traditional global reflection models under complex deformations such as dynamic bending and stretching, ensuring that the virtual object's specular reflection direction accurately matches the deformed surface geometry. Combining Monte Carlo integration with bilinear interpolation algorithms, this solution generates a continuous and smooth reflection intensity distribution, eliminating the blocky light spots and sudden reflection changes caused by traditional discrete sampling, and achieving a natural gradient effect in reflection intensity on complex surfaces.

[0094] In some embodiments, adjusting the relative position relationship between the virtual object and the light source in the real scene in real time based on the dynamic posture data includes:

[0095] 301. Convert the displacement variation in the dynamic posture data into a position variation of a virtual object coordinate system relative to a real scene light source coordinate system;

[0096] Displacement delta refers to the positional offset of a virtual object in the real-world coordinate system, typically described as a three-dimensional vector. The positional delta of the virtual object's coordinate system relative to the real-world light source coordinate system is the conversion of the virtual object's displacement into a relative displacement in a coordinate system centered on the light source.

[0097] In an embodiment of the present application, the displacement change of the virtual object is first extracted from the dynamic posture data. The displacement change includes the translation component of the virtual object in three-dimensional space. The vector of the virtual object's displacement change is converted from the object's local coordinate system to a global coordinate system with the real scene light source as the origin using a homogeneous coordinate transformation matrix to obtain the corresponding position change. For example, when the object moves in a certain direction, the displacement vector needs to be recalculated based on the initial position of the light source to ensure that the displacement change is accurately mapped to the position change in the light source coordinate system.

[0098] 302. Using the initial position of the light source in the real scene as a reference point, reversely superimpose the position change along the moving direction vector to generate a compensated position offset of the light source relative to the virtual object;

[0099] The compensation position offset refers to the position correction amount generated in the opposite direction of the object's movement to offset the relative position deviation of the light source caused by the displacement of the virtual object.

[0100] In this embodiment of the present application, the position change vector obtained in step 301 is reversed, using the initial position of the light source in the real scene as a reference point. This reverses the direction of the displacement change vector and multiplies it by a proportional coefficient based on the dynamic requirements of the scene to generate a compensatory position offset for the light source relative to the virtual object. For example, if the object moves to the right, the compensatory offset is generated to the left, ensuring that the virtual object maintains a logically consistent spatial relationship with the light source after the movement.

[0101] 303. Calculate a real-time projection distance of the light source in the moving direction of the virtual object based on the compensation position offset and the dynamic reference position of the vertex on the surface of the virtual object, and perform dynamic tilt correction on the illumination angle of the light source in the virtual scene based on the projection distance.

[0102] Real-time projection distance refers to the vertical projection length of the light source in the direction of the virtual object's movement. It is used to calculate the tilt correction for the light source's illumination angle. Dynamic tilt correction adjusts the light source's pitch or yaw angle based on the projection distance to align the light's direction with the object's motion.

[0103] In an embodiment of the present application, based on the compensation position offset and the dynamic reference position of the vertex on the surface of the virtual object, the light source coordinates are first projected onto the movement direction vector of the virtual object using a vector projection formula to obtain the real-time projection distance corresponding to the light source. Based on the illumination angle of the light source in the virtual scene at the real-time projection distance, the pitch angle correction value of the light source is calculated using a tangent function to dynamically correct the pitch angle of the light source. For example, as the projection distance increases, the pitch angle of the light source increases synchronously, making the light illumination closer to vertical.

[0104] 304. Superimpose the tilt-corrected illumination angle and the compensated position offset to obtain a relative position update parameter between the light source and the virtual object, and incrementally iterate the relative position update parameter based on the update frequency of the dynamic posture data to complete the relative position relationship adjustment between the light source and the virtual object.

[0105] Incremental iteration refers to the step-by-step cumulative correction of relative position parameters based on the update frequency of dynamic posture data to avoid image jitter caused by sudden changes.

[0106] In this embodiment of the present application, the compensated position offset from step 302 is superimposed with the tilt angle correction value from step 303 to generate updated parameters for the relative position of the light source and the virtual object. Based on the update frequency of the dynamic posture data, an incremental Kalman filter is used to iteratively optimize the position update parameters to adjust the relative position relationship between the light source and the virtual object. For example, only a portion of the position correction value for the current frame is applied to each frame, with the remaining portion accumulated to subsequent frames, ensuring smooth, non-judgmental adjustments to the light source position.

[0107] Here's a specific example:

[0108] In the virtual shooting scene of the robot moving sideways, the system first converts the displacement change of the robot moving to the right into a right-shift vector in the light source coordinate system; then, based on the initial position of the light source, it generates a compensation offset to the left, so that the light source is adjusted to the left to offset the logical deviation caused by the robot's displacement; then, it calculates the projection distance of the light source in the direction of the robot's movement, and corrects the pitch angle to make the light slightly downward to match the surface geometry changes when the robot tilts; finally, the compensation offset is superimposed on the angle correction value, and the light source position is smoothly adjusted in steps according to the frame rate, so that the highlight area of the robot's metal shell gradually shifts with the light source. At the same time, the shadow deformation is consistent with the real physical laws, presenting a film-level light and shadow effect without jumps and ghosting.

[0109] This solution dynamically corrects the relative position of virtual objects and real light sources through displacement conversion and offset generation, resolving the problem of light source logical misalignment caused by object movement. Combined with projection distance-driven tilt correction and incremental iterative optimization, it achieves smooth adjustment of light source angle and position, avoiding image jumps. In dynamic scenes, the highlight reflections and shadow deformations of virtual objects can match the changes in light source position in real time, significantly improving the realism and smoothness of virtual-reality interaction. This solution is suitable for virtual shooting scenarios requiring high-frequency light source interaction.

[0110] In some embodiments, the dynamic posture data is fused with depth information obtained by a binocular vision positioning system to generate a virtual and real scene space coordinate mapping relationship, including:

[0111] 401. Establish a three-dimensional coordinate system with the center of mass of the photographed object as the origin based on the baseline parameters of the binocular vision positioning system; convert the joint rotation angle and displacement in the dynamic posture data into a three-dimensional vector in the three-dimensional coordinate system;

[0112] Baseline parameters refer to the physical distance and angle between the two cameras in a binocular vision positioning system. They are used to construct the basic geometric model for 3D spatial measurement. A 3D coordinate system is a right-handed coordinate system established with the center of mass of the object being photographed as the origin and the binocular vision baseline as the reference axis.

[0113] In an embodiment of the present application, a three-dimensional coordinate system with the center of mass of the photographed object as the origin is first established based on the baseline parameters of the binocular vision positioning system. For example, if the distance between the binocular cameras is a specific distance and the angle is a specific angle, the X-axis of the coordinate system extends along the baseline direction, the Y-axis is vertically upward, and the Z-axis points in the depth direction. Then, the joint rotation angle in the dynamic posture data is converted into a rotation matrix, and combined with the displacement to generate a three-dimensional vector in the three-dimensional coordinate system. For example, the pitch angle data of the robot arm is converted into a three-dimensional motion vector in the coordinate system through the rotation matrix, which describes its motion direction and amplitude in space.

[0114] 402. Composite superposition of the three-dimensional vector and the three-dimensional coordinates of the joint feature points in the depth information obtained by the binocular vision positioning system to generate composite spatial data including dynamic deformation parameters;

[0115] Composite spatial data refers to a data set that integrates the coordinates of feature points on the surface of an object in the three-dimensional vector of dynamic posture and binocular vision depth information, and includes the geometric deformation and motion trajectory information of the object.

[0116] In this embodiment of the present application, the three-dimensional vector generated in step 401 is fused with the three-dimensional coordinates of the joint feature points extracted from the depth information of the binocular vision positioning system. The dynamic posture data and the visual data are temporally aligned using a Kalman filter, and weights are assigned based on the data confidence. For example, the dynamic posture data has a higher weight, while the visual data has a lower weight. After weighted fusion of the two, composite spatial data containing dynamic deformation parameters is generated to describe the real-time changes in the surface geometry of the object.

[0117] 403. Calculate the spatial offset of each vertex on the surface of the virtual object based on the deformation parameters in the composite spatial data; generate a motion trajectory of the vertices on the surface of the virtual object as they change with the dynamic posture based on the spatial offset and the object contour boundary in the depth information obtained by the binocular vision positioning system;

[0118] Vertex space offset refers to the three-dimensional position displacement of mesh vertices on the surface of a virtual object due to dynamic posture changes. Motion trajectory refers to the continuous position change path of a vertex as the object moves.

[0119] In this embodiment, deformation parameters are extracted from the composite spatial data, and the displacement vector of each vertex on the virtual object's surface is calculated using a finite element analysis algorithm. For example, when the robot's elbow bends, the tangential displacement of the forearm vertex is calculated based on the rate of change of curvature. Subsequently, based on this spatial offset and the object's contour boundary point cloud data from the depth information acquired by the binocular vision positioning system, a B-spline curve fitting algorithm is used to generate a smooth motion trajectory for the virtual object's surface vertices as they change in dynamic posture, ensuring that the trajectory conforms to physical motion constraints.

[0120] 404. Establish a dynamic mapping relationship between the vertex coordinates of the virtual object and the real scene coordinates based on the motion trajectory.

[0121] The dynamic mapping relationship refers to the real-time conversion model between the vertex coordinates of the virtual object and the real scene coordinates, which is used to describe the correspondence between the vertex position and the real scene as the object moves.

[0122] In this embodiment of the present application, based on the vertex motion trajectory generated in step 403, a mapping function is established using a non-rigid deformation model to transform vertex coordinates into real-world coordinates. This function is used to describe the dynamic mapping relationship between the virtual object's vertex coordinates and the real-world coordinates. For example, a thin plate spline interpolation algorithm is used to convert the motion trajectory of the robot's elbow vertex into a projection matrix of the real-world ground coordinate system. The mapping parameters are then updated frame by frame based on the timestamp, achieving dynamic submillimeter alignment of virtual and real coordinates.

[0123] Here's a specific example:

[0124] In the virtual shooting of the robot arm bending scene, the system establishes a three-dimensional coordinate system with the robot shoulder as the origin based on the binocular vision baseline parameters, and converts the arm rotation angle and displacement into a three-dimensional motion vector in the coordinate system; then it fuses the dynamic posture data with the coordinates of the elbow feature points extracted by binocular vision to generate composite spatial data including the curvature change rate; the forearm vertex displacement is calculated based on the curvature change rate, and a smooth motion trajectory is generated by combining the arm outer contour point cloud; finally, a dynamic mapping relationship between the forearm vertex and the real scene desktop is established through the deformation interpolation algorithm, so that during the bending process of the virtual arm, its surface vertex position is matched with the real desktop projection in real time, and the highlight reflection and shadow deformation of the metal joint naturally transition with the movement, presenting a virtual-real fusion effect without offset or separation.

[0125] This solution constructs a high-precision composite spatial description through the deep fusion of binocular vision and dynamic posture data, solving the spatial mapping error problem of traditional single data sources; generates smooth vertex motion trajectories based on deformation parameters and contour constraints to ensure that the deformation of virtual objects conforms to physical laws; and achieves real-time submillimeter alignment of virtual and real coordinates through dynamic mapping functions, significantly improving the spatial consistency of virtual and real fusion in dynamic scenes.

[0126] In some embodiments, the reflection intensity distribution and the adjusted relative position relationship are input into a light propagation model including material attenuation characteristics to generate a shadow boundary transition effect with continuous deformation matching, including:

[0127] 501. Calculate the direct illumination contribution value of each vertex under the illumination of the light source based on the initial value of the specular reflection intensity of each vertex on the surface of the virtual object in the reflection intensity distribution and the real-time projection distance of the light source in the adjusted relative position relationship.

[0128] Direct illumination contribution refers to the initial light intensity generated by a vertex on a virtual object's surface when directly illuminated by a light source. Real-time projection distance refers to the vertical projection length from the light source to the virtual object's surface, and is used to quantify the degree of attenuation of the light source's illumination intensity on the vertex.

[0129] In an embodiment of the present application, based on the initial value of the specular reflection intensity of each vertex on the surface of the virtual object in the reflection intensity distribution, combined with the real-time projection distance of the light source in the adjusted relative position relationship, the Phong illumination model is used to calculate the direct illumination contribution value of the vertex under the illumination of the light source. Specifically, a ray tracing algorithm is used to detect whether the path from the light source to the vertex is blocked. If it is not blocked, the cosine value of the incident angle of the light and the vertex normal vector is calculated, and the cosine value is multiplied by the specular reflection intensity to obtain the initial illumination contribution value. For example, the smaller the angle between the vertex normal vector and the light direction, the larger the cosine value, and the higher the direct illumination contribution value.

[0130] 502. According to the material absorption coefficient preset in the material attenuation characteristic, the direct illumination contribution value is attenuated layer by layer along the light propagation path from the light source to the surface of the virtual object to obtain the attenuated illumination intensity;

[0131] The material absorption coefficient refers to the ratio of light intensity attenuation along a light path by different materials, such as the high absorption rate of metal versus the low absorption rate of cloth. Layer-by-layer attenuation is the process of reducing light intensity in sections along the light path.

[0132] In this embodiment, the material absorption coefficient preset in the attenuation characteristics is invoked based on the material type, and a segmented integral calculation is performed along the light path from the light source to the vertex of the virtual object's surface. The light path is divided into several equal-length sub-segments, and the direct illumination contribution of each segment is gradually attenuated according to an exponential function of the absorption coefficient multiplied by the distance. For example, metal materials have a higher absorption coefficient, and the attenuation of light is significantly greater than that of plastic materials over the same distance, resulting in the final attenuated light intensity.

[0133] 503. Based on the deformation compensation coefficient in the spatial coordinate mapping relationship, perform dynamic deformation matching correction on the illumination intensity so that the change in illumination intensity between adjacent vertices conforms to the continuous deformation trend of the surface of the virtual object;

[0134] The deformation compensation coefficient refers to the illumination intensity correction parameter generated according to the degree of dynamic deformation of the virtual object surface, which is used to make the illumination change consistent with the deformation trend.

[0135] In the embodiment of the present application, a nonlinear deformation matching correction is performed on the attenuated light intensity based on the deformation compensation coefficient in the spatial coordinate mapping relationship. For example, if the surface curvature increases due to deformation of the vertex, the correction weight of the light intensity is increased according to the curvature change rate, so that the difference in light intensity between adjacent vertices is positively correlated with the degree of deformation. Specifically, the deformation parameters are weighted using a Gaussian function to generate a smoothed and corrected light intensity distribution, so that the light intensity variation between adjacent vertices conforms to the continuous deformation trend of the virtual object surface.

[0136] 504. Superimpose the corrected illumination intensity with the diffuse reflectance coefficient of the surface of the virtual object to generate a final reflection intensity of each vertex under the illumination of the real scene light source;

[0137] The final reflection intensity refers to the total reflected light intensity value of the vertex after comprehensive direct lighting contribution, attenuation correction and deformation matching, including the superposition result of specular reflection and diffuse reflection.

[0138] In this embodiment of the present application, the corrected illumination intensity is linearly superimposed with the diffuse reflectance of the vertices on the virtual object's surface. For example, the high specular reflection intensity of metal vertices is superimposed with the low diffuse reflectance to produce a reflection intensity distribution with bright highlights and soft shadows. Meanwhile, the low specular reflection of cloth vertices is superimposed with the high diffuse reflectance to produce a uniform scattering effect, generating the final reflection intensity of each vertex under the illumination of a real scene light source.

[0139] 505. Calculate the reflection intensity change gradient between adjacent vertices based on the final reflection intensity spatial distribution, and perform smooth transition interpolation on the boundary of the shadow area based on the change gradient to generate a shadow boundary transition effect with continuous deformation matching.

[0140] The reflection intensity gradient is the difference in reflection intensity between adjacent vertices and is used to quantify the sharpness of the shadow boundary. Smooth transition interpolation is to blur the shadow boundary based on the gradient value to eliminate aliasing.

[0141] In an embodiment of the present application, the gradient of the reflection intensity change between adjacent vertices is calculated based on the spatial distribution of the final reflection intensity. For example, a gradient value greater than a threshold is determined to be a shadow boundary. A smooth transition interpolation is then performed on the boundary of the shadow area based on the gradient change. The reflection intensity of the boundary area is smoothed using a bicubic interpolation algorithm to generate a shadow boundary transition effect with continuous deformation matching. For example, in the high-gradient area at the joint of a robot arm, the light intensity value of the transition pixel is generated by interpolation, so that the shadow boundary transitions from a sharp edge to a natural gradient effect.

[0142] Here's a specific example:

[0143] In the virtual shooting of the robot arm bending scene, the system first calculates the direct lighting contribution value of the metal surface based on the initial value of the mirror reflection intensity of the elbow vertex and the real-time projection distance of the light source; then, according to the high absorption coefficient of the metal material, the light propagation path is attenuated layer by layer to reduce the light intensity to the natural dark level; then, combined with the curvature change rate caused by the elbow bending deformation, the attenuated light intensity is dynamically corrected to ensure that the light intensity of the vertex on the inner side of the bend is lower than that on the outer side; the corrected light intensity is superimposed on the diffuse reflection coefficient to generate a reflection intensity distribution with a soft transition between metal highlights and dark areas; finally, based on the reflection intensity gradient of adjacent vertices, the shadow boundary between the elbow and forearm is smoothed by the bicubic interpolation algorithm, so that the sharp edge transitions to a natural gradient, completely eliminating the pixel segmentation at the boundary of virtual and real fusion, and presenting a film-level light and shadow effect.

[0144] This solution calculates the direct lighting contribution value vertex by vertex, combined with the attenuation simulation of the material absorption characteristics along the light path, to accurately restore the light and dark changes of complex materials such as metals and plastics under dynamic light sources; dynamically corrects the light intensity based on the deformation compensation coefficient, so that the light and shadow distribution is strictly synchronized with the deformation trends such as bending and stretching of the object surface, eliminating the highlight misalignment problem of traditional static models in dynamic scenes; by superimposing the diffuse reflection coefficient, the reflection intensity with a natural transition between highlights and dark areas is generated, and the shadow boundary is smoothly optimized based on the gradient-driven interpolation algorithm, completely solving the jagged edges and pixel-level fragmentation of the virtual-real fusion boundary.

[0145] In some embodiments, based on the dynamic posture change sequence in the spatial coordinate mapping relationship, the shadow boundary transition effect is spatially and temporally aligned with the light source position of the real scene to complete the light and shadow synchronization rendering of the virtual and real scene, including:

[0146] 601. Calculate the movement direction and displacement speed of each vertex on the surface of the virtual object in the three-dimensional space coordinate system based on the posture difference between consecutive time nodes in the dynamic posture change sequence in the spatial coordinate mapping relationship;

[0147] A dynamic pose change sequence is a chronological record of a virtual object's motion trajectory, including position and rotation information at each time point. The pose difference, which measures the magnitude of the change in the object's pose between adjacent time points, is used to infer vertex movement direction and displacement speed.

[0148] In this embodiment, the pose differences between consecutive time nodes are extracted from the dynamic pose change sequence in the spatial coordinate mapping relationship, and the displacement vector and rotation angle between two adjacent frames are calculated using a difference method. Based on the three-dimensional spatial coordinate system, a velocity interpolation algorithm is used to generate the vertex's movement direction vector along the three-dimensional axis, and the displacement velocity of this movement direction vector is calculated. For example, if a robot arm vertex moves along a specific axis within a specific time period, the displacement velocity is calculated by dividing the displacement by the time interval.

[0149] 602. Extracting a light source movement trajectory of a real scene, and establishing a direction difference parameter between a movement direction in the light source movement trajectory and a movement direction of a vertex of a virtual object;

[0150] The light source movement trajectory refers to the movement path of the light source in the three-dimensional space in the real scene. The direction difference parameter refers to the angular deviation between the light source movement direction and the movement direction of the virtual object vertex.

[0151] In the embodiment of the present application, the trajectory of a light source in a real scene is tracked using an optical flow method or a sensor to extract a direction vector of the light source trajectory. The light source direction vector and the vertex movement direction vector obtained in step 601 are input into a vector angle formula to calculate a direction difference parameter between the light source direction vector and the vertex movement direction vector. For example, if an angle exists between the light source movement direction and the vertex movement direction, the direction difference parameter is the magnitude of the angle.

[0152] 603. Match the brightness and darkness change rate of each pixel area in the shadow boundary transition effect with the direction difference parameter to obtain the shadow boundary deformation caused by the light source movement;

[0153] The brightness change rate refers to how quickly the brightness of pixels in the shadow boundary area changes over time. The shadow boundary deformation refers to the position offset of the shadow boundary caused by the movement of the light source.

[0154] In this embodiment of the present application, the rate of change between light and dark for each pixel region in the shadow boundary transition effect is extracted and matched with the directional difference parameter from step 602. A Kalman filter is used to weight the rate of change between light and dark and the directional difference parameter to generate the shadow boundary deformation caused by light source movement. For example, the larger the directional difference parameter, the higher the weight of the shadow boundary deformation correction.

[0155] 604. Performing a time-series synchronous interpolation calculation on the displacement speed of the virtual object vertex and the light source movement trajectory according to the time interval of the dynamic posture change sequence, and generating a correlation ratio between the shadow shape variable and the light source movement speed;

[0156] Temporal interpolation is the process of aligning the displacement velocity of a virtual object's vertices with the trajectory of a light source based on the time intervals of a dynamic pose change sequence. Correlation ratio is the dynamic relationship coefficient between the shadow shape and the light source's movement speed.

[0157] In this embodiment, based on the time intervals of the dynamic posture change sequence, the vertex displacement velocity calculated in step 601 and the light source movement trajectory extracted in step 602 are input into a bilinear interpolation algorithm, aligning the motion data of the displacement velocity and the light source movement trajectory by timestamp. A proportional integrator is used to generate a correlation ratio between the shadow shape variable and the change in light source movement speed. For example, when the light source movement speed doubles, the correlation ratio of the shadow shape variable increases simultaneously.

[0158] 605. Dynamically scale and adjust the light and dark distribution areas of the shadow boundary transition effect based on the correlation ratio so that the curvature of the virtual shadow boundary changes synchronously with the movement trajectory of the light source; and superimpose and fuse the adjusted shadow boundary transition effect with the contour of the illumination area of the real scene light source so that the deformation process of the virtual shadow boundary and the movement process of the real light source form a dynamic following relationship.

[0159] Dynamic scaling refers to scaling and deforming the shadow boundary area based on the associated ratio, so that its degree of deformation is synchronized with the movement of the light source. Dynamic tracking refers to the real-time matching of the deformation process of the virtual shadow boundary with the movement of the real light source.

[0160] In this embodiment, the light and dark distribution areas of the shadow boundary transition effect are dynamically scaled based on the correlation ratios obtained in step 604. A thin plate spline interpolation algorithm is used to adjust the curvature of the virtual shadow boundary, synchronizing the curvature with the light source's movement trajectory. Finally, the adjusted shadow boundary transition effect is superimposed on the contour of the actual light source's illuminated area. Alpha blending is used to achieve a smooth fusion of the virtual and real boundaries, ensuring that the deformation of the virtual shadow boundary dynamically follows the movement of the actual light source.

[0161] Here's a specific example:

[0162] In a virtual shooting scene of a robot moving quickly, the system first calculates the movement direction and speed of the robot's elbow vertex in three-dimensional space based on the dynamic posture sequence; then tracks the movement trajectory of the real spotlight and calculates the angle difference between its direction and the movement direction of the elbow vertex; matches the brightness and darkness change rate and direction difference parameters of the elbow shadow boundary to generate the shadow shape variable caused by the movement of the light source; aligns the vertex speed and the light source trajectory through time-series synchronous interpolation to establish a dynamic correlation ratio between the shadow shape variable and the light source speed; finally, dynamically scales the shadow boundary proportionally and merges it with the real light source contour through a hybrid algorithm, so that the shadow of the robot's elbow stretches and deforms in real time as the spotlight moves, and the curvature of the shadow boundary is strictly synchronized with the light source trajectory, presenting a virtual and real light and shadow fusion effect without delay and pixel dislocation.

[0163] This solution accurately calculates the spatial difference parameters between the movement direction of the virtual object's vertices and the movement direction of the real light source through the spatiotemporal alignment of the dynamic posture sequence and the light source movement trajectory, thus solving the problem of delayed misalignment between shadow deformation and light source movement in dynamic scenes. Based on the matching relationship between the brightness and darkness change rate and the direction difference, the influence of light source movement on the deformation of the shadow boundary is quantified to ensure that the shadow deformation direction is strictly synchronized with the light source movement trend. Through time-series interpolation calculation, a dynamic correlation ratio between the shadow deformation and the light source speed is established to achieve frame-level matching of the shadow boundary scaling adjustment with the real light source movement. Finally, through the virtual-real boundary fusion algorithm, the curvature and stretching deformation of the virtual shadow boundary are linked in real time with the real light source movement trajectory, eliminating the shadow lag or deformation distortion problems in traditional solutions.

[0164] In some embodiments, performing directional continuity interpolation on spatial deformation parameters of adjacent vertices in the spatial coordinate mapping relationship to generate a reflection intensity distribution of a continuous area on the surface of the virtual object includes:

[0165] 701. Calculate the deformation gradient change in the direction of the vertex connection line based on the difference in spatial deformation parameters between adjacent vertices in the spatial coordinate mapping relationship.

[0166] The spatial deformation parameter difference refers to the difference in displacement or rotation between adjacent vertices during dynamic deformation. The deformation gradient change refers to the rate of change of the deformation parameter along the line connecting the vertices, which is used to quantify the degree of deformation continuity.

[0167] In an embodiment of the present application, the spatial deformation parameters of adjacent vertices, such as displacement vectors or rotation matrices, are first extracted from the spatial coordinate mapping relationship. The difference in spatial deformation parameters between adjacent vertices, such as the displacement vector difference between vertex A and vertex B, is calculated using a vector difference method. Based on the geometric length of the vertex connection line and the difference in deformation parameters, the deformation gradient change in the direction of the vertex connection line is calculated. For example, if the vertex displacement difference is a specific value and the connection line length is a specific distance, the gradient change is the ratio of the displacement difference to the connection line length.

[0168] 702. Based on the deformation gradient change, set multiple intermediate interpolation points on the line connecting adjacent vertices, and calculate the spatial deformation parameter interpolation result corresponding to each intermediate interpolation point according to the distance ratio between each intermediate interpolation point and the adjacent vertex;

[0169] Intermediate interpolation points are virtual points inserted along the line connecting adjacent vertices to fill in the gaps in deformation parameters between vertices. Distance ratio refers to the ratio of the length of the line connecting the intermediate interpolation point to the adjacent vertex.

[0170] In this embodiment of the present application, based on the deformation gradient change calculated in step 701, multiple intermediate interpolation points are evenly spaced along the line connecting adjacent vertices. A bilinear interpolation algorithm is used to calculate the spatial deformation parameter interpolation result corresponding to each intermediate interpolation point based on the distance ratio between each interpolation point and adjacent vertices A and B. For example, if the interpolation point is 30% of the line length from vertex A, its deformation parameter is a weighted fusion of 70% of the vertex A parameter and 30% of the vertex B parameter.

[0171] 703. Perform weighted fusion on the interpolation result of the spatial deformation parameter and the initial value of the mirror reflection intensity of the corresponding vertex to obtain the reflection intensity value of the intermediate interpolation point;

[0172] Reflection intensity weighted fusion refers to the weighted superposition of the deformation parameter interpolation result of the intermediate interpolation point and the initial value of the mirror reflection intensity of its corresponding vertex to generate the reflection intensity value of the interpolation point.

[0173] In this embodiment, the spatial deformation parameter interpolation result obtained in step 702 and the initial specular reflection intensity values of the corresponding adjacent vertices are input into the weight distribution model. For example, if the spatial deformation parameter interpolation result is close to vertex A, vertex A is given a higher reflection intensity weight. The reflection intensity values of the intermediate interpolation points are calculated using a linear weighting formula to ensure a smooth transition between the reflection intensities of the intermediate interpolation points and the adjacent vertices.

[0174] 704. Construct a reflection intensity distribution of a continuous area on the surface of the virtual object based on the reflection intensity values of all intermediate interpolation points.

[0175] Continuous reflection intensity distribution refers to generating a reflection intensity map covering all areas of the virtual object surface by interpolating the reflection intensity values of the points, eliminating sudden changes in reflection intensity between vertices.

[0176] In the embodiments of the present application, a surface fitting algorithm, such as non-uniform rational B-splines, is used to construct the reflection intensity distribution of continuous areas on the surface of a virtual object based on the reflection intensity values of all intermediate interpolation points. For example, the reflection intensities of the interpolation points are used as control points to generate a smooth reflection intensity surface, ensuring that the reflection intensity values at any location on the surface dynamically match the deformation parameters.

[0177] Here's a specific example:

[0178] In a virtual shooting of a robot arm bending scene, the system first calculates the deformation gradient change along the connecting line based on the displacement difference of adjacent vertices of the elbow, and quantifies the deformation continuity of the metal surface during the bending process; then, multiple intermediate interpolation points are evenly inserted on the vertex line, and deformation parameters are distributed according to the distance ratio. For example, the closer the interpolation point is to the vertex, the closer its deformation parameter is to the vertex data; then, the deformation parameter of the interpolation point is fused with the initial value of the vertex mirror reflection intensity according to the weight to generate the reflection intensity value of the interpolation point, ensuring a smooth transition of light intensity in the bending area; finally, the reflection intensity values of all interpolation points are integrated through a surface fitting algorithm to construct a continuous reflection intensity distribution covering the elbow surface, so that the metal highlight naturally changes from the vertex to the interpolation point, completely eliminating the sudden change in reflection intensity caused by the traditional discrete vertex model, and presenting a film-level dynamic light and shadow effect.

[0179] This solution accurately captures the dynamic deformation trend of the virtual object surface by quantifying the deformation gradient changes of adjacent vertices; based on the deformation parameter calculation of intermediate interpolation points and the weighted fusion of reflection intensity, it fills the blank areas of reflection intensity between vertices and eliminates the problem of sudden changes in reflection intensity caused by traditional discrete vertex models; combined with the surface fitting algorithm, it constructs a continuous reflection intensity distribution covering the object surface, ensuring that the reflection intensity strictly matches the surface geometry under any deformation state.

[0180] Figure 2 The present invention provides a structural diagram of a virtual imaging system for virtual shooting, as shown in FIG. Figure 2 As shown, the system includes:

[0181] An acquisition module 21 is configured to synchronously acquire reflective characteristic data of an object surface using a multi-wavelength light source, input the reflective characteristic data into a pre-trained image feature extraction network, and generate shadow feature parameters including the specular reflectance and diffuse reflectance of the object surface;

[0182] A generation module 22 is configured to construct a spatial posture monitoring array based on multiple motion sensors, collect dynamic posture data of the captured object in real time, and fuse the dynamic posture data with depth information obtained through a binocular vision positioning system to generate a spatial coordinate mapping relationship between the virtual and real scenes;

[0183] a calculation module 23 for inputting the deformation correction parameters in the mapping relationship between the mirror reflectivity and the spatial coordinates into a light propagation path simulation module to calculate the reflection intensity distribution of each area on the surface of the virtual object, and adjusting the relative position relationship between the virtual object and the light source in the real scene in real time according to the dynamic posture data;

[0184] The generating module 22 is further configured to input the reflection intensity distribution and the relative position relationship into a light propagation model including material attenuation characteristics to generate a shadow boundary transition effect with continuous deformation matching, wherein the deformation correction parameters are synchronously updated with the shadow boundary transition effect using the real-time posture change data of the spatial posture monitoring array;

[0185] The alignment module 24 is used to align the shadow boundary transition effect with the light source position of the real scene in time and space based on the dynamic posture change sequence in the spatial coordinate mapping relationship, so as to complete the light and shadow synchronous rendering of the virtual and real scene.

[0186] Figure 2 The virtual imaging system for virtual shooting can be executed Figure 1 The implementation principles and technical effects of the virtual imaging method for virtual photography described in the illustrated embodiment are not further elaborated. The specific manner in which the various modules and units perform operations in the virtual imaging system for virtual photography described in the aforementioned embodiment have been described in detail in the related embodiments of the method and will not be further elaborated here.

[0187] In one possible design, Figure 2 The virtual imaging system for virtual shooting of the embodiment shown can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0188] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0189] The processing component 32 is used for the above Figure 1 The embodiment provides a virtual imaging method for virtual photography.

[0190] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0191] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0192] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0193] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0194] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0195] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0196] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The virtual imaging method of the illustrated embodiment is applied to virtual photography.

[0197] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0198] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0199] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A virtual imaging method for virtual shooting, characterized in that: include: Synchronously collecting and photographing reflection characteristic data of an object's surface using a multi-wavelength light source, inputting the reflection characteristic data into a pre-trained image feature extraction network to generate shadow feature parameters, wherein the shadow feature parameters include the specular reflectance and diffuse reflectance of the object's surface; Collecting dynamic posture data of the photographed object and fusing the dynamic posture data with depth information obtained through a binocular vision positioning system to generate a spatial coordinate mapping relationship between virtual and real scenes; Inputting the deformation correction parameters in the mapping relationship between the mirror reflectivity and the spatial coordinates into a light propagation path simulation module to calculate the reflection intensity distribution of each area on the surface of the virtual object, and adjusting the relative position relationship between the virtual object and the light source in the real scene in real time according to the dynamic posture data; Inputting the reflection intensity distribution and the adjusted relative position relationship into a light propagation model including material attenuation characteristics to generate a shadow boundary transition effect with continuous deformation matching; Based on the dynamic posture change sequence in the spatial coordinate mapping relationship, the shadow boundary transition effect is temporally and spatially aligned with the light source position of the real scene to complete the synchronous light and shadow rendering of the virtual and real scene; Adjusting the relative position relationship between the virtual object and the light source in the real scene in real time according to the dynamic posture data includes: The displacement variation in the dynamic posture data is converted into a position variation of the virtual object coordinate system relative to the real scene light source coordinate system; with the initial position of the light source in the real scene as the reference point, the position variation is reversely superimposed along the moving direction vector to generate a compensated position offset of the light source relative to the virtual object; based on the compensated position offset and the dynamic reference position of the vertex of the virtual object surface, the real-time projection distance of the light source in the moving direction of the virtual object is calculated, and the illumination angle of the light source in the virtual scene is dynamically tilted based on the projection distance; the tilt-corrected illumination angle is superimposed with the compensated position offset to obtain a relative position update parameter between the light source and the virtual object, and based on the update frequency of the dynamic posture data, the relative position update parameter is incrementally iterated to complete the adjustment of the relative position relationship between the light source and the virtual object; The reflection intensity distribution and the adjusted relative position relationship are input into a light propagation model including material attenuation characteristics to generate a shadow boundary transition effect with continuous deformation matching, including: Based on the initial value of the mirror reflection intensity of each vertex on the surface of the virtual object in the reflection intensity distribution, combined with the real-time projection distance of the light source in the adjusted relative position relationship, the direct illumination contribution value of each vertex under the illumination of the light source is calculated; according to the material absorption coefficient preset in the material attenuation characteristic, the direct illumination contribution value is attenuated layer by layer along the light propagation path from the light source to the surface of the virtual object to obtain the attenuated illumination intensity; based on the deformation compensation coefficient in the spatial coordinate mapping relationship, the illumination intensity is dynamically deformed and matched to make the illumination intensity change between adjacent vertices conform to the continuous deformation trend of the virtual object surface; the corrected illumination intensity is superimposed with the diffuse reflection coefficient of the virtual object surface to generate the final reflection intensity of each vertex under the illumination of the real scene light source; according to the spatial distribution of the final reflection intensity, the reflection intensity change gradient between adjacent vertices is calculated, and the boundary of the shadow area is smoothly transitioned and interpolated based on the change gradient to generate a shadow boundary transition effect with continuous deformation matching.

2. The method according to claim 1, characterized in that Inputting the deformation correction parameter in the mapping relationship between the mirror reflectivity and the spatial coordinates into the light propagation path simulation module to calculate the reflection intensity distribution of each area on the surface of the virtual object, including: Splitting the mirror reflectivity into multiple local reflectivity sets according to the spatial positions of the vertices on the surface of the virtual object, and performing light propagation direction offset correction on each local reflectivity set according to the spatial deformation parameters corresponding to the vertices in the spatial coordinate mapping relationship; Based on the corrected light propagation direction offset, the initial value of the mirror reflection intensity of each vertex at the preset light incident angle is calculated, and the spatial deformation parameters of adjacent vertices in the spatial coordinate mapping relationship are interpolated in a directional continuity manner to generate the reflection intensity distribution of the continuous area on the surface of the virtual object.

3. The method according to claim 1, characterized in that The dynamic posture data is integrated with the depth information obtained by the binocular vision positioning system to generate a virtual and real scene space coordinate mapping relationship, including: Establishing a three-dimensional coordinate system with the center of mass of the photographed object as the origin based on the baseline parameters of the binocular vision positioning system; converting the joint rotation angle and displacement in the dynamic posture data into a three-dimensional vector in the three-dimensional coordinate system; The three-dimensional vector is compositely superimposed with the three-dimensional coordinates of the joint feature points in the depth information obtained by the binocular vision positioning system to generate composite spatial data containing dynamic deformation parameters; Calculating the spatial offset of each vertex on the surface of the virtual object based on the deformation parameters in the composite spatial data; generating a motion trajectory of the vertices on the surface of the virtual object as the dynamic posture changes based on the spatial offset and the object contour boundary in the depth information obtained by the binocular vision positioning system; A dynamic mapping relationship between the vertex coordinates of the virtual object and the real scene coordinates is established based on the motion trajectory.

4. The method according to claim 1, wherein Based on the dynamic posture change sequence in the spatial coordinate mapping relationship, the shadow boundary transition effect is spatially and temporally aligned with the light source position of the real scene to complete the light and shadow synchronous rendering of the virtual and real scene, including: Calculating the movement direction and displacement speed of each vertex on the surface of the virtual object in the three-dimensional space coordinate system according to the posture difference of consecutive time nodes in the dynamic posture change sequence in the spatial coordinate mapping relationship; Extracting the light source movement trajectory of the real scene, and establishing a direction difference parameter between the movement direction in the light source movement trajectory and the movement direction of the virtual object vertex; Matching the brightness and darkness change rate of each pixel area in the shadow boundary transition effect with the direction difference parameter to obtain the shadow boundary deformation caused by the light source movement; Performing a time-series synchronous interpolation calculation on the displacement speed of the virtual object vertex and the light source movement trajectory according to the time interval of the dynamic posture change sequence, and generating a correlation ratio between the shadow shape amount and the light source movement speed; Based on the associated ratio, the light and dark distribution areas of the shadow boundary transition effect are dynamically scaled and adjusted so that the curvature of the virtual shadow boundary changes synchronously with the movement trajectory of the light source; the adjusted shadow boundary transition effect is superimposed and fused with the outline of the illumination area of the real scene light source, so that the deformation process of the virtual shadow boundary and the movement process of the real light source form a dynamic following relationship.

5. The method according to claim 1, wherein Performing directional continuity interpolation on the spatial deformation parameters of adjacent vertices in the spatial coordinate mapping relationship to generate a reflection intensity distribution of a continuous area on the surface of the virtual object, including: Calculating the deformation gradient change in the direction of the vertex connection line based on the difference in spatial deformation parameters between adjacent vertices in the spatial coordinate mapping relationship; Based on the deformation gradient change, multiple intermediate interpolation points are set on the line connecting adjacent vertices, and according to the distance ratio between each intermediate interpolation point and the adjacent vertex, the spatial deformation parameter interpolation result corresponding to each intermediate interpolation point is calculated; Performing weighted fusion on the interpolation result of the spatial deformation parameter and the initial value of the mirror reflection intensity of the corresponding vertex to obtain the reflection intensity value of the intermediate interpolation point; According to the reflection intensity values of all intermediate interpolation points, the reflection intensity distribution of the continuous area on the surface of the virtual object is constructed.

6. A virtual imaging system for virtual photography, configured to execute the virtual imaging method for virtual photography according to any one of claims 1 to 5, characterized in that: include: An acquisition module is used to synchronously acquire reflection characteristic data of the object surface using a multi-wavelength light source, input the reflection characteristic data into a pre-trained image feature extraction network, and generate shadow feature parameters including the specular reflectivity and diffuse reflectance coefficient of the object surface; A generation module is used to build a spatial posture monitoring array based on multiple motion sensors, collect dynamic posture data of the captured object in real time, and fuse the dynamic posture data with the depth information obtained by the binocular vision positioning system to generate a spatial coordinate mapping relationship between the virtual and real scenes; a calculation module, configured to input the deformation correction parameters in the mapping relationship between the mirror reflectivity and the spatial coordinates into a light propagation path simulation module to calculate the reflection intensity distribution of each area on the surface of the virtual object, and to adjust the relative position relationship between the virtual object and the light source in the real scene in real time according to the dynamic posture data; The generation module is further configured to input the reflection intensity distribution and the relative position relationship into a light propagation model including material attenuation characteristics to generate a shadow boundary transition effect with continuous deformation matching, wherein the deformation correction parameter is synchronously updated with the shadow boundary transition effect through the real-time posture change data of the spatial posture monitoring array; The alignment module is used to align the shadow boundary transition effect with the light source position of the real scene in time and space based on the dynamic posture change sequence in the spatial coordinate mapping relationship, so as to complete the synchronous light and shadow rendering of the virtual and real scene.

7. A computing device, characterized in that The method comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a virtual imaging method for virtual shooting as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, the virtual imaging method for virtual shooting according to any one of claims 1 to 5 is implemented.

Images