A video post-production method and system using virtual video technology

Abstract

The present invention discloses a video post-production method and system using virtual video technology, and relates to the technical field of video production. The method collects abundant camera internal parameters, vibration data and environmental data in the S1 stage, and obtains the camera parameter coefficient Csx through analysis and calculation in the S2 stage, thereby providing an accurate basis for subsequent virtual video production. This helps to eliminate camera parameter errors and improve the accuracy of video production. In the analysis and calculation of the S2 stage, the camera influence coefficient Yxx and the environmental shadow coefficient Hyx are obtained, and factors such as the external influence of the camera and the ambient lighting are fully considered, which helps to optimize the post-production more comprehensively. In the S4 stage, by using the first calibration, the second calibration results and the third calibration results to generate corresponding optimization strategies, the image data in the actual scene is projected by the camera, and a highly restored virtual video scene is generated, thereby improving the overall production efficiency and quality.

Description

Technical Field

[0001] The present invention relates to the technical field of video production, and in particular to a video post-production method and system using virtual video technology. Background Art

[0002] In this era of booming digital media, video production technology has been at the forefront of continuous innovation. Video post-production methods that apply virtual video technology are a key link in video creation, aiming to provide a more engaging and lifelike visual experience. However, despite our significant technological progress, traditional video post-production methods still face a series of challenges in some important aspects.

[0003] First, the accuracy and realism of traditional methods have obvious bottlenecks. As audiences' requirements for video quality increase day by day, the precise requirements for simulated cameras to operate in actual scenes have become more urgent. Traditional methods are difficult to fully restore factors such as lighting, shadows, and refraction in real scenes, resulting in insufficient accuracy and realism in the visual effects of the generated virtual videos. Secondly, traditional video post-production methods lack effective means to capture the internal and external parameters of the camera and environmental factors. There is a lack of effective means to accurately capture the internal and external parameters of the camera during operation, the influence of camera jitter and environmental factors. This limits the generation process of virtual videos and lacks sufficient controllability and stability. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides a video post-production method and system using virtual video technology to solve the problems mentioned in the background technology.

[0006] (II) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a video post-production method using virtual video technology, comprising the following steps:

[0008] S1. Collecting the original video material, and collecting the internal parameter data and the external parameter data of the camera in the actual scene of the original video shooting, to establish a camera parameter data set; collecting the vibration data and the moving speed data during the operation of the camera, to establish an impact data set; and collecting the environmental data of the actual scene, the environmental data including the light intensity, shadow data and refraction data, to establish an environmental data set;

[0009] S2, analyzing and calculating the camera parameter data set, the influence data set and the environment data set to obtain: the camera parameter coefficient Csx, the camera influence coefficient Yxx and the environment shadow coefficient Hyx;

[0010] S3. In the meta-video material, using a camera calibration algorithm, compare the camera parameter coefficient Csx with the first threshold to obtain a first calibration result; and compare the camera influence coefficient Yxx with the second threshold to obtain a second calibration result; and compare the ambient shadow coefficient Hyx with the third threshold to obtain a third calibration result;

[0011] S4, after generating corresponding optimization strategies using the first calibration results, the second calibration results and the third calibration results, perform camera projection on the image data in the actual scene, convert the two-dimensional image coordinates into three-dimensional world coordinates, and combine the three-dimensional coordinate point cloud obtained by the camera projection into a benchmark three-dimensional scene model; perform real-time rendering according to the corresponding optimization strategy to generate the final virtual video production product.

[0012] Preferably, the camera parameter data set includes camera internal parameter data and camera external parameter data;

[0013] The camera internal parameter data includes but is not limited to the following data: focal length value jj, radial distortion value jxjb, tangential distortion value qxjb, pixel size cxpm of the imaging plane and optical center distance value gxzx;

[0014] The camera external parameter data includes but is not limited to the following data: rotation matrix value xzjz, translation vector value psyl, camera angle value jdz and distance value jlz of the photographed object;

[0015] The focal length value jj is calculated by the camera calibration plate calibration algorithm; the radial distortion value jxjb and the tangential distortion value qxjb are obtained by image processing and calibration algorithm through the image collected during the camera calibration process; the pixel size cxpm of the imaging plane is calculated by the actual size of the pixel through the camera image information and the corresponding physical size recorded during the calibration process; the optical center distance value gxzx is calculated by detecting the feature points in the image during the calibration process of the camera calibration plate to obtain the position distance of the optical center;

[0016] The rotation matrix value xzjz is obtained by measuring the camera position measurement device; the translation vector value psyl is obtained by measuring the gyro sensor; the camera angle value jdz is obtained by measuring the torque sensor; and the distance value jlz of the photographed object is obtained by measuring the distance sensor.

[0017] Preferably, the influencing data set includes but is not limited to the following data: camera shake value ddz, camera moving speed ydsd and lens dust coefficient Hcx;

[0018] The camera shake value ddz is obtained by measuring by installing a vibration sensor on the camera; the camera movement speed is obtained by measuring by an acceleration sensor;

[0019] The lens dust coefficient Hcx is obtained by using a visual sensor to collect an overall image of the camera lens surface, identifying stains, dust or fingerprint features, obtaining the area mj1 of the stain, dust or fingerprint feature area, and generating the lens dust coefficient Hcx by the following formula:

[0020]

[0021] Where zmj represents the total surface area of ​​the lens.

[0022] Preferably, the environmental data set includes but is not limited to the following data: light intensity gzqd, color temperature value yswd, shadow value per square meter yyz, scene object transparency tmd and refractive index zsl;

[0023] The light intensity gzqd is obtained by collecting through a light sensor in an actual scene; the color temperature value yswd is obtained by measuring with a spectrometer; the shadow value yyz per square meter obtains the depth information of the object in the scene through a depth sensor, and calculates the position and shape of the shadow by comparing the position of the light source and the position of the object surface, and captures the image through a camera, and obtains the shadow area through image processing technology; the transparency of the scene object tmd is obtained by measuring with a projection spectrometer; the refractive index zsl measures the refraction angle through a refractometer, and calculates the refractive index through Snell's law.

[0024] Preferably, the camera parameter coefficient Csx is obtained by extracting the focal length value jj, radial distortion value jxjb, tangential distortion value qxjb, pixel size cxpm of the imaging plane and optical center distance value gxzx from the camera internal parameter data; after dimensionless processing, the camera internal parameter coefficient Nbxs is calculated by the following formula:

[0025]

[0026] Wherein, 0≤w1≤1, 0≤w2≤1, 0≤w3≤1, and w1+w2+w3=1, w1, w2 and w3 are weights, and their specific values ​​are adjusted and set by the user, C1 is the first constant correction coefficient, and ln 3 represents the logarithmic operation with base 3;

[0027] The rotation matrix value xzjz, translation vector value psyl, camera angle value jdz and distance value jlz of the photographed object in the camera external parameter data are extracted; after dimensionless processing, the camera external parameter coefficient Wbxs is calculated by the following formula:

[0028]

[0029] Wherein, 0≤w4≤1, 0≤w5≤1, 0≤w6≤1, and w4+w5+w6=1, w4, w5 and w6 are weights, and their specific values ​​are adjusted and set by the user, C2 is the second constant correction coefficient, and ln 2 represents a logarithmic operation with base 2;

[0030] Then the camera internal parameter coefficient Nbxs and the camera external parameter coefficient Wbxs are combined to generate the camera parameter coefficient Csx through the following correlation formula:

[0031] Csx=α*Nbxs+β*Wbxs

[0032] Wherein, 0≤α≤1, 0≤β≤1, and α+β=1, α and β are weights, and their specific values ​​are adjusted and set by the user. Preferably, the camera influence coefficient Yxx is obtained by extracting the camera shake value ddz, the camera movement speed ydsd and the lens dust coefficient Hcx in the influence data set; after dimensionless processing, the camera influence coefficient Yxx is generated by the following formula:

[0033] Hjx=w7*jddz 2 +w8*ydsd+w9*Hcx+C3

[0034] Wherein, 0≤w7≤1, 0≤w8≤1, 0≤w9≤1, and w7+w8+w9=1, w7, w8 and w9 are weights, and their specific values ​​are adjusted and set by the user, and C3 is the third constant correction coefficient;

[0035] The ambient shadow coefficient Hyx is obtained by extracting the ambient light intensity gzqd, the color temperature value yswd, the shadow value per square meter yyz, the scene object transparency tmd and the refractive index zsl from the environmental data set; after dimensionless processing, the ambient shadow coefficient Hyx is generated by the following formula:

[0036]

[0037] a1, a2, a3, a4 and a5 represent preset proportional coefficients of ambient light intensity gzqd, color temperature value yswd, shadow value per square meter yyz, scene object transparency tmd and refractive index zsl, respectively; among which: 0.15≤a1≤0.18, 0.08≤a2≤0.11, 0.25≤a3≤0.28, 0.21≤a4≤0.26, 0.11≤a5≤0.17, and 0.80≤a1+a2+a3+a4+a5≤1.0.

[0038] Preferably, the first calibration result includes:

[0039] When the camera parameter coefficient Csx ≥ the first threshold, it means that the calibration accuracy and parameters are qualified;

[0040] When the camera parameter coefficient Csx is less than the first threshold, it indicates that the calibration accuracy is unqualified, and a first optimization strategy is generated;

[0041] The second calibration result includes:

[0042] When the camera influence coefficient Yxx ≥ the second threshold, it means that the camera external influence is unqualified, and the second optimization strategy is generated;

[0043] When the camera influence coefficient Yxx is less than the second threshold, it indicates that the camera external influence is qualified;

[0044] The three calibration results include:

[0045] When the ambient shadow coefficient Hyx is less than the third threshold, it indicates that the ambient shadow is unqualified, and a third optimization strategy is generated;

[0046] When the ambient shadow coefficient Hyx≥the third threshold, it indicates that the ambient shadow is qualified.

[0047] Preferably, the first optimization strategy includes: adjusting the focal length of the camera to reach a calibration threshold; correcting and optimizing the camera position, including the focal length and the principal point position; optimizing and adjusting the rotation matrix, the translation vector, the camera angle and the distance of the photographed object to reach a standard threshold range;

[0048] The second optimization strategy includes: adjusting the focal length of the camera to reach the calibration threshold; correcting and using a more stable camera bracket to reduce the vibration and shaking of the camera; using a dust cover after cleaning the camera lens to ensure that the lens remains clean, and using a stable mobile device to slow down the movement of the camera;

[0049] The third optimization strategy includes: adding lighting equipment, adjusting the position of the camera scene, optimizing the intensity and uniformity of light, optimizing the layout of objects in the scene, and controlling the number of objects to reduce shadows.

[0050] Preferably, S4 includes:

[0051] S41, camera projection: using the first optimization strategy, the second optimization strategy and the third optimization strategy, applying the camera parameter coefficient Csx, the camera influence coefficient Yxx and the ambient shadow coefficient Hyx to the camera projection model; converting the two-dimensional image coordinates in the actual scene into three-dimensional world coordinates to generate a three-dimensional coordinate point cloud;

[0052] S42, generating a reference 3D scene model: combining the 3D coordinate point cloud obtained by camera projection into a reference 3D scene model; constructing a basic virtual scene according to the structure and texture information of the 3D scene model;

[0053] S43, real-time rendering: using the optimized parameters and the benchmark 3D scene model, based on the meta-video material, applying the real-time rendering engine to generate images; using the graphics processing unit GPU hardware acceleration technology, real-time calculation of lighting, shadow, texture mapping visual effects to obtain a virtual scene;

[0054] S44. On the basis of real-time rendering, combine each frame of the image sequence into the final virtual video, apply video editing technology, add audio, adjust clips and apply special effects to generate the final virtual video production product.

[0055] A video post-production system using virtual video technology, comprising an original video material acquisition module, a camera parameter data acquisition module, an impact data acquisition module, an environmental data acquisition module, a data analysis and calculation module, a calibration and optimization module, a real-time rendering module and a video editing and synthesis module;

[0056] The original video material acquisition module is responsible for acquiring original video materials;

[0057] The camera parameter data acquisition module is used to collect internal and external parameter data of the camera and establish a camera parameter data set;

[0058] The impact data collection module is responsible for collecting vibration data and movement speed data during the operation of the camera and establishing an impact data set;

[0059] The environmental data acquisition module is used to collect data including light intensity, shadow data and refraction data in the actual shooting, and establish an environmental data set;

[0060] The data analysis and calculation module is used to analyze and calculate the camera parameter data set to obtain the camera parameter coefficient Csx; analyze and calculate the influence data set to obtain the camera influence coefficient Yxx; analyze and calculate the environment data set to obtain the environment shadow coefficient Hyx;

[0061] The calibration and optimization module is used to use a camera calibration algorithm in the meta-video material to compare the camera parameter coefficient Csx with a first threshold value to obtain a first calibration result; and compare the camera influence coefficient Yxx with a second threshold value to obtain a second calibration result; and compare the ambient shadow coefficient Hyx with a third threshold value to obtain a third calibration result; and analyze the first calibration result, the second calibration result and the third calibration result, determine the calibration accuracy and parameter eligibility, and generate corresponding optimization strategies, including adjusting the camera focal length, correcting the position and posture parameters;

[0062] The real-time rendering module is used to apply the optimized parameters to the camera projection model, generate a three-dimensional coordinate point cloud, combine it into a benchmark three-dimensional scene model, and apply the real-time rendering engine to generate images, including lighting, shadow and texture mapping effects;

[0063] The video editing and synthesis module is used to combine each frame of the image sequence into a final virtual video, add audio, perform editing and apply special effects to generate a final virtual video production product.

[0064] (III) Beneficial effects

[0065] The present invention provides a video post-production method and system using virtual video technology, which has the following beneficial effects:

[0066] (1) This video post-production method using virtual video technology collects rich camera internal parameters, vibration data and environmental data in the S1 stage, and obtains the camera parameter coefficient Csx through analysis and calculation in the S2 stage, which provides an accurate basis for subsequent virtual video production. This helps to eliminate camera parameter errors and improve the accuracy of video production. The data set collected in the S1 stage includes vibration data and movement speed data during camera operation, as well as environmental data of the actual scene. In the analysis and calculation of the S2 stage, the camera influence coefficient Yxx and the environmental shadow coefficient Hyx are obtained, which fully considers factors such as the external influence of the camera and environmental lighting, which helps to optimize post-production more comprehensively.

[0067] (2) In the video post-production method using virtual video technology, in the S4 stage, the image data in the actual scene is projected by the camera by using the optimization strategy generated by the first calibration result, the second calibration result and the third calibration result, so as to generate a highly restored virtual video scene. This step effectively integrates virtual elements into the actual scene, creates a more realistic visual effect, and improves the audience's viewing experience.

[0068] (3) This video post-production method using virtual video technology ensures that the finished virtual video production is more realistic visually by combining real-time rendering technology and real-time adjustment of the scene according to the optimization strategy in the S4 stage. This combination is relatively new in video production, making post-production more flexible and adaptable to different requirements and changes, and improving the overall production efficiency and quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A schematic diagram of the steps of a video post-production method using virtual video technology according to the present invention;

[0070] Figure 2 The present invention is a schematic diagram of a video post-production system block diagram using virtual video technology. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] Example 1

[0073] See also Figure 1 The present invention provides a video post-production method using virtual video technology, please refer to Figure 1 , including the following steps,

[0074] S1. Collecting the original video material, and collecting the internal parameter data and the external parameter data of the camera in the actual scene of the original video shooting, to establish a camera parameter data set; collecting the vibration data and the moving speed data during the operation of the camera, to establish an impact data set; and collecting the environmental data of the actual scene, the environmental data including the light intensity, shadow data and refraction data, to establish an environmental data set;

[0075] S2, analyzing and calculating the camera parameter data set, the influence data set and the environment data set to obtain: the camera parameter coefficient Csx, the camera influence coefficient Yxx and the environment shadow coefficient Hyx;

[0076] S3. In the meta-video material, using a camera calibration algorithm, compare the camera parameter coefficient Csx with the first threshold to obtain a first calibration result; and compare the camera influence coefficient Yxx with the second threshold to obtain a second calibration result; and compare the ambient shadow coefficient Hyx with the third threshold to obtain a third calibration result;

[0077] S4, after generating corresponding optimization strategies using the first calibration results, the second calibration results and the third calibration results, perform camera projection on the image data in the actual scene, convert the two-dimensional image coordinates into three-dimensional world coordinates, and combine the three-dimensional coordinate point cloud obtained by the camera projection into a benchmark three-dimensional scene model; perform real-time rendering according to the corresponding optimization strategy to generate the final virtual video production product.

[0078] In this embodiment, the present invention provides a video post-production method using virtual video technology, which innovatively integrates key steps such as data collection, analysis and calculation, and optimization strategy generation. By collecting the internal parameters of the camera, vibration data, and environmental data of the actual scene in the meta-video material, the method performs a comprehensive analysis and calculation in the S2 stage to obtain the camera parameter coefficient Csx, the camera influence coefficient Yxx, and the environmental shadow coefficient Hyx. Then, in the S3 stage, these parameters are compared through the camera calibration algorithm to obtain accurate calibration results. Finally, through the optimization strategy generation and real-time rendering in the S4 stage, the camera projection of the image data of the actual scene is realized, and a highly restored virtual video production product is generated.

[0079] This novel approach helps overcome the limitations of traditional video production methods and brings higher accuracy and realism to virtual video production. By using comprehensive data sets and advanced analytical computing techniques, the invention not only improves the controllability and stability of video production, but also creates a more fascinating and realistic visual experience for users.

[0080] Embodiment 2: This embodiment is an explanation of Embodiment 1. Specifically, the camera parameter data set includes camera internal parameter data and camera external parameter data.

[0081] The camera internal parameter data includes but is not limited to the following data: focal length value jj, radial distortion value jxjb, tangential distortion value qxjb, pixel size cxpm of the imaging plane and optical center distance value gxzx;

[0082] The camera external parameter data includes but is not limited to the following data: rotation matrix value xzjz, translation vector value psyl, camera angle value jdz and distance value jlz of the photographed object;

[0083] The focal length value jj is calculated by the camera calibration plate calibration algorithm; the radial distortion value jxjb and the tangential distortion value qxjb are obtained by image processing and calibration algorithm through the image collected during the camera calibration process; the pixel size cxpm of the imaging plane is calculated by the actual size of the pixel through the camera image information and the corresponding physical size recorded during the calibration process; the optical center distance value gxzx is calculated by detecting the feature points in the image during the calibration process of the camera calibration plate to obtain the position distance of the optical center;

[0084] The rotation matrix value xzjz is obtained by measuring the camera position measurement device; the translation vector value psyl is obtained by measuring the gyro sensor; the camera angle value jdz is obtained by measuring the torque sensor; and the distance value jlz of the photographed object is obtained by measuring the distance sensor.

[0085] In this embodiment, the internal and external parameters of the camera, such as focal length, distortion value, translation vector, rotation matrix, etc., are successfully obtained through methods such as camera calibration board calibration algorithm, image processing and sensor measurement. This helps to accurately restore the real scene in virtual video production and improve the fidelity of visual effects. Using a variety of sensors such as position measurement equipment, gyro sensors, torque sensors and ranging sensors, various parameters of the camera operation process can be obtained in real time. This ensures that the virtual video can be dynamically adjusted and optimized during the video production process to make it more in line with the changes in the actual scene. By comprehensively calibrating and correcting the internal and external parameters of the camera, the virtual video can better adapt to various complex scenes during the production process. This helps to eliminate the problems caused by inaccurate parameters in traditional video post-production methods and improve the controllability and stability of the generated video. By fusing the data of multiple sensors, such as camera calibration boards, gyro sensors, torque sensors and ranging sensors, a more comprehensive and real camera parameter data set is formed. This multi-sensor fusion application provides more comprehensive and reliable data support for virtual video production.

[0086] Example 3: This example is an explanation of Example 1. Specifically, the influencing data set includes but is not limited to the following data: camera shake value ddz, camera moving speed ydsd and lens dust coefficient Hcx;

[0087] The camera shake value ddz is obtained by measuring by installing a vibration sensor on the camera; the camera movement speed is obtained by measuring by an acceleration sensor;

[0088] The lens dust coefficient Hcx is obtained by using a visual sensor to collect an overall image of the camera lens surface, identifying stains, dust or fingerprint features, obtaining the area mj1 of the stain, dust or fingerprint feature area, and generating the lens dust coefficient Hcx by the following formula:

[0089]

[0090] Where zmj represents the total surface area of ​​the lens.

[0091] In this embodiment, the acquisition of the data set is affected, especially the accurate measurement of camera shake, moving speed and lens dust coefficient Hcx; by installing a vibration sensor on the camera, the camera shake value is successfully measured and obtained. This helps to suppress and adjust the movement of the camera during the virtual video production process, and improve the stability and viewing experience of the video. The camera movement speed is measured in real time using an acceleration sensor. Such data provides accurate information about the movement of the camera in the actual scene, so that the virtual video can better simulate the movement in the real environment. By using a visual sensor to collect the overall image of the camera lens surface, features such as stains, dust or fingerprints are successfully identified, and the area of ​​these feature areas is calculated. Through this process, the lens dust coefficient can be calculated in real time, providing a more realistic visual effect for the production of virtual videos. By real-time monitoring and adjustment of factors such as camera shake, moving speed and lens dust, the production of virtual videos can be better adapted to the actual scene, improving the quality and viewing experience of the video. The audience will be able to enjoy more stable, real and clear virtual video content. The impact data set collection method in the invention not only increases the real-time monitoring of camera movement and lens status, but also further improves the realism and fidelity of virtual video by real-time calculation of factors such as lens dust coefficient Hcx, providing a better visual experience in the digital media era.

[0092] Example 4: This example is an explanation of Example 1. Specifically, the environmental data set includes but is not limited to the following data: light intensity gzqd, color temperature value yswd, shadow value per square meter yyz, scene object transparency tmd and refractive index zsl;

[0093] The light intensity gzqd is obtained by collecting through a light sensor in an actual scene; the color temperature value yswd is obtained by measuring with a spectrometer; the shadow value yyz per square meter obtains the depth information of the object in the scene through a depth sensor, and calculates the position and shape of the shadow by comparing the position of the light source and the position of the object surface, and captures the image through a camera, and obtains the shadow area through image processing technology; the transparency of the scene object tmd is obtained by measuring with a projection spectrometer; the refractive index zsl measures the refraction angle through a refractometer, and calculates the refractive index through Snell's law.

[0094] In this embodiment, the light intensity is collected in real time by the light sensor, so that the virtual video can restore the light intensity of the real scene under different lighting conditions. This helps to improve the realism of the virtual video and make it closer to the actual shooting environment. By using a spectrometer to measure the color temperature value, the virtual video production can more accurately adjust the color temperature of the scene to make it consistent with the spectral characteristics of the real environment and improve the color restoration of the video. The depth sensor obtains the shadow value per square meter, calculates the position and shape of the shadow by comparing the position of the light source and the surface of the object, and detects the shadow area through image processing technology. This makes the shadow in the virtual video more accurately reflect the occlusion relationship of the object in the actual scene and improves the visual realism. The transparency and refractive index of the scene objects are measured by the projection spectrometer and the refractometer, so that the virtual video can more accurately restore the optical properties of the transparent objects in the scene. This helps to improve the realism of transparent objects in the virtual video and make them appear more realistic in the virtual environment.

[0095] Embodiment 5: This embodiment is an explanation of Embodiment 1. Specifically, the camera parameter coefficient Csx is obtained by extracting the focal length value jj, radial distortion value jxjb, tangential distortion value qxjb, pixel size cxpm of the imaging plane and optical center distance value gxzx from the camera internal parameter data; after dimensionless processing, the camera internal parameter coefficient Nbxs is calculated by the following formula:

[0096]

[0097] Wherein, 0≤w1≤1, 0≤w2≤1, 0≤w3≤1, and w1+w2+w3=1, w1, w2 and w3 are weights, and their specific values ​​are adjusted and set by the user, C1 is the first constant correction coefficient, and ln 3 represents the logarithmic operation with base 3;

[0098] The rotation matrix value xzjz, translation vector value psyl, camera angle value jdz and distance value jlz of the photographed object in the camera external parameter data are extracted; after dimensionless processing, the camera external parameter coefficient Wbxs is calculated by the following formula:

[0099]

[0100] Wherein, 0≤w4≤1, 0≤w5≤1, 0≤w6≤1, and w4+w5+w6=1, w4, w5 and w6 are weights, and their specific values ​​are adjusted and set by the user, C2 is the second constant correction coefficient, and ln 2 represents a logarithmic operation with base 2;

[0101] Then the camera internal parameter coefficient Nbxs and the camera external parameter coefficient Wbxs are combined to generate the camera parameter coefficient Csx through the following correlation formula:

[0102] Csx=α*Nbxs+β*Wbxs

[0103] In the formula, 0≤α≤1, 0≤β≤1, and α+β=1, α and β are weights, and their specific values ​​are adjusted and set by the user.

[0104] In this embodiment, by extracting the camera's internal and external parameter data, including focal length, distortion value, translation vector, etc., dimensionless processing and weight correction are used to generate the camera's internal parameter coefficient Nbxs and external parameter coefficient Wbxs. The camera parameter coefficient Csx calculation process helps to accurately restore the camera's internal and external parameters in actual shooting, and improves the accuracy of virtual video in camera parameter simulation.

[0105] Embodiment 6: This embodiment is an explanation of Embodiment 1. Specifically, the camera influence coefficient Yxx is obtained by extracting the camera shake value ddz, the camera movement speed ydsd and the lens dust coefficient Hcx in the influence data set; after dimensionless processing, the camera influence coefficient Yxx is generated by the following formula:

[0106] Hjx=w7*jddz 2 +w8*ydsd+w9*Hcx+C3

[0107] Wherein, 0≤w7≤1, 0≤w8≤1, 0≤w9≤1, and w7+w8+w9=1, w7, w8 and w9 are weights, and their specific values ​​are adjusted and set by the user, and C3 is the third constant correction coefficient;

[0108] The ambient shadow coefficient Hyx is obtained by extracting the ambient light intensity gzqd, the color temperature value yswd, the shadow value per square meter yyz, the scene object transparency tmd and the refractive index zsl from the environmental data set; after dimensionless processing, the ambient shadow coefficient Hyx is generated by the following formula:

[0109]

[0110] a1, a2, a3, a4 and a5 represent preset proportional coefficients of ambient light intensity gzqd, color temperature value yswd, shadow value per square meter yyz, scene object transparency tmd and refractive index zsl, respectively; among which: 0.15≤a1≤0.18, 0.08≤a2≤0.11, 0.25≤a3≤0.28, 0.21≤a4≤0.26, 0.11≤a5≤0.17, and 0.80≤a1+a2+a3+a4+a5≤1.0.

[0111] In this embodiment, the camera influence coefficient Yxx takes into account multiple factors such as camera shake value, moving speed and lens dust coefficient, and generates the camera influence coefficient Yxx through dimensionless processing and weight correction. This makes the influencing factors of virtual video in the process of simulating camera operation more comprehensive and accurate. The environmental shadow coefficient Hyx comprehensively considers multiple environmental factors such as ambient light intensity, color temperature value, shadow value per square meter, transparency and refractive index of scene objects. By combining these factors and calculating the environmental shadow coefficient Hyx, it is possible to more comprehensively simulate the lighting and shadow conditions in the real scene and improve the realism of the virtual video. By setting the value range of a1, a2, a3, a4 and a5 and limiting their sum, it is ensured that the calculation result is within a certain range to prevent the influence of extreme values. This helps to generate a more stable and controllable environmental shadow coefficient and improves the reliability of the calculation method.

[0112] Embodiment 7: This embodiment is an explanation of Embodiment 1. Specifically, the first calibration result includes:

[0113] When the camera parameter coefficient Csx ≥ the first threshold, it means that the calibration accuracy and parameters are qualified;

[0114] When the camera parameter coefficient Csx is less than the first threshold, it indicates that the calibration accuracy is unqualified, and a first optimization strategy is generated;

[0115] The second calibration result includes:

[0116] When the camera influence coefficient Yxx ≥ the second threshold, it means that the camera external influence is unqualified, and the second optimization strategy is generated;

[0117] When the camera influence coefficient Yxx is less than the second threshold, it indicates that the camera external influence is qualified;

[0118] The three calibration results include:

[0119] When the ambient shadow coefficient Hyx is less than the third threshold, it indicates that the ambient shadow is unqualified, and a third optimization strategy is generated;

[0120] When the ambient shadow coefficient Hyx≥the third threshold, it indicates that the ambient shadow is qualified.

[0121] In this embodiment, the setting of the first calibration result includes the determination of calibration accuracy and parameter eligibility, providing an intuitive and effective calibration result feedback. When the camera parameter coefficient Csx reaches or exceeds the first threshold, it indicates that the calibration accuracy and parameters are qualified, reflecting the accurate calibration of the internal and external parameters of the camera. When the camera parameter coefficient Csx is lower than the first threshold, it indicates that the calibration accuracy is unqualified, thereby triggering the generation of the first optimization strategy, providing a direction for subsequent optimization. The setting of the second calibration result focuses more on the influence of external factors of the camera. When the camera influence coefficient Yxx reaches or exceeds the second threshold, it means that the external influence of the camera is unqualified, which may include factors such as camera jitter and moving speed. Conversely, when the camera influence coefficient Yxx is lower than the second threshold, it means that the external influence of the camera is qualified. Such a determination helps to identify and adjust the external influence of the camera in different scenarios, and provides key information for improving video quality. The setting of the third calibration result involves the environmental shadow coefficient Hyx, that is, the influence of environmental factors on the virtual video. When the environmental shadow coefficient Hyx is lower than the third threshold, it means that the environmental shadow is unqualified, which may involve factors such as lighting and color temperature. On the contrary, when the environmental shadow coefficient Hyx reaches or exceeds the third threshold, it indicates that the environmental shadow is qualified. This determination helps to optimize the environmental effect of the virtual video and improve the fidelity and realism of the video.

[0122] This multi-level calibration result setting fully considers the influence of the internal and external parameters of the camera and environmental factors, and provides a clear direction for subsequent optimization strategies. This beneficial effect is reflected in improving the controllability and adaptability of the virtual video production process, ensuring that the final virtual video is more in line with the actual scene and meets the user's high requirements for video quality.

[0123] Embodiment 8: This embodiment is explained in Embodiment 1. Specifically, the first optimization strategy includes: adjusting the focal length of the camera to reach the calibration threshold; correcting and optimizing the camera position, including the focal length and the principal point position; optimizing and adjusting the rotation matrix, the translation vector, the camera angle and the distance of the photographed object to reach the standard threshold range;

[0124] The second optimization strategy includes: adjusting the focal length of the camera to reach the calibration threshold; correcting and using a more stable camera bracket to reduce the vibration and shaking of the camera; using a dust cover after cleaning the camera lens to ensure that the lens remains clean, and using a stable mobile device to slow down the movement of the camera;

[0125] The third optimization strategy includes: adding lighting equipment, adjusting the position of the camera scene, optimizing the intensity and uniformity of light, optimizing the layout of objects in the scene, and controlling the number of objects to reduce shadows.

[0126] In this embodiment, the setting of the first optimization strategy revolves around the internal and external parameters of the camera, providing a multi-faceted adjustment scheme to improve the calibration accuracy and eligibility. By adjusting the camera focal length, correcting the camera position (including focal length and principal point position), and optimizing the rotation matrix, translation vector, camera angle and distance of the object being photographed, it is intended to make these parameters reach the standard threshold range. Such adjustments help to optimize the internal and external parameters of the camera and improve the realism and fidelity of the virtual video. The main focus of the second optimization strategy is to reduce the external influence of the camera. By adjusting the camera focal length, using a more stable camera bracket to reduce vibration and shaking, and keeping the lens clean, the external factors of the camera are effectively controlled. The use of these methods helps to reduce the jitter and instability factors that may appear in the virtual video and improve the shooting quality and stability of the video. The third optimization strategy focuses on the optimization of environmental factors. By adding lighting equipment, adjusting the position of the camera scene, optimizing the intensity and uniformity of light, and optimizing the layout of objects in the scene, it helps to improve the shadow effect in the virtual video. This strategy aims to improve the visual effect of the video and make it more vivid and fascinating.

[0127] Embodiment 9: This embodiment is an explanation of Embodiment 1. Specifically, S4 includes:

[0128] S41, camera projection: using the first optimization strategy, the second optimization strategy and the third optimization strategy, applying the camera parameter coefficient Csx, the camera influence coefficient Yxx and the ambient shadow coefficient Hyx to the camera projection model; converting the two-dimensional image coordinates in the actual scene into three-dimensional world coordinates to generate a three-dimensional coordinate point cloud;

[0129] S42, generating a reference 3D scene model: combining the 3D coordinate point cloud obtained by camera projection into a reference 3D scene model; constructing a basic virtual scene according to the structure and texture information of the 3D scene model;

[0130] S43, real-time rendering: using the optimized parameters and the benchmark 3D scene model, based on the meta-video material, applying the real-time rendering engine to generate images; using the graphics processing unit GPU hardware acceleration technology, real-time calculation of lighting, shadow, texture mapping visual effects to obtain a virtual scene;

[0131] S44. On the basis of real-time rendering, combine each frame of the image sequence into the final virtual video, apply video editing technology, add audio, adjust clips and apply special effects to generate the final virtual video production product.

[0132] In this embodiment, the first optimization strategy, the second optimization strategy and the third optimization strategy are used to apply the camera parameter coefficient Csx, the camera influence coefficient Yxx and the ambient shadow coefficient Hyx to the camera projection model. This step converts the two-dimensional image coordinates of the actual scene into three-dimensional world coordinates to generate a three-dimensional coordinate point cloud. The beneficial effect is that the authenticity and accuracy of the image are improved. It helps to improve the efficiency of virtual video production and ensure that the generated virtual video achieves high-quality visual effects and viewing experience.

[0133] Example 10, a video post-production system using virtual video technology, please refer to Figure 2 , including original video material acquisition module, camera parameter data acquisition module, impact data acquisition module, environmental data acquisition module, data analysis and calculation module, calibration and optimization module, real-time rendering module and video editing and synthesis module;

[0134] The original video material acquisition module is responsible for acquiring original video materials;

[0135] The camera parameter data acquisition module is used to collect internal and external parameter data of the camera and establish a camera parameter data set;

[0136] The impact data collection module is responsible for collecting vibration data and movement speed data during the operation of the camera and establishing an impact data set;

[0137] The environmental data acquisition module is used to collect data including light intensity, shadow data and refraction data in the actual shooting, and establish an environmental data set;

[0138] The data analysis and calculation module is used to analyze and calculate the camera parameter data set to obtain the camera parameter coefficient Csx; analyze and calculate the influence data set to obtain the camera influence coefficient Yxx; analyze and calculate the environment data set to obtain the environment shadow coefficient Hyx;

[0139] The calibration and optimization module is used to use a camera calibration algorithm in the meta-video material to compare the camera parameter coefficient Csx with a first threshold value to obtain a first calibration result; and compare the camera influence coefficient Yxx with a second threshold value to obtain a second calibration result; and compare the ambient shadow coefficient Hyx with a third threshold value to obtain a third calibration result; and analyze the first calibration result, the second calibration result and the third calibration result, determine the calibration accuracy and parameter eligibility, and generate corresponding optimization strategies, including adjusting the camera focal length, correcting the position and posture parameters;

[0140] The real-time rendering module is used to apply the optimized parameters to the camera projection model, generate a three-dimensional coordinate point cloud, combine it into a benchmark three-dimensional scene model, and apply the real-time rendering engine to generate images, including lighting, shadow and texture mapping effects;

[0141] The video editing and synthesis module is used to combine each frame of the image sequence into a final virtual video, add audio, perform editing and apply special effects to generate a final virtual video production product.

[0142] In this embodiment, the system provides a comprehensive and efficient solution for video production using virtual video technology through meticulous data collection, analysis and optimization steps, as well as key technologies such as real-time rendering and video editing. The beneficial effect of the system is to enhance the realism, fidelity and entertainment of virtual videos, meeting the demand for high-quality visual experience in today's digital media era.

[0143] Application example: In a science fiction movie, the actual scene is a city street. The camera equipment captures the video footage of the street and records the camera's position, angle, and environmental lighting, shadows, and reflections. The captured video footage is analyzed using the camera calibration algorithm to obtain the camera parameter coefficient Csx, the camera influence coefficient Yxx, and the environmental shadow coefficient Hyx. By comparing these data with the pre-set threshold, the first calibration result, the second calibration result, and the third calibration result are obtained to determine the calibration accuracy and parameter eligibility. This includes adjusting the camera focal length, correcting the position and attitude parameters; for unqualified situations, generating corresponding optimization strategies, which involve adjusting the camera focal length, correcting the position and attitude parameters, etc.; using the camera calibration algorithm, these data are aligned to form a benchmark three-dimensional city model.

[0144] On this model, computer graphics technology is used to generate a three-dimensional model of a flying car as a virtual element. Through real-time rendering, the position and angle of the car are adjusted to ensure that it crosses the street realistically, and the appearance of the car is adjusted according to the effect of real-time observation.

[0145] Subsequently, color correction is performed to ensure that the color of the virtual car is consistent with the lighting of the actual scene. Finally, in the rendering and output stage, a high-quality virtual video containing the virtual car is generated and presented in the movie, creating a sci-fi future city scene.

[0146] Real-time rendering technology: Shading and lighting: Use real-time rendering technology to shade and illuminate the scene to ensure that the virtual objects are consistent with the lighting and color of the actual scene. Texture mapping: Use texture mapping technology to map the optimized image information to the 3D scene model to improve the realism of the virtual scene. Shadow processing: Calculate shadows in real time, taking into account the position and intensity of the light source to ensure that the projection of virtual objects in the scene looks natural. Depth and perspective effects: Use depth information and perspective technology to make the depth of virtual objects more consistent with the actual scene and enhance the sense of reality. Real-time response: Use the performance of the real-time rendering engine to achieve real-time response of the scene and adapt to camera or scene changes.

[0147] Hardware acceleration: Use hardware acceleration technologies such as GPU to improve the speed and efficiency of real-time rendering. In the rendering and output stages, generate high-quality virtual videos containing virtual cars, add audio, adjust clips, and apply special effects through video editing and synthesis technology, and finally present them in the movie.

[0148] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A video post-production method using virtual video technology, characterized in that: The following steps are included: S1, collecting the original video material, and collecting the camera internal parameter data and the camera external parameter data in the actual scene of the original video shooting, and establishing a camera parameter data set; The vibration data and movement speed data of the camera during operation are collected to establish an impact data set; and collecting environmental data of the actual scene, wherein the environmental data includes light intensity, shadow data and refraction data, and establishing an environmental data set; S2, analyzing and calculating the camera parameter data set, the influence data set and the environment data set to obtain: the camera parameter coefficient Csx, the camera influence coefficient Yxx and the environment shadow coefficient Hyx; S3. In the original video material, using a camera calibration algorithm, compare the camera parameter coefficient Csx with the first threshold to obtain a first calibration result; and compare the camera influence coefficient Yxx with the second threshold to obtain a second calibration result; and comparing the ambient shadow coefficient Hyx with the third threshold to obtain a third calibration result; The first calibration result includes: When the camera parameter coefficient Csx ≥ the first threshold, it means that the calibration accuracy and parameters are qualified; When the camera parameter coefficient Csx is less than the first threshold, it indicates that the calibration accuracy is unqualified, and a first optimization strategy is generated; The second calibration result includes: When the camera influence coefficient Yxx ≥ the second threshold, it means that the camera external influence is unqualified, and the second optimization strategy is generated; When the camera influence coefficient Yxx is less than the second threshold, it indicates that the camera external influence is qualified; The three calibration results include: When the ambient shadow coefficient Hyx is less than the third threshold, it indicates that the ambient shadow is unqualified, and a third optimization strategy is generated; When the ambient shadow coefficient Hyx ≥ the third threshold, it means that the ambient shadow is qualified; The first optimization strategy includes: adjusting the focal length of the camera to reach the calibration threshold; correcting and optimizing the camera position, including the focal length and the principal point position; optimizing and adjusting the rotation matrix, translation vector, camera angle and the distance of the photographed object to reach the standard threshold range; The second optimization strategy includes: adjusting the focal length of the camera to reach the calibration threshold; correcting and using a more stable camera bracket to reduce the vibration and shaking of the camera; using a dust cover after cleaning the camera lens to ensure that the lens remains clean, and using a stable mobile device to slow down the movement of the camera; The third optimization strategy includes: adding lighting equipment, adjusting the camera scene position, optimizing the light intensity and uniformity, and optimizing the layout of objects in the scene to control the number of objects that reduce shadows; S4, after generating corresponding optimization strategies using the first calibration results, the second calibration results and the third calibration results, perform camera projection on the image data in the actual scene, convert the two-dimensional image coordinates into three-dimensional world coordinates, and combine the three-dimensional coordinate point cloud obtained by the camera projection into a benchmark three-dimensional scene model; perform real-time rendering according to the corresponding optimization strategy to generate the final virtual video production product.

2. The video post-production method using virtual video technology according to claim 1, characterized in that: The camera parameter data set includes camera internal parameter data and camera external parameter data; The camera internal parameter data includes the following data: focal length value jj, radial distortion value jxjb, tangential distortion value qxjb, pixel size cxpm of the imaging plane and optical center distance value gxzx; The camera external parameter data includes the following data: rotation matrix value xzjz, translation vector value psyl, camera angle value jdz and distance value jlz of the photographed object; The focal length value jj is calculated by the camera calibration plate calibration algorithm; the radial distortion value jxjb and the tangential distortion value qxjb are obtained by image processing and calibration algorithm through the image collected during the camera calibration process; the pixel size cxpm of the imaging plane is calculated by the actual size of the pixel through the camera image information and the corresponding physical size recorded during the calibration process; the optical center distance value gxzx is calculated by detecting the feature points in the image during the calibration process of the camera calibration plate to obtain the position distance of the optical center; The rotation matrix value xzjz is obtained by measuring the camera position measurement device; the translation vector value psyl is obtained by measuring the gyro sensor; the camera angle value jdz is obtained by measuring the torque sensor; and the distance value jlz of the photographed object is obtained by measuring the distance sensor.

3. The video post-production method using virtual video technology according to claim 1, characterized in that: The influencing data set includes the following data: camera shake value ddz, camera moving speed ydsd and lens dust coefficient Hcx; The camera shake value ddz is obtained by measuring by installing a vibration sensor on the camera; the camera movement speed is obtained by measuring by an acceleration sensor; The lens dust coefficient Hcx is obtained by using a visual sensor to collect an overall image of the camera lens surface, identifying stains, dust or fingerprint features, obtaining the area mj1 of the stain, dust or fingerprint feature area, and generating the lens dust coefficient Hcx by the following formula: ； Where zmj represents the total surface area of ​​the lens.

4. The video post-production method using virtual video technology according to claim 1, characterized in that: The environmental data set includes the following data: light intensity gzqd, color temperature value yswd, shadow value per square meter yyz, scene object transparency tmd and refractive index zsl; The light intensity gzqd is obtained by collecting through a light sensor in an actual scene; the color temperature value yswd is obtained by measuring with a spectrometer; the shadow value yyz per square meter obtains the depth information of the object in the scene through a depth sensor, and calculates the position and shape of the shadow by comparing the position of the light source and the position of the object surface, and captures the image through a camera, and obtains the shadow area through image processing technology; the transparency of the scene object tmd is obtained by measuring with a projection spectrometer; the refractive index zsl measures the refraction angle through a refractometer, and calculates the refractive index through Snell's law.

5. The video post-production method using virtual video technology according to claim 1, characterized in that: The camera parameter coefficient Csx is obtained by extracting the focal length value jj, radial distortion value jxjb, tangential distortion value qxjb, pixel size cxpm of the imaging plane and optical center distance value gxzx from the camera internal parameter data; after dimensionless processing, the camera internal parameter coefficient Nbxs is calculated by the following formula: ； in, , , ,and , w1, w2 and w3 are weights, and their specific values ​​are adjusted by the user is the first constant correction coefficient, Indicates logarithmic operation with base 3; The rotation matrix value xzjz, translation vector value psyl, camera angle value jdz and distance value jlz of the photographed object in the camera external parameter data are extracted; after dimensionless processing, the camera external parameter coefficient Wbxs is calculated by the following formula: ； in, , , ,and , w4, w5 and w6 are weights, and their specific values ​​are adjusted by the user is the second constant correction coefficient, Indicates logarithmic operation with base 2; Then the camera internal parameter coefficient Nbxs and the camera external parameter coefficient Wbxs are combined to generate the camera parameter coefficient Csx through the following correlation formula: ； In the formula, , ,and , is the weight, and its specific value is adjusted by the user.

6. The video post-production method using virtual video technology according to claim 1, characterized in that: The camera influence coefficient Yxx is obtained by extracting the camera shake value ddz, the camera movement speed ydsd and the lens dust coefficient Hcx in the influence data set; after dimensionless processing, the camera influence coefficient Yxx is generated by the following formula: ； in, , , ,and , w7, w8 and w9 are weights, and their specific values ​​are adjusted by the user is the third constant correction coefficient; The ambient shadow coefficient Hyx is obtained by extracting the ambient light intensity gzqd, the color temperature value yswd, the shadow value per square meter yyz, the scene object transparency tmd and the refractive index zsl from the environmental data set; after dimensionless processing, the ambient shadow coefficient Hyx is generated by the following formula: ； a1, a2, a3, a4 and a5 represent preset proportional coefficients of ambient light intensity gzqd, color temperature value yswd, shadow value per square meter yyz, scene object transparency tmd and refractive index zsl, respectively; among which: 0.15≤a1≤0.18, 0.08≤a2≤0.11, 0.25≤a3≤0.28, 0.21≤a4≤0.26, 0.11≤a5≤0.17, and 0.80≤a1+a2+a3+a4+a5≤1.

0.

7. The video post-production method using virtual video technology according to claim 1, characterized in that: The S4 includes: S41, camera projection: using the first optimization strategy, the second optimization strategy and the third optimization strategy, applying the camera parameter coefficient Csx, the camera influence coefficient Yxx and the ambient shadow coefficient Hyx to the camera projection model; converting the two-dimensional image coordinates in the actual scene into three-dimensional world coordinates to generate a three-dimensional coordinate point cloud; S42, generating a reference 3D scene model: combining the 3D coordinate point cloud obtained by camera projection into a reference 3D scene model; constructing a basic virtual scene according to the structure and texture information of the 3D scene model; S43, real-time rendering: using the optimized parameters and the benchmark 3D scene model, based on the meta-video material, applying the real-time rendering engine to generate images; using the graphics processing unit GPU hardware acceleration technology, real-time calculation of lighting, shadow, texture mapping visual effects to obtain a virtual scene; S44. On the basis of real-time rendering, combine each frame of the image sequence into the final virtual video, apply video editing technology, add audio, adjust clips and apply special effects to generate the final virtual video production product.

8. A video post-production system using virtual video technology, comprising a video post-production method using virtual video technology as claimed in any one of claims 1 to 7, characterized in that: It includes original video material acquisition module, camera parameter data acquisition module, impact data acquisition module, environmental data acquisition module, data analysis and calculation module, calibration and optimization module, real-time rendering module and video editing and synthesis module; The original video material acquisition module is responsible for acquiring original video materials; The camera parameter data acquisition module is used to collect internal and external parameter data of the camera and establish a camera parameter data set; The impact data collection module is responsible for collecting vibration data and movement speed data during the operation of the camera and establishing an impact data set; The environmental data acquisition module is used to collect data including light intensity, shadow data and refraction data in the actual shooting, and establish an environmental data set; The data analysis and calculation module is used to analyze and calculate the camera parameter data set to obtain the camera parameter coefficient Csx; analyze and calculate the influence data set to obtain the camera influence coefficient Yxx; Analyze and calculate the environmental data set to obtain the environmental shadow coefficient Hyx; The calibration and optimization module is used to compare the camera parameter coefficient Csx with the first threshold value in the meta-video material using the camera calibration algorithm to obtain a first calibration result; and compare the camera influence coefficient Yxx with the second threshold value to obtain a second calibration result; The ambient shadow coefficient Hyx is compared with the third threshold value to obtain a third calibration result; the first calibration result, the second calibration result and the third calibration result are analyzed to determine the calibration accuracy and parameter eligibility, and generate a corresponding optimization strategy, including adjusting the camera focal length, correcting the position and attitude parameters; The real-time rendering module is used to apply the optimized parameters to the camera projection model, generate a three-dimensional coordinate point cloud, combine it into a benchmark three-dimensional scene model, and apply the real-time rendering engine to generate images, including lighting, shadow and texture mapping effects; The video editing and synthesis module is used to combine each frame of the image sequence into a final virtual video, add audio, perform editing and apply special effects to generate a final virtual video production product.

Images