A 3D video rendering method and system based on template dynamic replacement

Abstract

The present application relates to the technical field of multimedia, and particularly relates to a 3D video rendering method and system based on template dynamic replacement, in the present application, joints in a 3D model are bound with corresponding sites in a skeleton and rendering is performed, video information after rendering is obtained, the motion trajectory of the corresponding sites of the 3D model in the video information is determined, the obtained motion trajectory is compared with an expected motion trajectory to determine a matching degree, features are marked according to the matching degree, and then whether the generated 3D video is qualified is analyzed according to the proportion of the number of each type of marked feature, and the qualified 3D video is output, different models are applied again to realize batch production of the video, when it is determined that the generated 3D video is unqualified, secondary determination is performed based on the non-coinciding trajectories in the secondary features, so that the control accuracy of the video generation process is improved, the control accuracy of the rendering process is improved, and the rendering efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multimedia, in particular to a 3D video rendering method and system based on template dynamic replacement. BACKGROUND

[0002] Template dynamic replacement in 3D video rendering is a technology for quickly updating content based on a preset template, which realizes efficient rendering and personalized output by dynamically replacing specified elements (such as models, materials, and animation data) in the template. The existing technology uses Unity Editor to build a 3D scene, uses a behavior tree editor to edit a behavior tree based on the performance content and interactive content required by each object in the 3D scene, inputs the behavior tree data into a pre-rendering host and disassembles each complete piece of behavior tree data under the condition node to perform 3D real-time rendering. However, the control accuracy of the motion trajectory of the model after material replacement is low, and rework due to unqualified motion trajectory of the model will result in low rendering efficiency.

[0003] Chinese Patent Application No. CN202111182089.4 discloses a method and terminal for interactive video production and playback based on 3D rendering. Unity Editor is used to build a 3D scene. A behavior tree editor is used to edit a behavior tree based on the performance content and interactive content required by each object in the 3D scene, and output corresponding behavior tree data. The interactive content is a condition node of the behavior tree. The behavior tree data is input into a pre-rendering host and disassembled into each complete piece of behavior tree data under the condition node to perform 3D real-time rendering and video recording, obtaining a plurality of video files. The behavior tree data converted into a video file is simplified into a "play video" behavior, obtaining simplified behavior tree data. The simplified behavior tree data and video file are input into an interactive video player. The user triggers the condition node to execute the "play video" behavior and play the corresponding video file. The overall production and playback of the interactive video realizes interactivity, 3D real-time rendering, high concurrency, and low cost.

[0004] However, the existing technology still has the following problems:

[0005] The control accuracy of the motion trajectory of the model after material replacement is low, and rework due to unqualified motion trajectory of the model will result in low rendering efficiency. SUMMARY

[0006] Therefore, the present application provides a 3D video rendering method and system based on template dynamic replacement to overcome the problem of low control accuracy of the motion trajectory of the model after material replacement in the prior art, and rework due to unqualified motion trajectory of the model will result in low rendering efficiency.

[0007] To achieve the above object, the application provides a 3D video rendering method based on template dynamic replacement.

[0008] Step S1, extract materials to construct a template for generating a 3D video, obtain a 3D model to be rendered, bind joints in the 3D model with corresponding sites in a skeleton and perform rendering, and use a camera to collect video information after rendering is completed;

[0009] Step S2, obtain the motion trajectory of each corresponding site in the 3D model in the video information, compare the obtained motion trajectory with an expected motion trajectory to determine a matching degree, analyze whether the rendering processing for the 3D video is qualified based on the matching degree, and mark the features corresponding to the motion trajectory;

[0010] Step S3, count the number of each type of marked feature, analyze whether the generated 3D video is qualified based on the proportion of the number of each type of feature, and when it is preliminarily determined that the generated 3D video is unqualified, perform secondary determination on whether the generated 3D video is qualified based on the proportion of non-coincidence trajectories of secondary features, or analyze the reason why the generated 3D video is unqualified based on the distribution of each tertiary feature.

[0011] Further, in step S2, the obtained motion trajectory is compared with the expected motion trajectory to determine a matching degree, and whether the rendering processing for the 3D video is qualified is analyzed based on the matching degree, which includes:

[0012] the frames of the obtained motion trajectory and the frames of the expected motion trajectory are determined respectively,

[0013] the number of frames in which the obtained motion trajectory coincides with the expected motion trajectory is counted,

[0014] a ratio of the number of coinciding frames to the total number of frames of the expected motion trajectory is calculated to obtain the matching degree,

[0015] if the matching degree is greater than or equal to a first preset matching degree, it is determined that the rendering processing for the 3D video is qualified, and the features corresponding to the motion trajectory are marked as primary features;

[0016] if the matching degree is less than the first preset matching degree and greater than or equal to a second preset matching degree, it is preliminarily determined that the rendering processing for the 3D video is unqualified, and whether the rendering processing for the 3D video is qualified is determined again based on the number of frames in which the motion trajectory does not coincide with the expected motion trajectory;

[0017] if the matching degree is less than the second preset matching degree, it is determined that the rendering processing for the 3D video is unqualified, and the features corresponding to the motion trajectory are marked as tertiary features.

[0018] Further, the number of frames in which the motion trajectory does not coincide with the expected motion trajectory is calculated.

[0019] The number of frames in which the motion trajectory does not coincide with the expected motion trajectory is calculated.

[0020] The ratio of the number of frames in which the motion trajectory does not coincide to the total number of frames of the expected motion trajectory is calculated to obtain a non-coincidence trajectory proportion.

[0021] If the non-coincidence trajectory proportion is less than or equal to a preset non-coincidence trajectory proportion, it is determined that the motion trajectory is stuck, and the feature corresponding to the motion trajectory is recorded as a secondary feature.

[0022] If the non-coincidence trajectory proportion is greater than the preset non-coincidence trajectory proportion, it is determined that the rendering process for the 3D video is not qualified, and the feature corresponding to the motion trajectory is recorded as a tertiary feature.

[0023] Further, in step S3, whether the generated 3D video is qualified is analyzed based on the proportion of the number of each type of feature, comprising:

[0024] The proportion of the number of each type of feature to the total number of features is calculated to obtain the proportion of the number of each type of feature.

[0025] If the proportion of the number of tertiary features is less than or equal to a preset proportion of the number of tertiary features and the proportion of the number of primary features is greater than or equal to a preset proportion of the number of primary features, it is determined that the generated 3D video is qualified, and the 3D video is output.

[0026] If the proportion of the number of tertiary features is less than or equal to the preset proportion of the number of tertiary features and the proportion of the number of primary features is less than the preset proportion of the number of primary features, it is preliminarily determined that the generated 3D video is not qualified, and whether the generated 3D video is qualified is determined again based on the non-coincidence trajectory proportion of the secondary feature.

[0027] If the proportion of the number of tertiary features is greater than the preset proportion of the number of tertiary features, it is determined that the generated 3D video is not qualified, and the reason why the generated 3D video is not qualified is analyzed based on the distribution of each tertiary feature.

[0028] Further, the secondary determination of whether the generated 3D video is qualified based on the non-coincidence trajectory proportion of the secondary feature comprises:

[0029] The variance of the non-coincidence trajectory proportions of each secondary feature is calculated.

[0030] If the variance is less than or equal to a preset variance, it is determined that the rendering ratio is not qualified.

[0031] If the variance is greater than the preset variance, it is determined that the generated 3D video is unqualified, and the reason why the generated 3D video is unqualified is analyzed based on the distribution of each tertiary feature.

[0032] Further, under the condition that the rendering multiple is determined to be unqualified, the rendering multiple is adjusted based on the proportion of the number of secondary features,

[0033] Wherein, the reduction range of the rendering multiple is positively correlated with the proportion of the number of secondary features.

[0034] Further, the reason why the generated 3D video is unqualified analyzed based on the distribution of each tertiary feature includes:

[0035] Determine the key frame in the generated 3D video, calculate the average distribution distance of the coordinates of each tertiary feature in the picture corresponding to the key frame,

[0036] If the average distribution distance is less than or equal to the preset average distribution distance, it is determined that the reason why the generated 3D video is unqualified is that the model binding is unqualified, and a re-binding notification is issued;

[0037] If the average distribution distance is greater than the preset average distribution distance, it is determined that the reason why the generated 3D video is unqualified is that the model size and the skeleton do not match, and the scaling multiple of the model is re-determined.

[0038] Further, the scaling multiple of the model is re-determined, including:

[0039] Calculate the average value of the matching degree of each tertiary feature, wherein the adjustment range of the scaling multiple is positively correlated with the average value.

[0040] The present application provides a kind of 3D video rendering system based on template dynamic replacement. Including:

[0041] Template storage unit, to store material, material includes: scene model, illumination site, illumination brightness, camera site, joint skeleton, model action;

[0042] Template extraction unit, which is connected with the template storage unit, is used to extract material from the template storage unit to construct the template of 3D video;

[0043] Model generation unit, which is connected with the template extraction unit, is used to determine the pose of 3D model by binding the corresponding joint in 3D model with the corresponding site of skeleton;

[0044] Rendering unit, which is connected with the model generation unit, is used for rendering processing of scene;

[0045] Video generation unit, which is connected with the rendering unit, is used to play template to obtain video information through camera;

[0046] a feature extraction unit connected with the video generation unit, configured to collect motion trajectories of features in the model in the video information;

[0047] a matching unit connected with the feature extraction unit, configured to compare the collected motion trajectories with corresponding expected trajectories;

[0048] an analysis unit connected with the model generation unit, the rendering unit, the video generation unit, the feature extraction unit and the matching unit respectively, configured to analyze whether the generated 3D video is qualified based on the comparison result of the matching unit, and when it is preliminarily determined that the generated 3D video is unqualified, make a secondary determination on whether the generated 3D video is qualified based on the proportion of non-coincidence trajectories of the secondary features, or determine the reason why the generated 3D video is unqualified based on the distribution of each tertiary feature;

[0049] a video output unit connected with the analysis unit, configured to output the 3D video when the analysis unit determines that the generated 3D video is qualified.

[0050] Compared with the prior art, the beneficial effects of the present application are that, in the present application, the joints in the 3D model are bound with the corresponding positions in the skeleton and rendered, the video information after rendering is obtained, the motion trajectories of the corresponding positions of the 3D model in the video information are determined, the obtained motion trajectories are compared with the expected motion trajectories to determine the matching degree, the features are marked according to the matching degree, and then whether the generated 3D video is qualified is analyzed according to the proportion of the number of each type of marked features, and the qualified 3D video is output, different models are applied again to realize batch production of videos, and the actual application purpose is to find the most suitable material by continuous testing, when it is determined that the generated 3D video is unqualified, a secondary determination is made based on the non-coincidence trajectories in the secondary features, so as to improve the control accuracy of the video generation process, improve the control accuracy of the rendering process, and further improve the rendering efficiency.

[0051] Further, in the present application, the obtained motion trajectories are compared with the expected motion trajectories, the matching degree is calculated according to the coincidence, whether the rendering processing of the 3D video is qualified is analyzed according to the matching degree, and the features of the video are marked, when the matching degree is between the first preset matching degree and the second preset matching degree, whether the rendering processing of the 3D video is qualified is determined again according to the number of frames of the non-coincidence part of the motion trajectories and the expected motion trajectories, so as to further improve the control accuracy of the rendering processing and improve the analysis accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 a flowchart of the 3D video rendering method based on template dynamic replacement of the present application;

[0053] Figure 2 Flow chart of the 3D video rendering system based on template dynamic replacement of the present application;

[0054] Figure 3 Flow chart of the judgment of whether the rendering process for 3D video is qualified or not;

[0055] Figure 4 Flow chart of the judgment of whether the rendering process for 3D video is qualified or not. DETAILED DESCRIPTION

[0056] In order to make the objects and advantages of the present application clearer, the present application will be further described in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0057] It should be noted that the data in the present embodiment are obtained by analyzing and evaluating the historical data and corresponding historical judgment results of the past 6 months before the present judgment by the system of the present application. Those skilled in the art can understand that the determination method of the system of the present application for a single parameter can be to select the value with the highest proportion as the preset standard parameter according to the data distribution, to use weighted summation to obtain the value as the preset standard parameter, to substitute the historical data into a specific formula and to obtain the value by the formula as the preset standard parameter, or other selection methods, as long as the system of the present application can clearly define different specific situations in the single judgment process by the obtained value.

[0058] The preferred embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not used to limit the protection scope of the present application.

[0059] It should be noted that in the description of the present application, the terms of direction or position relationship such as "up", "down", "left", "right", "inner", "outer" and the like are based on the direction or position relationship shown in the drawings, which is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application.

[0060] In addition, it should also be noted that in the description of the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connection" and "connection" should be understood in a broad sense, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be direct connection, or indirect connection through an intermediate medium, or internal communication of two elements. Those skilled in the art can understand the specific meaning of the above terms in the present application according to the specific circumstances.

[0061] Referring to Figure 1 As shown in the figure, it is a flow chart of the 3D video rendering method based on template dynamic replacement of the present application.

[0062] The 3D video rendering method based on template dynamic replacement provided by the embodiment comprises:

[0063] Step S1, extract the materials to construct the template for generating the 3D video, acquire the 3D model to be rendered, bind the joints in the 3D model with the corresponding sites in the skeleton and perform rendering, and adopt the camera to collect the video information after rendering;

[0064] Step S2, acquire the motion trajectory of each corresponding site in the 3D model in the video information, compare the acquired motion trajectory with the expected motion trajectory to determine the matching degree, analyze whether the rendering processing for the 3D video is qualified based on the matching degree, and mark the features corresponding to the motion trajectory;

[0065] Step S3, count the number of each type of marked features, analyze whether the generated 3D video is qualified based on the proportion of the number of each type of features, and when it is preliminarily determined that the generated 3D video is unqualified, perform secondary determination on whether the generated 3D video is qualified based on the proportion of the non-coincidence trajectory of the secondary features, or analyze the reason why the generated 3D video is unqualified based on the distribution of each tertiary feature.

[0066] In the present application, the joints in the 3D model are bound with the corresponding sites in the skeleton and rendered to acquire the video information after rendering, the motion trajectory of the corresponding sites of the 3D model in the video information is determined, the acquired motion trajectory is compared with the expected motion trajectory to determine the matching degree, the features are marked according to the matching degree, and then whether the generated 3D video is qualified is analyzed according to the proportion of the number of each type of marked features, and the qualified 3D video is output, different models are applied again to realize batch production of the video, and the actual application purpose is to find the most suitable materials by continuous testing, when it is determined that the generated 3D video is unqualified, secondary determination is performed based on the non-coincidence trajectory in the secondary features, so as to improve the control accuracy of the video generation process, improve the control accuracy of the rendering process, and further improve the rendering efficiency.

[0067] Referring to Figure 2 As shown in the figure, it is a flow chart of the 3D video rendering system based on template dynamic replacement of the present application.

[0068] The 3D video rendering system based on template dynamic replacement provided by the embodiment comprises:

[0069] The template storage unit is used to store the materials, and the materials comprise: scene model, light site, light intensity, camera site, joint skeleton, and model action.

[0070] a template extraction unit connected to the template storage unit, configured to extract the template from the template storage unit to construct a 3D video;

[0071] a model generation unit connected to the template extraction unit, configured to determine a pose of the 3D model by binding corresponding joints in the 3D model to corresponding positions of the skeleton;

[0072] a rendering unit connected to the model generation unit, configured to perform rendering processing on the scene;

[0073] a video generation unit connected to the rendering unit, configured to play the template to obtain video information through a camera;

[0074] a feature extraction unit connected to the video generation unit, configured to collect a motion trajectory of a feature in the model in the video information;

[0075] a matching unit connected to the feature extraction unit, configured to compare the obtained motion trajectory with a corresponding expected trajectory;

[0076] an analysis unit connected to the model generation unit, the rendering unit, the video generation unit, the feature extraction unit and the matching unit respectively, configured to analyze whether the generated 3D video is qualified based on a comparison result of the matching unit, and when it is preliminarily determined that the generated 3D video is not qualified, perform secondary determination on whether the generated 3D video is qualified based on a proportion of non-coincidence trajectories of secondary features, or analyze a reason why the generated 3D video is not qualified based on a distribution of each tertiary feature;

[0077] a video output unit connected to the analysis unit, configured to output the 3D video when the analysis unit determines that the generated 3D video is qualified.

[0078] Specifically, the specific structures of the template storage unit, the template extraction unit, the model generation unit, the rendering unit, the video generation unit, the feature extraction unit, the matching unit, the analysis unit and the video output unit are not limited, and they can be composed of logical components including a field programmable processor, a computer and a microprocessor in the computer.

[0079] Please refer to Figure 3 shown, which is a determination flowchart for analyzing whether rendering processing for a 3D video is qualified.

[0080] Specifically, in the step S2, the obtained motion trajectory is compared with an expected motion trajectory to determine a matching degree, and whether the rendering processing for the 3D video is qualified is analyzed based on the matching degree, including:

[0081] determine the frames of the acquired motion trajectory and the expected motion trajectory respectively,

[0082] count the number of frames in which the acquired motion trajectory coincides with the expected motion trajectory,

[0083] calculate the ratio of the number of coinciding frames to the total number of frames of the expected motion trajectory to obtain a matching degree,

[0084] if the matching degree is greater than or equal to a first preset matching degree, it is determined that the rendering processing for the 3D video is qualified, and the feature corresponding to the motion trajectory is recorded as a first-level feature;

[0085] if the matching degree is less than the first preset matching degree and greater than or equal to a second preset matching degree, it is preliminarily determined that the rendering processing for the 3D video is unqualified, and whether the rendering processing for the 3D video is qualified is determined again based on the number of frames in which the motion trajectory does not coincide with the expected motion trajectory;

[0086] if the matching degree is less than the second preset matching degree, it is determined that the rendering processing for the 3D video is unqualified, and the feature corresponding to the motion trajectory is recorded as a third-level feature.

[0087] Specifically, in the embodiment, the first preset matching degree is selected in the interval [0.88, 0.95], and the second preset matching degree is selected in the interval [0.8, 0.85].

[0088] In the present application, the acquired motion trajectory is compared with the expected motion trajectory, the matching degree is calculated according to the coincidence, whether the rendering processing for the 3D video is qualified is analyzed according to the matching degree, and the features of the video are marked. When the matching degree is between the first preset matching degree and the second preset matching degree, whether the rendering processing for the 3D video is qualified is determined again according to the number of frames in which the motion trajectory does not coincide with the expected motion trajectory, so as to further improve the control accuracy of the rendering processing and improve the analysis accuracy.

[0089] Please refer to Figure 4 , which is a determination flow chart of determining whether the rendering processing for the 3D video is qualified again.

[0090] Specifically, the determination whether the rendering processing for the 3D video is qualified again based on the number of frames in which the motion trajectory does not coincide with the expected motion trajectory comprises:

[0091] calculate the number of frames in which the acquired motion trajectory does not coincide with the expected motion trajectory,

[0092] calculate the ratio of the number of non-coinciding frames to the total number of frames of the expected motion trajectory to obtain a non-coinciding trajectory proportion,

[0093] If the proportion of non-coincidence trajectories is less than or equal to the preset proportion of non-coincidence trajectories, it is determined that the motion trajectory appears to be stuck, and the feature corresponding to the motion trajectory is recorded as a secondary feature.

[0094] If the proportion of non-coincidence trajectories is greater than the preset proportion of non-coincidence trajectories, it is determined that the rendering process for the 3D video is unqualified, and the feature corresponding to the motion trajectory is recorded as a tertiary feature.

[0095] Specifically, in the embodiment, the preset proportion of non-coincidence trajectories is selected between the interval [0.1, 0.15].

[0096] Specifically, in the step S3, whether the generated 3D video is qualified is analyzed based on the proportion of the number of each type of feature, including:

[0097] The proportion of the number of each type of feature to the total number of features is calculated respectively to obtain the proportion of the number of each type of feature,

[0098] If the proportion of the number of tertiary features is less than or equal to the preset proportion of the number of tertiary features and the proportion of the number of primary features is greater than or equal to the preset proportion of the number of primary features, it is determined that the generated 3D video is qualified, and the 3D video is output;

[0099] If the proportion of the number of tertiary features is less than or equal to the preset proportion of the number of tertiary features and the proportion of the number of primary features is less than the preset proportion of the number of primary features, it is preliminarily determined that the generated 3D video is unqualified, and whether the generated 3D video is qualified is determined again based on the proportion of non-coincidence trajectories of the secondary feature;

[0100] If the proportion of the number of tertiary features is greater than the preset proportion of the number of tertiary features, it is determined that the generated 3D video is unqualified, and the reason why the generated 3D video is unqualified is analyzed based on the distribution of each tertiary feature.

[0101] Specifically, in the embodiment, the preset proportion of the number of tertiary features is 10%, and the preset proportion of the number of primary features is 90%.

[0102] Specifically, the secondary determination of whether the generated 3D video is qualified based on the proportion of non-coincidence trajectories of the secondary feature includes:

[0103] The variance of the proportion of non-coincidence trajectories of each secondary feature is calculated,

[0104] If the variance is less than or equal to the preset variance, it is determined that the rendering ratio is unqualified;

[0105] If the variance is greater than the preset variance, it is determined that the generated 3D video is unqualified, and the reason why the generated 3D video is unqualified is analyzed based on the distribution of each tertiary feature.

[0106] Specifically, in the embodiment, the preset variance is obtained by pre-measurement, the generation data of a plurality of qualified 3D videos are acquired, the variance of the proportion of non-coincidence trajectories of each secondary feature is calculated, the mean of the variances is solved, and the preset variance is obtained.

[0107] Specifically, under the condition that the rendering multiple is determined to be unqualified, the rendering multiple is adjusted based on the proportion of the number of secondary features,

[0108] The reduction range of the rendering multiple is positively correlated with the proportion of the number of secondary features.

[0109] In the embodiment, optionally,

[0110] The proportion of the number of secondary features is compared with a first preset proportion of the number of secondary features and a second preset proportion of the number of secondary features,

[0111] If the proportion of the number of secondary features is less than or equal to the first preset proportion of the number of secondary features, the rendering multiple is reduced to 0.95 times of the initial value;

[0112] If the proportion of the number of secondary features is greater than the first preset proportion of the number of secondary features and less than or equal to the second preset proportion of the number of secondary features, the rendering multiple is reduced to 0.9 times of the initial value;

[0113] If the proportion of the number of secondary features is greater than the second preset proportion of the number of secondary features, the rendering multiple is reduced to 0.8 times of the initial value.

[0114] The first preset proportion of the number of secondary features is 20%, and the second preset proportion of the number of secondary features is 30%.

[0115] Specifically, the reason why the generated 3D video is unqualified based on the distribution analysis of each tertiary feature includes:

[0116] The key frame in the generated 3D video is determined, and the average distribution distance of the coordinates of each tertiary feature in the picture corresponding to the key frame is calculated,

[0117] If the average distribution distance is less than or equal to a preset average distribution distance, it is determined that the reason why the generated 3D video is unqualified is that the model binding is unqualified, and a re-binding notification is issued;

[0118] If the average distribution distance is greater than the preset average distribution distance, it is determined that the reason why the generated 3D video is unqualified is that the model size and the skeleton do not match, and the scaling multiple of the model is re-determined.

[0119] Specifically, in the embodiment, the preset average distribution distance is obtained by pre-measurement, the generation data of a plurality of qualified 3D videos are acquired, the average distribution distance of the coordinates of each tertiary feature in the picture corresponding to the key frame in the 3D video is calculated, the average of each average distribution distance is solved, and the preset average distribution distance is obtained.

[0120] Specifically, the scaling factor of the re-determined model is determined, including:

[0121] The average value of the matching degree of each tertiary feature is calculated, and the adjustment range of the scaling factor is positively correlated with the average value.

[0122] Specifically, in the embodiment, the magnification or reduction factor is determined according to the binding condition of the model and the skeleton, the distance between the motion trajectory constructed by the tertiary feature and the geometric center of the model and the expected motion trajectory is calculated, if the distance between the expected motion trajectory and the geometric center of the model is greater than the distance between the motion trajectory and the geometric center of the model, the model is magnified, that is, the magnification scaling factor is calculated, if the distance between the expected motion trajectory and the geometric center of the model is less than the distance between the motion trajectory and the geometric center of the model, the scaling factor is reduced.

[0123] So far, the technical solutions of the present application have been described in combination with the preferred embodiments shown in the drawings, but those skilled in the art can easily understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical solutions after the changes or replacements will fall within the protection scope of the present application.

[0124] The above description is only the preferred embodiments of the present application and is not used to limit the present application; for those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A 3D video rendering method based on template dynamic replacement, characterized in that, The method comprises the following steps: Step S1, extracting materials to construct a template for generating a 3D video, obtaining a 3D model to be rendered, binding joints in the 3D model with corresponding positions in a skeleton and rendering, and collecting video information after rendering by using a camera; Step S2, obtaining a motion trajectory of each corresponding position in the 3D model in the video information, comparing the obtained motion trajectory with an expected motion trajectory to determine a matching degree, analyzing whether the rendering processing for the 3D video is qualified based on the matching degree, and marking a feature corresponding to the motion trajectory; Step S3, counting the number of each type of marked feature, analyzing whether the generated 3D video is qualified based on the proportion of the number of each type of feature, and when it is preliminarily determined that the generated 3D video is unqualified, performing secondary determination on whether the generated 3D video is qualified based on the proportion of non-coincidence trajectories of secondary features, or analyzing the reason why the generated 3D video is unqualified based on the distribution of each tertiary feature; In step S2, the obtained motion trajectory is compared with the expected motion trajectory to determine the matching degree, and whether the rendering processing for the 3D video is qualified is analyzed based on the matching degree, which comprises the following steps: The frames of the obtained motion trajectory and the expected motion trajectory are determined respectively, The number of frames in which the obtained motion trajectory coincides with the expected motion trajectory is counted, The ratio of the number of coinciding frames to the total number of frames of the expected motion trajectory is calculated to obtain the matching degree, If the matching degree is greater than or equal to a first preset matching degree, it is determined that the rendering processing for the 3D video is qualified, and the feature corresponding to the motion trajectory is marked as a primary feature; If the matching degree is less than the first preset matching degree and greater than or equal to a second preset matching degree, it is preliminarily determined that the rendering processing for the 3D video is unqualified, and whether the rendering processing for the 3D video is qualified is determined again based on the number of frames in which the motion trajectory does not coincide with the expected motion trajectory; If the matching degree is less than the second preset matching degree, it is determined that the rendering processing for the 3D video is unqualified, and the feature corresponding to the motion trajectory is marked as a tertiary feature; In the secondary determination of whether the rendering processing for the 3D video is qualified based on the number of frames in which the motion trajectory does not coincide with the expected motion trajectory, the following steps are included: The number of frames in which the obtained motion trajectory does not coincide with the expected motion trajectory is calculated, The ratio of the number of non-coincidence frames to the total number of frames of the expected motion trajectory is calculated to obtain a non-coincidence trajectory proportion, If the non-coincidence trajectory proportion is less than or equal to a preset non-coincidence trajectory proportion, it is determined that the motion trajectory appears to be stuck, and the feature corresponding to the motion trajectory is marked as a secondary feature; If the non-coincidence trajectory proportion is greater than the preset non-coincidence trajectory proportion, it is determined that the rendering processing for the 3D video is unqualified, and the feature corresponding to the motion trajectory is marked as a tertiary feature.

2. The 3D video rendering method based on template dynamic replacement according to claim 1, characterized in that, In step S3, whether the generated 3D video is qualified is analyzed based on the proportion of the number of each type of feature, which comprises the following steps: The proportion of the number of each type of feature to the total number of features is calculated to obtain the proportion of the number of each type of feature, If the proportion of the number of the third-level features is less than or equal to the preset proportion of the number of the third-level features and the proportion of the number of the first-level features is greater than or equal to the preset proportion of the number of the first-level features, it is determined that the generated 3D video is qualified, and the 3D video is outputted; If the proportion of the number of the third-level features is less than or equal to the preset proportion of the number of the third-level features and the proportion of the number of the first-level features is less than the preset proportion of the number of the first-level features, it is preliminarily determined that the generated 3D video is unqualified, and whether the generated 3D video is qualified is determined again based on the proportion of the non-overlapping trajectories of the second-level features; If the proportion of the number of the third-level features is greater than the preset proportion of the number of the third-level features, it is determined that the generated 3D video is unqualified, and the reason why the generated 3D video is unqualified is analyzed based on the distribution of each third-level feature.

3. The 3D video rendering method based on template dynamic replacement according to claim 2, characterized in that, The determining again whether the generated 3D video is qualified based on the proportion of the non-overlapping trajectories of the second-level features comprises: calculating the variance of the proportion of the non-overlapping trajectories of each second-level feature, if the variance is less than or equal to a preset variance, it is determined that the rendering magnification is unqualified; if the variance is greater than the preset variance, it is determined that the generated 3D video is unqualified, and the reason why the generated 3D video is unqualified is analyzed based on the distribution of each third-level feature.

4. The 3D video rendering method based on template dynamic replacement according to claim 3, characterized in that, Under the condition that the rendering magnification is unqualified, the rendering magnification is adjusted based on the proportion of the number of the second-level features, wherein the decreasing range of the rendering magnification is positively correlated with the proportion of the number of the second-level features.

5. The 3D video rendering method based on template dynamic replacement according to claim 4, characterized in that, The analyzing the reason why the generated 3D video is unqualified based on the distribution of each third-level feature comprises: determining the key frame in the generated 3D video, calculating the average distribution distance of the coordinates of each third-level feature in the picture corresponding to the key frame, if the average distribution distance is less than or equal to a preset average distribution distance, it is determined that the reason why the generated 3D video is unqualified is that the model binding is unqualified, and a re-binding notification is sent; if the average distribution distance is greater than the preset average distribution distance, it is determined that the reason why the generated 3D video is unqualified is that the model size and the skeleton are not matched, and the scaling magnification of the model is re-determined.

6. The 3D video rendering method based on template dynamic replacement according to claim 5, characterized in that, The re-determining the scaling magnification of the model comprises: calculating the average value of the matching degrees of each third-level feature, wherein the adjusting range of the scaling magnification is positively correlated with the average value.

7. A 3D video rendering system based on template dynamic replacement implementing the method of any one of claims 1 to 6, characterized in that, It comprises: a template storage unit for storing materials, the materials including scene models, light points, light intensities, camera points, joint skeletons, and model actions; a template extraction unit connected with the template storage unit, for extracting materials from the template storage unit to construct a template of a 3D video; a model generation unit connected with the template extraction unit, for determining the posture of a 3D model by binding corresponding joints in the 3D model to corresponding points of the skeleton; a rendering unit connected with the model generation unit, for rendering the scene; a video generation unit connected with the rendering unit, for playing the template to obtain video information through a camera; a feature extraction unit connected with the video generation unit, for collecting the motion trajectories of the features in the model in the video information; a matching unit connected with the feature extraction unit, for comparing the obtained motion trajectories with corresponding expected trajectories; an analysis unit connected with the model generation unit, the rendering unit, the video generation unit, the feature extraction unit and the matching unit respectively, configured to analyze whether the generated 3D video is qualified based on the comparison result of the matching unit, and when it is preliminarily determined that the generated 3D video is unqualified, make a secondary determination on whether the generated 3D video is qualified based on the proportion of the non-coincidence track of the secondary features, or determine the reason why the generated 3D video is unqualified based on the distribution of each tertiary feature; a video output unit connected with the analysis unit, configured to output the 3D video when the analysis unit determines that the generated 3D video is qualified.

Images