A real-time rendering and audio synchronization system for animation scenes

Abstract

The application belongs to the technical field of audio synchronization, and provides an animation scene real-time rendering and audio synchronization system, which comprises the following steps: extracting real-time video and audio from an animation scene video stream, analyzing the synchronization matching degree of the real-time video and audio, adding time stamps to the video and various types of audio, distinguishing between synchronous and asynchronous time stamps, and evaluating whether the current animation scene video and the current type of audio are synchronized; if not, performing deviation analysis on the asynchronous audio to determine the degree of asynchrony between the current video and all types of corresponding audio; if the degree of asynchrony is serious, adjusting the frame rate of the audio to prioritize adjusting the video with a serious degree of asynchrony, adopting multi-frame smooth transition, and determining the audio frame rate of the adjusted audio frame in the asynchronous audio; and the application determines the basic adjustment amount and the final adjustment amount of the audio frame rate, and combines the multi-frame smooth transition technology to effectively adjust the frame rate of the asynchronous audio.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of audio synchronization, and in particular relates to a real-time rendering and audio synchronization system for an animation scene. BACKGROUND

[0002] In the production of an animation and a real-time interaction scene, real-time rendering technology can quickly generate dynamic visual pictures, and audio can add atmosphere and emotion to the scene.

[0003] However, there are many problems and defects in the prior art. On the one hand, a single type of audio is used for adjustment, and the asynchronous coverage range and deviation degree of different types of audio are not quantified, resulting in unreasonable optimization of resource allocation. On the other hand, real-time rendering and audio of an animation scene often appear out of sync, which seriously affects the immersive experience of users and reduces the quality and watchability of works.

[0004] In view of the above problems, in order to improve the quality of real-time rendering of an animation scene and the user experience, the application provides a real-time rendering and audio synchronization system for an animation scene. SUMMARY

[0005] In order to make up for the deficiencies of the prior art and solve at least one technical problem proposed in the background art.

[0006] The technical scheme adopted by the application to solve the technical problems is: a real-time rendering and audio synchronization system for an animation scene, specifically comprising the following modules:

[0007] A data acquisition module: acquires an animation scene video stream, extracts real-time video and audio from the animation scene video stream, analyzes the synchronization matching degree of the real-time video and audio, adds timestamps to the video and each type of audio, distinguishes between synchronous and asynchronous timestamps, determines the asynchronous timestamp quantity proportion and asynchronous timestamp offset degree ratio;

[0008] A matching degree analysis module: according to the asynchronous timestamp quantity proportion and asynchronous timestamp offset degree ratio, it is determined whether the current animation scene video and the current type of audio are synchronous;

[0009] An asynchronous degree analysis module: if not, the asynchronous degree of the current video and all types of corresponding audio is determined by analyzing the deviation of the asynchronous audio;

[0010] An audio frame rate adjustment module: if the asynchronous degree is serious, the frame rate of the audio is adjusted to preferentially adjust the video with a serious asynchronous degree, the final adjustment amount of the frame rate of the asynchronous audio in the video with a serious asynchronous degree is determined, multi-frame smooth transition is adopted, and the audio frame rate of the adjusted audio frame in the asynchronous audio is determined.

[0011] As a further scheme of the present application, the asynchronous timestamp acquisition process is:

[0012] Extract all audio data corresponding to any video segment, wherein the audio data comprises audio types, add timestamps to the video and all types of audio;

[0013] Integrate the timestamps added by any type of audio into an audio timestamp sequence; extract all timestamps corresponding to the current video and integrate them into a video timestamp sequence;

[0014] According to the audio timestamp sequence and the video timestamp sequence, calculate the time difference between the audio timestamp and the video timestamp;

[0015] If the time difference is greater than or equal to the time difference standard value, the corresponding timestamp is recorded as an asynchronous timestamp.

[0016] As a further scheme of the present application, the specific process of evaluating whether the current animation scene video and the current type of audio are synchronous is:

[0017] Extract real-time video and audio from the animation scene video stream and analyze them to obtain the asynchronous timestamp quantity ratio and the asynchronous timestamp offset degree ratio;

[0018] Multiply the asynchronous timestamp quantity ratio and the asynchronous timestamp offset degree ratio to obtain a synchronization matching value;

[0019] If the synchronization matching value is greater than or equal to the synchronization matching threshold value, it indicates that the current animation scene video and the current type of audio are not synchronous.

[0020] As a further scheme of the present application, the acquisition method of the asynchronous timestamp quantity ratio is:

[0021] Calculate the asynchronous timestamp quantity ratio by counting the number of asynchronous timestamps in all timestamps.

[0022] As a further scheme of the present application, the acquisition method of the asynchronous timestamp offset degree ratio is:

[0023] Subtract the time difference corresponding to the asynchronous timestamp from the time difference standard value to obtain the asynchronous timestamp offset, sum all asynchronous timestamp offsets corresponding to the asynchronous timestamps and take the average, and then perform ratio processing with the time difference standard value to obtain the asynchronous timestamp offset degree ratio.

[0024] As a further scheme of the present application, the specific process of judging the asynchronous degree of the current video and all types of corresponding audio is:

[0025] Record the audio that is not synchronized with the video as asynchronous audio and perform deviation analysis to obtain the asynchronous audio quantity ratio and the asynchronous deviation degree value;

[0026] The asynchronous audio quantity proportion is multiplied by the asynchronous deviation degree value to obtain an asynchronous degree value.

[0027] As a further scheme of the present application, the asynchronous audio quantity proportion is obtained in the following manner:

[0028] The number of asynchronous audios in all types of audios is counted, and the asynchronous audio quantity proportion in all types of audios is calculated.

[0029] As a further scheme of the present application, the asynchronous deviation degree value is obtained in the following manner:

[0030] The synchronous matching value corresponding to the asynchronous audio is extracted, the synchronous matching value corresponding to the asynchronous audio is subtracted from the synchronous matching threshold to obtain an asynchronous deviation value, the asynchronous deviation values corresponding to all asynchronous audios are summed and averaged, and then the asynchronous deviation degree value is obtained by ratio processing with the synchronous matching threshold.

[0031] As a further scheme of the present application, the final adjustment amount of the frame rate of the asynchronous audio is obtained in the following manner:

[0032] The video frame interval and the original audio frame rate are obtained, and the basic adjustment amount of the audio frame rate is calculated in combination with the asynchronous deviation mean value.

[0033] The final adjustment amount is calculated by introducing an experience parameter according to the basic adjustment amount of the audio frame rate.

[0034] As a further scheme of the present application, the audio frame rate of the audio frame adjusted in the asynchronous audio is obtained in the following manner:

[0035] The frame rate adjustment amount safety range is set, and the adjusted frame rate is calculated according to the final adjustment amount.

[0036] The original audio frame rate and the transition frame number of the audio frame in the asynchronous audio are obtained by using multi-frame smooth transition, and the audio frame rate of the audio frame adjusted in the asynchronous audio is calculated in combination with the adjusted frame rate.

[0037] The present application has the following beneficial effects:

[0038] 1. Obtain an animation scene video stream, extract real-time video and audio from the animation scene video stream, analyze the synchronization matching degree of the real-time video and audio, add timestamps to the video and each type of audio, distinguish between synchronous and asynchronous timestamps, determine the asynchronous timestamp quantity ratio and asynchronous timestamp offset degree ratio, and evaluate whether the current animation scene video and the current type of audio are synchronized according to the asynchronous timestamp quantity ratio and the asynchronous timestamp offset degree ratio; the present application forms a standardized synchronization matching value through the comprehensive calculation of the asynchronous timestamp quantity ratio and the offset degree ratio, which not only reflects the universality of the asynchronous problem, but also quantifies the severity of the deviation, providing a scientific basis for synchronization determination, while being conducive to quickly responding and adjusting the audio and video triggering time or buffer strategy, reducing the cost of manual debugging, improving development efficiency and user experience, and ensuring the smoothness and immersion of audio-visual synchronization in the animation scene.

[0039] 2. If not synchronized, the asynchronous degree of the current video and all types of corresponding audio is determined by analyzing the deviation of the asynchronous audio, and if the asynchronous degree is serious, the frame rate of the audio is adjusted to prioritize adjustment of the video with serious asynchronous degree, the final adjustment amount of the frame rate of the asynchronous audio in the video with serious asynchronous degree is determined, and the audio frame rate after adjustment of the audio frame is determined by using multi-frame smooth transition; the present application integrates the asynchronous audio quantity ratio and the deviation degree value through the asynchronous degree value to form a quantitative evaluation of the global asynchronous state of audio and video, accurately reflects the coverage and severity of the asynchronous problem, provides a scientific basis for subsequent optimization, and improves the real-time rendering quality of the animation scene; secondly, for scenes with serious asynchronous degree, the basic adjustment amount and the final adjustment amount of the audio frame rate are calculated, and the multi-frame smooth transition technology is combined to effectively adjust the frame rate of the asynchronous audio, avoid auditory discomfort caused by frame mutation, and ensure the smoothness of audio-visual synchronization; finally, by setting a safe range for the frame rate adjustment amount, audio distortion caused by excessive adjustment is prevented, and the robustness and user experience of the optimization strategy are improved. BRIEF DESCRIPTION OF DRAWINGS

[0040] The present application will be further described below with reference to the accompanying drawings.

[0041] Figure 1 is a system block diagram of an animation scene real-time rendering and audio synchronization system according to an embodiment of the present application;

[0042] Figure 2 is a flowchart of the acquisition step in an animation scene real-time rendering and audio synchronization method according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the technical means, creative features, purposes and effects achieved by the present application easy to understand, the present application will be further described below with reference to the specific embodiments.

[0044] Embodiment 1

[0045] Please see Figure 1 As shown in the embodiment of the present invention, a real-time rendering and audio synchronization system for an animation scene includes the following modules:

[0046] Data acquisition module: Acquire animation scene video stream, extract real-time video and audio from the animation scene video stream, analyze the synchronization matching degree of real-time video and audio, add timestamps to video and various types of audio, distinguish between synchronous and asynchronous timestamps, and determine the proportion of asynchronous timestamps and the degree of asynchronous timestamp offset.

[0047] In real-time rendering of animation scenes (such as gamified animation and virtual production), the engine triggers audio events based on the rendering timing of video frames:

[0048] When rendering a video, the engine will simultaneously trigger the corresponding audio based on the character's movements contained in the video;

[0049] Extract all audio data corresponding to any video segment. The audio data includes the audio type. Add timestamps to the video and all types of audio.

[0050] Combine the timestamps added to any type of audio into an audio timestamp sequence V = [v1, v2, ...];

[0051] Extract all timestamps corresponding to the current video and integrate them into a video timestamp sequence A = [a1, a2, ...];

[0052] It should be noted that each type of audio is associated with a specific video, and the timestamp added to the audio corresponds to the time of the video.

[0053] Based on the audio timestamp sequence and the video timestamp sequence, calculate the time difference between the audio timestamp and the video timestamp. The specific calculation formula is as follows:

[0054] Δt i =|v i -a i |

[0055] Where i = 1, 2, ..., n, n represents the total number of timestamps;

[0056] Compare the time difference with a standard time difference value:

[0057] If the time difference is greater than or equal to the standard time difference value, the corresponding timestamp will be recorded as an asynchronous timestamp.

[0058] If the time difference is less than the standard time difference value, the corresponding timestamp will be recorded as the synchronization timestamp.

[0059] It should be noted that the standard value for the time difference was set by those skilled in the art based on historical experience;

[0060] Count the number of asynchronous timestamps among all timestamps and calculate the percentage of asynchronous timestamps.

[0061] The asynchronous timestamp offset is obtained by subtracting the time difference corresponding to the asynchronous timestamp from the standard time difference value. The asynchronous timestamp offsets corresponding to all asynchronous timestamps are summed and averaged to obtain the asynchronous timestamp offset average. The asynchronous timestamp offset average is then compared with the time difference standard value to obtain the asynchronous timestamp offset degree ratio.

[0062] Matching analysis module: Based on the proportion of asynchronous timestamps and the ratio of asynchronous timestamp offset, it assesses whether the current animation scene video and the current type of audio are synchronized;

[0063] The synchronization matching value is obtained by multiplying the proportion of asynchronous timestamps by the ratio of asynchronous timestamp offset.

[0064] In some embodiments, the synchronization matching value is compared with the synchronization matching threshold. The specific comparison process is as follows:

[0065] If the synchronization matching value is greater than or equal to the synchronization matching threshold, it means that the current animation scene video and the current type of audio are out of sync.

[0066] If the synchronization matching value is less than the synchronization matching threshold, it means that the current animation scene video and the current type of audio are synchronized;

[0067] The purpose of setting the synchronization matching value is:

[0068] Function 1: By reflecting the prevalence of asynchronous problems through the proportion of asynchronous timestamps, and by reflecting the severity of asynchronous deviations through the ratio of asynchronous timestamp offsets, a comprehensive quantitative indicator is formed to determine the synchronization between the current video and a specific type of audio.

[0069] Function 2: Provides standardized decision-making basis for audio-visual synchronization optimization in real-time animation rendering, making it easier for the engine or system to respond quickly and execute subsequent adjustment strategies;

[0070] The technical solution of this embodiment is as follows: Acquire an animation scene video stream, extract real-time video and audio from the animation scene video stream, analyze the synchronization matching degree of the real-time video and audio, add timestamps to the video and various types of audio, distinguish between synchronous and asynchronous timestamps, determine the proportion of asynchronous timestamps and the ratio of asynchronous timestamp offset, and evaluate whether the current animation scene video and the current type of audio are synchronized based on the proportion of asynchronous timestamps and the ratio of asynchronous timestamp offset. This invention forms a standardized synchronization matching value through the comprehensive calculation of the proportion of asynchronous timestamps and the offset ratio. This value reflects both the universality of asynchronous problems and the severity of deviations, providing a scientific basis for synchronization determination. Simultaneously, it facilitates rapid response and adjustment of audio and video triggering timing or buffering strategies, reduces manual debugging costs, improves development efficiency and user experience, and ensures the smoothness and immersiveness of audio-visual synchronization in animation scenes.

[0071] Example 2

[0072] Please see Figure 1 As shown in the embodiment of the present invention, a real-time rendering and audio synchronization system for an animation scene includes the following modules:

[0073] Asynchronousness Analysis Module: If there is a lack of synchronization, the module performs deviation analysis on the asynchronous audio to determine the degree of asynchrony between the current video and all corresponding types of audio.

[0074] Record the corresponding audio type as asynchronous audio, count the number of asynchronous audio in all types of audio, and calculate the percentage of asynchronous audio in all types of audio;

[0075] Extract the synchronization matching value corresponding to the asynchronous audio, subtract the synchronization matching value corresponding to the asynchronous audio from the synchronization matching threshold to obtain the asynchronous deviation value, sum and average the asynchronous deviation values ​​corresponding to all asynchronous audio to obtain the asynchronous deviation mean, and ratio the asynchronous deviation mean to the synchronization matching threshold to obtain the asynchronous deviation degree value.

[0076] The asynchronous audio quantity ratio is multiplied by the asynchronous deviation value to obtain the asynchronous degree value;

[0077] In some embodiments, the asynchronicity level value is compared with the asynchronicity level threshold. The specific comparison process is as follows:

[0078] If the asynchronous level value is greater than or equal to the asynchronous level threshold, it indicates that the current video and all corresponding audio types are severely asynchronous.

[0079] If the asynchrony level value is less than the asynchrony level threshold, it means that the current video and all corresponding audio types are not significantly asynchronous.

[0080] The purpose of setting the asynchronous level value is:

[0081] Function 1: By integrating the proportion of asynchronous audio, it reflects the coverage of asynchronous audio types, and by combining the asynchronous deviation value, it reflects the severity of the deviation of individual asynchronous audio, forming a comprehensive quantitative assessment of the overall asynchronous status of the current video and all types of audio.

[0082] The second function is to determine the global severity of asynchronous problems, which is beneficial for subsequently prioritizing the adjustment of the frame rate of high-deviation audio in scenarios with severe asynchronous behavior.

[0083] Audio frame rate adjustment module: If the asynchrony is severe, the frame rate of the audio is adjusted first, and the video with severe asynchrony is adjusted first. The final adjustment amount of the frame rate of the asynchronous audio in the video with severe asynchrony is determined. Multi-frame smooth transition is used to determine the audio frame rate after the audio frame is adjusted in the asynchronous audio.

[0084] Specifically, the video frame interval and the original audio frame rate are obtained, and combined with the average asynchronous deviation, the basic adjustment amount JT of the audio frame rate is calculated. The specific calculation formula is as follows:

[0085]

[0086] In the formula, ΔTa represents the mean asynchronous deviation, Tf represents the video frame interval, and f ao Indicates the original audio frame rate;

[0087] Based on the basic adjustment amount of the audio frame rate, an empirical parameter is introduced to calculate the final adjustment amount ZT. The specific calculation formula is as follows:

[0088] ZT = JT × (1 + ∝ × D)

[0089] In the formula, ∝ represents an empirical parameter, and D represents the degree of asynchrony.

[0090] To avoid audio distortion due to excessive adjustment, set a safe range for frame rate adjustment [f]. min f max ] Calculate the adjusted frame rate TH;

[0091] TH = Clamp(f ao ±ZT,f min f max )

[0092] A multi-frame smooth transition is employed to obtain the original audio frame rate and the number of transition frames in the asynchronous audio. Combined with the adjusted frame rate, the adjusted audio frame rate f in the asynchronous audio is calculated. an (t), the specific calculation formula is as follows:

[0093]

[0094] In the formula, R represents the number of transition frames, fac This represents the original audio frame rate in asynchronous audio.

[0095] Understandably, the purpose of using multi-frame smooth transition is to effectively prevent auditory discomfort caused by abrupt changes in audio frames;

[0096] The technical solution of this embodiment is as follows: If there is asynchrony, deviation analysis is performed on the asynchronous audio to determine the degree of asynchrony between the current video and all corresponding types of audio. If the asynchrony is severe, the frame rate of the audio is adjusted first, prioritizing the adjustment of the video with severe asynchrony. The final adjustment amount of the frame rate of the asynchronous audio in the video with severe asynchrony is determined, and a multi-frame smooth transition is used to determine the adjusted audio frame rate of the asynchronous audio. This invention integrates the proportion of asynchronous audio and the deviation value by the asynchrony degree value to form a quantitative assessment of the global asynchronous state of audio and video, accurately reflecting the coverage and severity of the asynchronous problem, providing a scientific basis for subsequent optimization, and improving the real-time rendering quality of animation scenes. Secondly, for scenarios with severe asynchrony, the basic adjustment amount and the final adjustment amount of the audio frame rate are calculated, and combined with multi-frame smooth transition technology, the frame rate of the asynchronous audio is effectively adjusted to avoid auditory discomfort caused by frame abrupt changes and ensure the smoothness of audio-visual synchronization. Finally, by setting a safe range for the frame rate adjustment amount, audio distortion caused by over-adjustment is prevented, improving the robustness of the optimization strategy and user experience.

[0097] Example 3

[0098] Please see Figure 2 As shown in the figure, a method for real-time rendering and audio synchronization of an animation scene according to an embodiment of the present invention includes the following steps:

[0099] Step 1: Obtain the animation scene video stream, extract real-time video and audio from the animation scene video stream, analyze the synchronization matching degree of real-time video and audio, add timestamps to the video and various types of audio, distinguish between synchronous and asynchronous timestamps, and determine the proportion of asynchronous timestamps and the degree of asynchronous timestamp offset.

[0100] Extract all audio data corresponding to any video segment. The audio data includes the audio type. Add timestamps to the video and all types of audio.

[0101] Combine the timestamps added to any type of audio into an audio timestamp sequence;

[0102] Extract all timestamps corresponding to the current video and integrate them into a video timestamp sequence;

[0103] Calculate the time difference between the audio timestamp and the video timestamp based on the audio timestamp sequence and the video timestamp sequence;

[0104] If the time difference is greater than or equal to the standard time difference value, the corresponding timestamp will be recorded as an asynchronous timestamp.

[0105] Count the number of asynchronous timestamps among all timestamps and calculate the percentage of asynchronous timestamps.

[0106] The asynchronous timestamp offset is obtained by subtracting the time difference corresponding to the asynchronous timestamp from the standard time difference value. The asynchronous timestamp offsets corresponding to all asynchronous timestamps are summed and averaged to obtain the asynchronous timestamp offset average. The asynchronous timestamp offset average is then compared with the time difference standard value to obtain the asynchronous timestamp offset degree ratio.

[0107] Step 2: Based on the proportion of asynchronous timestamps and the ratio of asynchronous timestamp offsets, assess whether the current animation scene video and the current type of audio are synchronized;

[0108] The synchronization matching value is obtained by multiplying the proportion of asynchronous timestamps by the ratio of asynchronous timestamp offset.

[0109] If the synchronization matching value is greater than or equal to the synchronization matching threshold, it means that the current animation scene video and the current type of audio are out of sync.

[0110] Step 3: If there is a lack of synchronization, perform deviation analysis on the asynchronous audio to determine the degree of asynchrony between the current video and all corresponding types of audio;

[0111] Record the corresponding audio type as asynchronous audio, count the number of asynchronous audio in all types of audio, and calculate the percentage of asynchronous audio in all types of audio;

[0112] Extract the synchronization matching value corresponding to the asynchronous audio, subtract the synchronization matching value corresponding to the asynchronous audio from the synchronization matching threshold to obtain the asynchronous deviation value, sum and average the asynchronous deviation values ​​corresponding to all asynchronous audio to obtain the asynchronous deviation mean, and ratio the asynchronous deviation mean to the synchronization matching threshold to obtain the asynchronous deviation degree value.

[0113] The asynchronous audio quantity ratio is multiplied by the asynchronous deviation value to obtain the asynchronous degree value;

[0114] If the asynchronous level value is greater than or equal to the asynchronous level threshold, it indicates that the current video and all corresponding audio types are severely asynchronous.

[0115] If the asynchrony level value is less than the asynchrony level threshold, it means that the current video and all corresponding audio types are not significantly asynchronous.

[0116] Step 4: If the asynchrony is severe, adjust the audio frame rate first, then adjust the video with severe asynchrony, determine the final adjustment amount of the asynchronous audio frame rate in the video with severe asynchrony, and use multi-frame smooth transition to determine the audio frame rate after the audio frame rate adjustment in the asynchronous audio.

[0117] The video frame interval and the original audio frame rate are obtained, and the basic adjustment amount of the audio frame rate is calculated by combining the average asynchronous deviation.

[0118] Based on the basic adjustment amount of the audio frame rate, empirical parameters are introduced to calculate the final adjustment amount;

[0119] To avoid audio distortion due to excessive adjustment, set a safe range for frame rate adjustment and calculate the adjusted frame rate;

[0120] Using a multi-frame smooth transition, the audio frame rate after audio frame adjustment in asynchronous audio is calculated.

[0121] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A real-time rendering and audio synchronization system for animation scenes, characterized in that: Specifically comprising the following modules: Data acquisition module: acquire the animation scene video stream, extract real-time video and audio from the animation scene video stream, analyze the synchronization matching degree of real-time video and audio, add timestamps to the video and each type of audio, distinguish between synchronous and asynchronous timestamps, determine the asynchronous timestamp quantity proportion and asynchronous timestamp offset degree ratio; Matching degree analysis module: according to the asynchronous timestamp quantity proportion and asynchronous timestamp offset degree ratio, evaluate whether the current animation scene video and the current type of audio are synchronized; The specific process of evaluating whether the current animation scene video and the current type of audio are synchronized is: According to the extracted real-time video and audio in the animation scene video stream, analysis is performed to obtain the asynchronous timestamp quantity proportion and asynchronous timestamp offset degree ratio; The asynchronous timestamp quantity proportion and asynchronous timestamp offset degree ratio are multiplied to obtain the synchronization matching value; If the synchronization matching value is greater than or equal to the synchronization matching threshold, it indicates that the current animation scene video and the current type of audio are not synchronized; Asynchronous degree analysis module: if not synchronized, analyze the deviation of the asynchronous audio to determine the asynchronous degree of the current video and all types of corresponding audio; The specific process of determining the asynchronous degree of the current video and all types of corresponding audio is: The video and audio asynchronous type audio is recorded as asynchronous audio, and deviation analysis is performed to obtain the asynchronous audio quantity proportion and asynchronous deviation degree value; The asynchronous audio quantity proportion and asynchronous deviation degree value are multiplied to obtain the asynchronous degree value; If the asynchronous degree value is greater than or equal to the asynchronous degree threshold, it indicates that the asynchronous degree of the current video and all types of corresponding audio is serious; Audio frame rate adjustment module: if the asynchronous degree is serious, adjust the frame rate of the audio to prioritize adjusting the video with serious asynchronous degree, determine the final adjustment amount of the frame rate of the asynchronous audio in the asynchronous serious video, and use multi-frame smooth transition to determine the audio frame rate after adjusting the audio frame of the asynchronous audio.

2. The real-time rendering and audio synchronization system for animation scene according to claim 1, wherein: The asynchronous timestamp acquisition process is: Extract all audio data corresponding to any video, where the audio data includes audio types, add timestamps to the video and all types of audio; Integrate the timestamps added by any type of audio into an audio timestamp sequence; extract all timestamps corresponding to the current video and integrate them into a video timestamp sequence; According to the audio timestamp sequence and the video timestamp sequence, calculate the time difference between the audio timestamp and the video timestamp; If the time difference is greater than or equal to the time difference standard value, the corresponding timestamp is recorded as an asynchronous timestamp.

3. The real-time rendering and audio synchronization system of an animation scene according to claim 1, wherein: The asynchronous timestamp quantity proportion acquisition method is: Calculate the asynchronous timestamp quantity proportion by counting the number of asynchronous timestamps in all timestamps.

4. The real-time rendering and audio synchronization system of an animation scene according to claim 1, wherein: The asynchronous timestamp offset degree ratio acquisition method is: Difference processing is performed on the time difference corresponding to the asynchronous timestamp and the time difference standard value to obtain the asynchronous timestamp offset, the sum of the asynchronous timestamp offset corresponding to all asynchronous timestamps is taken as the mean value, and then the ratio processing is performed on the time difference standard value to obtain the asynchronous timestamp offset degree ratio.

5. The real-time rendering and audio synchronization system of an animation scene according to claim 1, wherein: The asynchronous audio quantity proportion acquisition method is: The number of asynchronous audios in all types of audios is counted, and the proportion of the number of asynchronous audios in all types of audios is calculated.

6. The real-time rendering and audio synchronization system of an animation scene according to claim 1, wherein: The acquisition process of the asynchronous deviation degree value is: The synchronous matching value corresponding to the asynchronous audio is extracted, the synchronous matching value corresponding to the asynchronous audio is subtracted from the synchronous matching threshold to obtain the asynchronous deviation value, the asynchronous deviation values corresponding to all asynchronous audios are summed and averaged, and then the asynchronous deviation degree value is obtained by ratio processing with the synchronous matching threshold.

7. The real-time rendering and audio synchronization system of an animation scene according to claim 6, wherein: The acquisition process of the final adjustment amount of the frame rate of the asynchronous audio is: The video frame interval and the original audio frame rate are obtained, and the basic adjustment amount of the audio frame rate is calculated in combination with the asynchronous deviation mean value; According to the basic adjustment amount of the audio frame rate, an experience parameter is introduced to calculate the final adjustment amount.

8. The real-time rendering and audio synchronization system of an animation scene according to claim 7, wherein: The acquisition process of the audio frame rate of the audio frame adjusted in the asynchronous audio is: A frame rate adjustment amount safety range is set, and the adjusted frame rate is calculated according to the final adjustment amount; Multi-frame smooth transition is adopted to obtain the original audio frame rate of the audio frame in the asynchronous audio and the transition frame number, and the adjusted audio frame rate of the audio frame in the asynchronous audio is calculated in combination with the adjusted frame rate.

Images