Video rendering method and device, electronic equipment and storage medium

Abstract

The invention provides a video rendering method and device, electronic equipment and a storage medium, and the method is applied to the field of computers, and comprises the steps: responding to a preset triggering condition, and determining the number of to-be-rendered frames; according to the frame number of the to-be-rendered frames and the rated frame rate, determining the interval duration; and rendering the next frame of the current frame after a period of interval duration by taking the moment when the current frame is rendered as a starting point. According to the method, the situation that the video rendering process is fast and slow can be effectively avoided, the fluency and continuity in the video playing process are ensured, and the watching experience of a user is remarkably improved.

Description

Technical Field

[0001] The present application relates to the computer field, and more specifically, to a video rendering method, device, electronic device and storage medium in the computer field. Background Art

[0002] In the prior art, with the rapid advancement of in-vehicle infotainment system technology, in-vehicle video playback has become an increasingly popular demand. However, due to the instability of the video data stream transmission rate of the in-vehicle system, directly decoding and rendering each frame of video data often causes the picture to be fast or slow, or even freezes and distorted screens, affecting the smoothness of video playback. Therefore, how to ensure the smoothness of video playback is a technical problem that needs to be solved at present. Summary of the invention

[0003] The present application provides a method, device, electronic device and storage medium for video rendering, which can make the rendering process more stable and improve the fluency of video playback.

[0004] In a first aspect, a method for video rendering is provided, the method comprising: determining the number of frames to be rendered in response to a preset trigger condition; determining an interval duration based on the number of frames and a rated frame rate; taking the moment when the current frame rendering is completed as the starting point, rendering the next frame of the current frame after an interval of the interval duration.

[0005] The above technical solution, when the preset trigger conditions are met, obtains the number of frames to be rendered at this time. Since the rated frame rate is usually fixed, if the number of frames to be rendered changes dynamically, that is, the backlog number of frames to be rendered changes dynamically, the interval duration determined according to the number of frames and the rated frame rate will also change dynamically. The above technical solution, on the premise of meeting the preset trigger conditions, will determine a suitable interval duration based on the current number of frames to be rendered and the rated frame rate, and will render the next frame after the interval duration, so that the rendering speed can adapt to the current backlog number of frames to be rendered, which is conducive to avoiding the situation where the video frames appear fast or slow during the rendering process, ensuring the smoothness and continuity of the video playback process, and significantly improving the user's viewing experience.

[0006] In combination with the first aspect, in certain implementations of the first aspect, the interval duration is determined based on the number of frames and the rated frame rate, including: if the number of frames is less than the rated frame rate, calculating the proportion of the number of frames to the rated frame rate; based on the proportion, determining the interval duration; wherein the interval duration is negatively correlated with the proportion.

[0007] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, determining the interval duration includes: calculating the standard interval duration between rendering two frames at the rated frame rate; determining a correction ratio for the standard interval duration based on the proportion; wherein the correction ratio is 1 minus the difference between the proportion; and correcting the standard interval duration based on the correction ratio to obtain the interval duration.

[0008] The above technical scheme calculates the standard interval duration according to the rated frame rate, and then corrects the standard interval duration according to the number of frames to be rendered to obtain the interval duration. By correcting the standard interval duration to determine the interval duration, when the ratio of the number of frames to be rendered to the rated frame rate gradually increases, the interval duration can be gradually reduced, and when the ratio of the number of frames to be rendered to the rated frame rate gradually decreases, the interval time can be gradually increased, thereby avoiding large fluctuations in the interval duration and achieving the purpose of stabilizing the rendering speed.

[0009] In combination with the first aspect and the above-mentioned implementation methods, in certain implementation methods of the first aspect, the standard interval duration is corrected according to the correction ratio to obtain the interval duration, including: calculating the product of the correction ratio and the standard interval duration; when the product is greater than or equal to a preset minimum duration, using the product as the interval duration; when the product is less than the minimum duration, using the minimum duration as the interval duration.

[0010] The above technical solution pre-sets the minimum duration, and then finally determines the interval duration by comparing the calculated interval duration and the minimum duration. This can avoid the renderer rendering too fast during the rendering process due to the interval duration being too short, thereby affecting the smoothness of the video content.

[0011] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, the interval duration is determined based on the number of frames and the rated frame rate, including: if the number of frames is greater than or equal to the rated frame rate, then the preset minimum duration is determined as the interval duration.

[0012] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, determining the number of frames to be rendered includes: determining the number of frames to be rendered from a pre-created rendering buffer channel; wherein the rendering buffer channel is used to store decoded frames to be rendered; taking the moment when the current frame rendering is completed as the starting point, rendering the next frame of the current frame after the interval length, including: taking the moment when the current frame rendering is completed as the starting point, after the interval length, obtaining the next frame of the current frame from the rendering buffer channel for rendering.

[0013] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, before determining the number of frames to be rendered in the pre-created rendering buffer channel, the method also includes: storing each video frame decomposed from the video stream in a pre-created decoding buffer channel; obtaining the video frame from the decoding buffer channel for decoding to obtain the decoded frame to be rendered, and storing the decoded frame to be rendered in the rendering buffer channel.

[0014] The above technical solution pre-sets a decoding buffer channel, and stores the video frames decomposed from the video stream in the decoding buffer channel. When the previous video frame is decoded, the next decomposed video frame is obtained from the decoding buffer channel for decoding. By setting the decoding buffer channel, it helps to smooth the decoding speed of the video frames and play a buffering role. At the same time, when the number of video frames after the video stream is decomposed is large, even if the decoding cannot be done in time, the decoding buffer channel can also be used to temporarily store the decomposed but not yet decoded video frames, thereby avoiding the loss of video frames and further avoiding the phenomenon of screen distortion during video playback.

[0015] In a second aspect, a video rendering device is provided, which includes: a first determination module, used to determine the number of frames to be rendered in response to satisfying a preset trigger condition; a second determination module, used to determine the interval duration based on the number of frames and the rated frame rate; a rendering module, used to take the moment when the current frame rendering is completed as the starting point, and render the next frame of the current frame after the interval duration.

[0016] In combination with the second aspect, in certain implementations of the second aspect, the second determination module is specifically used to: if the number of frames to be rendered is less than the rated frame rate, calculate the proportion of the number of frames to be rendered to the rated frame rate; determine the interval duration based on the proportion; wherein the interval duration is negatively correlated with the proportion.

[0017] In combination with the second aspect, in certain implementations of the second aspect, the second determination module is specifically used to: calculate the standard interval duration between rendering two frames at the rated frame rate; determine the correction ratio for the standard interval duration based on the proportion; wherein the correction ratio is 1 minus the difference between the proportion; and according to the correction ratio, correct the standard interval duration to obtain the interval duration.

[0018] In combination with the second aspect, in certain implementations of the second aspect, the second determination module is specifically used to: calculate the product of the correction ratio and the standard interval duration; when the product is greater than or equal to a preset minimum duration, use the product as the interval duration; when the product is less than the minimum duration, use the minimum duration as the interval duration.

[0019] In combination with the second aspect, in some implementations of the second aspect, the second determination module is specifically used to: if the number of frames is greater than or equal to the rated frame rate, determine the preset minimum duration as the interval duration.

[0020] In combination with the second aspect, in certain implementations of the second aspect, the rendering module includes a decomposition unit and a rendering unit, the decomposition unit is specifically used to: store the video frames decomposed from the video stream in a pre-created decoding buffer channel; the decoding unit is specifically used to: obtain the video frames from the decoding buffer channel for decoding, obtain the decoded frames to be rendered, and store the decoded frames to be rendered in the rendering buffer channel.

[0021] In combination with the second aspect, in some implementations of the second aspect, the rendering module is further specifically used to: take the moment when the current frame rendering is completed as the starting point, and after an interval of time, obtain the next frame of the current frame from the rendering buffer channel for rendering.

[0022] In a third aspect, an electronic device is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the electronic device executes the method in the first aspect or any possible implementation of the first aspect.

[0023] In a fourth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0024] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a timing diagram of a method for decoding and rendering a video data stream provided by the prior art;

[0026] Figure 2 is a schematic flow chart of a video rendering method provided in an embodiment of the present application;

[0027] Figure 3 is a timing diagram of a video rendering method provided in an embodiment of the present application;

[0028] Figure 4 is a schematic flow chart of another video rendering method provided in an embodiment of the present application;

[0029] Figure 5 is a structural schematic diagram of a video rendering device provided in an embodiment of the present application;

[0030] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0032] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0033] To facilitate understanding of the embodiments of the present application, the technical terms involved in the embodiments of the present application are first explained below:

[0034] H264 video raw stream: H264 is an advanced video coding (AVC) standard, widely used in video compression. H264 video raw stream refers to the original video data stream that is not encapsulated in any container format.

[0035] Decoder: A device or software used to restore the encoded video data stream to unencoded video frames. In this solution, the decoder processes the H264 video raw stream and outputs the decoded video frames.

[0036] Renderer: A device or software used to convert decoded video frames into visual images.

[0037] Buffer channel: a buffer used to temporarily store data to ensure smooth data processing. The buffer channels involved in this embodiment include: a decoding buffer channel and a rendering buffer channel. Among them, the decoding buffer channel is used to store frame data to be decoded, and the rendering buffer channel is used to store frame data to be rendered.

[0038] Frame rate timer: A device or software used to control the interval length of frame data rendering. The interval length is dynamically adjusted according to the number of frames in the rendering buffer channel to ensure the continuity and smoothness of video playback.

[0039] Rated frame rate: The number of video stream frames collected by the camera in one second. The rated frame rate is usually obtained from the camera's factory parameters.

[0040] In the current vehicle-mounted system, there is a core component, which includes a decoder and a renderer. Generally, the video data stream is acquired by the vehicle-mounted camera and decoded and rendered on the decoder and renderer of the vehicle-mounted system.

[0041] Figure 1 It is a timing diagram of a method for decoding and rendering a video data stream provided by the prior art.

[0042] For example, Figure 1 As shown, the method 100 includes:

[0043] S101: Input a video data stream obtained from a video data source into a decoder.

[0044] S102: The decoder decodes the video data stream to obtain decoded frame data.

[0045] S103, input the decoded frame data into a renderer.

[0046] S104: A renderer renders the decoded frame data.

[0047] In the above scheme, when the transmission rate of the video data stream is unstable, the video screen may be played too slowly, or even freeze during the video playback. For example, if the input video data stream is too fast for the decoder to process in time, it may cause the loss of frame data, or even the phenomenon of flower screen during the video playback. If the input video data stream is insufficient and causes decoding waiting, or there is a mismatch between the decoding and rendering rates, especially when the decoding rate is faster than the rendering rate, a large number of decoded frames to be rendered will accumulate in the renderer, so that the renderer renders the decoded frames to be rendered continuously, resulting in the video screen being played too fast. On the contrary, if the decoding rate is slower than the rendering rate, the number of frames to be rendered after decoding in the renderer is small, so that the renderer may not be able to continuously render the decoded frames to be rendered, which may cause the video screen to be played too slowly, or even freeze during the video playback. The above-mentioned freeze and flower screen phenomena not only destroy the fluency and continuity of the video content, but also seriously affect the user's viewing experience.

[0048] That is to say, in the current common video data stream decoding and rendering schemes, due to the instability of the video data stream transmission rate of the vehicle system, directly decoding and rendering each frame of video data will often cause the picture to be sometimes fast and sometimes slow, or even cause the picture to be stuck and the screen to be distorted.

[0049] In order to solve the above technical problems, the embodiment of the present application provides a method for video rendering, the execution subject of the method is an electronic device, and the electronic device can be used to process the video stream, for example, the electronic device can be a mobile phone, a computer, a smart TV, a vehicle computer, etc. The following is an example of applying the video stream processing method to a vehicle computer in a vehicle:

[0050] Figure 2 It is a schematic flowchart of a video rendering method provided in an embodiment of the present application.

[0051] For example, Figure 2 The method 200 comprises:

[0052] S201, in response to satisfying a preset trigger condition, determining the number of frames to be rendered.

[0053] S202: Determine the interval duration according to the number of frames and the rated frame rate.

[0054] S203, taking the moment when the current frame rendering is completed as the starting point, rendering the next frame of the current frame after the interval time.

[0055] In an embodiment of the present application, when the preset trigger conditions are met, the number of frames to be rendered at this time is obtained. Since the rated frame rate is usually fixed, if the number of frames to be rendered changes dynamically, that is, the backlog number of frames to be rendered changes dynamically, the interval duration determined according to the number of frames and the rated frame rate will also change dynamically. Under the premise of meeting the preset trigger conditions, the above technical scheme will determine a suitable interval duration based on the number of frames to be rendered and the rated frame rate, and render the next frame after the interval duration, so that the rendering speed can adapt to the backlog number of frames to be rendered, which is conducive to avoiding the situation where the video frames appear fast or slow during the rendering process, ensuring the smoothness and continuity of the video playback process, and significantly improving the user's viewing experience.

[0056] Below Figure 2 The implementation method of each step in the embodiment shown is specifically described:

[0057] As for S201, it can be understood that the above trigger condition is a pre-set condition for triggering the step of determining the number of frames to be rendered. Whenever it is detected that the preset trigger condition is met, the step of determining the number of frames to be rendered will be executed.

[0058] Exemplarily, the trigger condition may be: detecting that the current frame is being rendered. The current frame refers to the video frame being rendered in the renderer, that is, the video frame being rendered. In other words, if it is detected that the current frame is being rendered, the number of frames to be rendered can be determined. The above-mentioned frames to be rendered refer to frame data that has been decoded in the decoder but has not yet been rendered, that is, frames waiting for the renderer to render. Correspondingly, the number of frames to be rendered can be understood as the number of backlog frames that have not yet been rendered by the renderer.

[0059] Exemplarily, detecting that the current frame is being rendered may specifically include: detecting that the current frame starts to be rendered, detecting that the current frame is rendered completely, or detecting that the current frame starts to be rendered but is not rendered completely.

[0060] In some embodiments, the above-mentioned determination of the number of frames to be rendered includes: determining the number of frames to be rendered from a pre-created rendering buffer channel; wherein the rendering buffer channel is used to store decoded frames to be rendered.

[0061] Specifically, a rendering buffer channel may be created in advance, frames to be rendered that have been decoded in the decoder but not yet rendered are placed in the pre-created rendering buffer channel, and then the number of frames to be rendered is determined from the pre-created rendering buffer channel.

[0062] Optionally, the structure of the rendering buffer channel can be implemented in a queue manner. When the renderer is ready to render, the frame data of the decoded frame to be rendered is taken out from the rendering buffer channel and rendered. The queue method can ensure that the frame data to be rendered in the rendering buffer channel is arranged in the order in which it is added to the rendering buffer channel. The earliest added frame data to be rendered is rendered first. This feature enables the frame data to be rendered to be rendered in order.

[0063] It is understandable that when the decoder outputs multiple frames to be rendered, there may be a short period of time when there are no video frames in the decoder, which will cause the decoding process to be interrupted. However, since the decoded frames to be rendered are accumulated in the rendering buffer channel, the renderer can still take out the decoded frames to be rendered from the rendering buffer channel for rendering, so that the rendering process can continue.

[0064] Exemplarily, when a rendering buffer channel is created, the trigger condition may also be: detecting a change in the number of frames to be rendered in the rendering buffer channel. For example, when detecting that a new video frame is added to the rendering buffer channel, or when detecting that a video frame is taken out of the rendering buffer channel, it may be considered that a change in the number of frames to be rendered in the rendering buffer channel is detected.

[0065] In some embodiments, before determining the number of frames to be rendered from a pre-created rendering buffer channel, the method also includes: storing video frames decomposed from a video stream in a pre-created decoding buffer channel; obtaining video frames from the decoding buffer channel for decoding to obtain decoded frames to be rendered, and storing the decoded frames to be rendered in the rendering buffer channel.

[0066] Specifically, a decoding buffer channel may be created in advance, and the decomposed video frames may be stored in the decoding buffer channel instead of being immediately input into the decoder for decoding; wherein the decoding buffer channel is used to store video frames that have not yet been decoded by the decoder.

[0067] Optionally, the structure of the decoding buffer channel can be implemented in a queue manner. When the decoder is ready to decode, the decoder takes out the video frames to be decoded from the decoding buffer channel. The queue manner can ensure that the video frames to be decoded in the decoding buffer channel are arranged in the order in which they are added to the decoding buffer channel. The earliest added video frames to be decoded are decoded first. This feature enables the video frames to be decoded in sequence to prepare for subsequent sequential rendering.

[0068] It is understandable that when the number of video frames after the video stream is decomposed is large, even if the decoder cannot decode in time, the decoding buffer channel can be used to temporarily store the decomposed but not yet decoded video frames, thereby avoiding video frame loss and further avoiding the phenomenon of video screen distortion.

[0069] In some embodiments, after the video stream is acquired, operations related to video stream decomposition may be performed on the video stream to form decomposed video frames.

[0070] Specifically, the video stream may be a video stream regularly acquired from an onboard camera of the vehicle, and may include real-time road conditions during the driving of the vehicle, including road conditions ahead, traffic signs, other vehicles, pedestrians, etc.

[0071] The vehicle's onboard camera usually continuously outputs video streams in a specific encoding format (such as H264). Among them, H264 is a widely used video encoding standard, which is mainly used to efficiently compress video data, reduce the size of video files, and try to maintain the quality of the video. Correspondingly, the video stream output in the H264 encoding standard is an H264 video raw stream, which refers to the original video data stream that is not encapsulated in any container format.

[0072] Regarding the above S202, it can be understood that the above number of frames is the number of frames to be rendered in the rendering buffer channel.

[0073] Furthermore, the interval duration may be determined based on the number of frames to be rendered obtained from the rendering buffer channel and the rated frame rate. Since the number of frames to be rendered in the rendering buffer channel may change dynamically, the interval duration determined based on this may also change dynamically.

[0074] In one possible implementation, the interval duration is determined based on the number of frames and the rated frame rate, including: if the number of frames is less than the rated frame rate, calculating the proportion of the number of frames to the rated frame rate; based on the proportion, determining the interval duration; wherein the interval duration is negatively correlated with the proportion.

[0075] Specifically, the relationship between the number of frames to be rendered in the rendering buffer channel and the rated frame rate can be first determined. When the number of frames to be rendered in the rendering buffer channel is less than the rated frame rate, it means that the number of frames to be rendered in the rendering buffer channel is relatively small, so the interval duration can be further determined in combination with the proportion of the number of frames to the rated frame rate. The above interval duration is negatively correlated with the above proportion. For example, when the proportion is large, it means that the number of frames to be rendered in the rendering buffer channel is large. In order to avoid the phenomenon of stuttering during video playback caused by too slow rendering, this embodiment determines a smaller interval duration through the above larger proportion to speed up the rendering speed of the renderer, thereby avoiding video stuttering. Conversely, when the proportion is small, it means that the number of frames to be rendered in the rendering buffer channel is small. In order to avoid the phenomenon of video acceleration during video playback caused by too fast rendering, this embodiment determines a larger interval duration through the above smaller proportion to slow down the rendering speed of the renderer, thereby avoiding video acceleration.

[0076] A possible implementation method is to determine the interval duration according to the above proportion, including: according to the above proportion, a preset correspondence table can be queried to obtain the interval duration corresponding to the proportion, and the correspondence table can be pre-calibrated. Assuming that the rated frame rate is 30 frames, the above correspondence table can be shown in Table 1 below.

[0077] Table 1

[0078]

[0079]

[0080]

[0081] It can be seen from Table 1 that when the number of frames to be rendered in the rendering buffer channel gradually increases, that is, when the ratio of the number of frames to be rendered in the rendering buffer channel to the rated frame rate gradually increases, the interval duration gradually decreases, so the interval duration and the ratio of the number of frames to be rendered in the rendering buffer channel to the rated frame rate are negatively correlated.

[0082] It should be understood that the numerical values ​​in Table 1 are only illustrative examples and are not specifically limited in this embodiment.

[0083] In another possible implementation, the interval duration is determined based on the above-mentioned ratio, including: calculating the standard interval duration between rendering two frames at the rated frame rate; determining a correction ratio for the standard interval duration based on the ratio; wherein the correction ratio is 1 minus the difference between the ratio; and correcting the standard interval duration based on the correction ratio to obtain the interval duration.

[0084] It can be understood that the standard interval time between rendering two frames is calculated based on the rated frame rate.

[0085] For example, if the unit of the interval duration is milliseconds, the calculation formula of the standard interval duration is: t=1000 / P. For example, when the rated frame rate P is 30 frames, the standard interval duration between rendering two frames is 33 milliseconds.

[0086] For example, if the unit of the interval duration is seconds, the calculation formula of the standard interval duration is: t=1 / P. For example, when the rated frame rate P is 30 frames, the standard interval duration between rendering two frames is 0.033 seconds.

[0087] Furthermore, according to the number of frames δ to be rendered in the above-mentioned rendering buffer channel and the proportion δ / P of the rated frame rate P, the difference between 1 and the proportion δ / P is calculated, and the standard interval duration is corrected according to the calculated difference. Since the number of frames to be rendered in the rendering buffer channel is different, the calculated difference changes, which in turn affects the interval duration and further affects the rendering speed, thereby avoiding the video from being too fast or too slow during playback and ensuring the smoothness of the video playback.

[0088] In a possible implementation, the standard interval duration is corrected according to the correction ratio to obtain the interval duration, including: calculating the product of the correction ratio and the standard interval duration, and using the product as the interval duration.

[0089] Exemplarily, if the unit of the interval duration is milliseconds, the calculation formula of the interval duration is: T=(1000 / P)*(1-δ / P).

[0090] Exemplarily, if the unit of the interval duration is seconds, the calculation formula of the interval duration is: T=(1 / P)*(1-δ / P).

[0091] In another possible implementation, the standard interval duration is corrected according to the correction ratio to obtain the interval duration, including: calculating the product of the correction ratio and the standard interval duration; when the product is greater than or equal to a preset minimum duration, using the product as the interval duration; when the product is less than the minimum duration, using the minimum duration as the interval duration.

[0092] It is understandable that after calculating the product of the correction ratio and the standard interval duration, a logical judgment needs to be made, which is based on a preset minimum duration, which is a minimum safety threshold preset by the renderer to avoid rendering too fast. The minimum duration can be set based on experience, such as about 10 milliseconds.

[0093] Specifically, when the product is greater than or equal to the minimum duration, it means that the product can be directly used as the interval duration. If the product is less than the minimum duration, it means that if the product is directly used as the interval duration, the rendering speed may be too fast. Therefore, in this case, the minimum duration needs to be used as the interval duration to avoid too fast rendering speed.

[0094] In a possible implementation, according to the number of frames to be rendered and the rated frame rate, the process of determining the interval duration can be calculated by the following formula:

[0095] T=max((1000 / P)*(max(1-(δ / P),0)),X)

[0096] Wherein, T is the interval duration, P is the rated frame rate, δ is the number of frames to be rendered in the rendering buffer channel, X is the above-mentioned preset minimum duration, for example, X=10 milliseconds, and 1000 / P is the standard interval duration between rendering two frames. In this formula, the unit of T is milliseconds.

[0097] In another possible implementation, according to the number of frames to be rendered and the rated frame rate, the process of determining the interval duration can be calculated by the following formula:

[0098] T=max((1 / P)*(max(1-(δ / P),0)),X)

[0099] Wherein, T is the interval duration, P is the rated frame rate, δ is the number of frames to be rendered in the rendering buffer channel, X is the above-mentioned preset minimum duration, for example, X = 0.01 seconds, and 1 / P is the standard interval duration between rendering two frames. In this formula, the unit of T is seconds.

[0100] It can be seen from the above interval duration formula that T is dynamically calculated based on the number of frames to be rendered δ accumulated in the rendering buffer channel. According to the ratio δ / P of the number of frames to be rendered δ accumulated in the rendering buffer channel to the rated frame rate P, it is determined how long the next frame should be rendered after the current frame is rendered. The interval duration T is dynamically calculated because, considering that when the number of frames to be rendered output by the decoder changes greatly, the user can visually clearly see the video acceleration or freeze. By dynamically calculating the interval duration T, it can be ensured that when the decoder outputs multiple frames to be rendered at a time, each frame can still be rendered in sequence according to the interval duration T, so that the video that could have been accelerated becomes relatively smooth. After the decoder outputs multiple frames to be rendered at a time, the decoder may still have no video frames for a short time, which will lead to decoding interruption. However, since the decoded frames to be rendered are accumulated in the rendering buffer channel, the renderer can still take out the decoded frames to be rendered from the rendering buffer channel for rendering, so it can avoid the phenomenon of video freeze during playback.

[0101] For example, assuming that the minimum duration is 10 milliseconds, when the rated frame rate P is 30 frames, and the number of frames to be rendered in the rendering buffer channel δ is 20 frames, 1-δ / P=0.33. According to the above formula for calculating the standard interval duration, the standard interval duration t between rendering two frames is 33 milliseconds. The standard interval duration t between rendering two frames is multiplied by 1-δ / P, and the product calculated is 11 milliseconds. Since 11 milliseconds is greater than the minimum duration (10 milliseconds), it can be determined that the interval duration is 11 milliseconds.

[0102] For example, assuming that the minimum duration is 10 milliseconds, when the rated frame rate P is 30 frames, and the number of frames to be rendered in the rendering buffer channel δ is 25 frames, 1-δ / P=0.17. According to the above formula for calculating the standard interval duration, the standard interval duration t between rendering two frames is 33 milliseconds. The standard interval duration t between rendering two frames is multiplied by 1-δ / P, and the product calculated is 5.6 milliseconds. Since 5.6 milliseconds is less than the minimum duration (10 milliseconds), the above minimum duration is determined as the interval duration, so the interval duration is 10 milliseconds.

[0103] In a possible implementation, the interval duration is determined according to the number of frames and the rated frame rate, including: if the number of frames is greater than or equal to the rated frame rate, determining the preset minimum duration as the interval duration.

[0104] It can be understood that when the number of frames to be rendered in the rendering buffer channel is greater than or equal to the rated frame rate, it means that the number of frames to be rendered in the rendering buffer channel is too large. In order to avoid the phenomenon of freezes during video playback caused by the renderer rendering speed being too slow, this embodiment speeds up the rendering speed of the renderer by reducing the interval duration. However, since the rendering speed of the renderer cannot be increased endlessly, the pre-set minimum duration can be used as the current interval duration to avoid the phenomenon of too fast video images due to the continuous increase in the rendering speed of the renderer.

[0105] For the above S203, it can be understood that, with the moment when the above current frame rendering is completed as the starting point, after the above interval, the next frame of the current frame is taken out from the rendering buffer channel for rendering. The above current frame refers to the video frame being rendered in the renderer, that is, the video frame being rendered. The moment when the current frame rendering is completed can be understood as: the time point when the picture rendering process of the current frame ends.

[0106] In some embodiments, taking the moment when the current frame rendering is completed as the starting point, after an interval, the next frame of the current frame is obtained from the rendering buffer channel for rendering.

[0107] Specifically, when the renderer is ready to render, it will read the frame data of the frame to be rendered from the rendering buffer channel, put it into the renderer for rendering, and after each frame is rendered, it will take an interval and then take out the next frame to be rendered from the rendering buffer channel for rendering.

[0108] Exemplarily, assuming that the rated frame rate is 30 frames, and the current frame is the 3rd frame in the decomposed video stream, when the 3rd frame is rendered, if it is determined that the number of frames to be rendered in the rendering buffer channel is 15 frames, then the interval is determined to be 17 milliseconds based on the number of frames to be rendered and the rated frame rate, so that after the 3rd frame is rendered, the next frame of the 3rd frame is taken from the rendering buffer channel, that is, the 4th frame in the decomposed video stream, and the 4th frame is rendered. When the 4th frame is rendered, if it is determined that the number of frames to be rendered in the rendering buffer channel is 20 frames, then the interval is determined to be 11 milliseconds based on the number of frames to be rendered and the rated frame rate, so that after the 4th frame is rendered, the next frame of the 4th frame is taken from the rendering buffer channel, that is, the 5th frame in the decomposed video stream, and the 5th frame is rendered. When the rendering of the 5th frame is completed, if it is determined that the number of frames to be rendered in the rendering buffer channel is 25 frames, the interval duration is determined to be 10 milliseconds based on the number of frames to be rendered and the rated frame rate. Therefore, after the rendering of the 5th frame is completed, an interval of 10 milliseconds is taken from the rendering buffer channel to take the next frame of the 5th frame, that is, the 6th frame in the decomposed video stream, and render the 6th frame, and so on.

[0109] Specifically, the above process is presented in a table, as shown in Table 2 below.

[0110] Table 2

[0111]

[0112] It can be seen from Table 2 that when the rendering of the current frame is completed, after an interval of time, the next frame of the current frame is taken out from the rendering buffer channel for rendering. The interval time is dynamically calculated based on the number of frames accumulated in the rendering buffer channel. When calculating the interval time of the 5th frame, the product of the correction ratio and the standard interval time calculation is equal to 5.6 milliseconds. Since 5.6 milliseconds is less than 10 milliseconds (the preset minimum time), 10 milliseconds is determined as the corrected interval time.

[0113] Figure 3 is a timing diagram of a video rendering method provided in an embodiment of the present application;

[0114] For example, Figure 3 As shown, the method 300 includes:

[0115] S301, decomposing the acquired video stream to obtain video frames, and inputting the video frames into a decoding buffer channel.

[0116] S302, taking out a video frame from a decoding buffer channel and inputting it into a decoder.

[0117] S303: The decoder decodes the video frame to obtain a decoded frame to be rendered.

[0118] S304: Input the decoded frame to be rendered into the rendering buffer channel.

[0119] S305 , taking out the decoded frame to be rendered from the rendering buffer channel according to the interval duration and inputting it into the renderer.

[0120] S306: The renderer renders the decoded frame to be rendered.

[0121] The above technical solution, by introducing a decoding buffer channel and a rendering buffer channel, can effectively alleviate the problem of inconsistent video data flow rates during decoding and rendering, ensure the smoothness of video playback, and avoid screen freezes. In addition, when the number of video frames to be decoded after decomposition is large, if the decoder cannot decode in time, video frames may be lost. The design of the decoding buffer channel can avoid video frame loss, further avoiding the occurrence of screen distortion during video playback. By introducing the interval duration, the rendering interval time can be dynamically adjusted according to the number of frames to be rendered in the rendering buffer channel in real time, and then the rendering speed of the renderer can be further adjusted, effectively avoiding the situation where the video rendering process is sometimes fast and sometimes slow, ensuring the smoothness and continuity of the video playback process, thereby significantly improving the user's viewing experience.

[0122] Figure 4 is a schematic flow chart of another video rendering method provided in an embodiment of the present application;

[0123] For example, Figure 4 As shown, the method 400 includes:

[0124] S401: Create a decoding buffer channel.

[0125] S402, decomposing the acquired video stream into video frames, and storing the video frames in a decoding buffer channel.

[0126] S403, creating a rendering buffer channel.

[0127] S404, taking out the video frame from the decoding buffer channel, and inputting the video frame into the decoder for decoding.

[0128] It can be understood that when the decoder performs decoding, the decoder takes out the video frames decomposed from the video stream from the decoding buffer channel for decoding.

[0129] Specifically, after the decoding of the previous video frame is completed, the decoder takes out the next video frame from the decoding buffer channel for decoding.

[0130] S405: Store the decoded frame to be rendered into a rendering buffer channel.

[0131] S406: Create a frame rate timer.

[0132] It can be understood that the above-mentioned frame rate timer is software running independently in the vehicle system, which is used to control the rendering interval length of the frame data to be rendered. The interval length is dynamically adjusted according to the number of frames to be rendered in the rendering buffer channel to ensure the continuity and smoothness of the video playback.

[0133] S407, dynamically calculating the interval length of the frame rate timer according to a formula.

[0134] It is understandable that the frame rate timer can dynamically calculate the interval duration according to the number of frames to be rendered in the rendering buffer channel.

[0135] Furthermore, to calculate the interval duration, in addition to obtaining the number of frames to be rendered in the rendering buffer channel, it is also necessary to obtain the rated frame rate, and determine the interval duration according to the number of frames to be rendered and the rated frame rate.

[0136] Specifically, the frame rate timer can take out the decoded frame data to be rendered from the rendering buffer channel according to the interval duration, and input the frame data to be rendered into the renderer.

[0137] In some embodiments, the calculation formula is: T = max((1000 / P)*(max(1-(δ / P),0)),10), where T is the interval duration, P is the rated frame rate, δ is the number of frames to be rendered in the rendering buffer channel, and the unit of T is milliseconds.

[0138] In some embodiments, the calculation formula is: T = max((1 / P)*(max(1-(δ / P),0)),0.01), where T is the interval duration, P is the rated frame rate, δ is the number of frames to be rendered in the rendering buffer channel, and the unit of T is seconds.

[0139] S408: Check whether the frame rate timer is running.

[0140] It is understandable that if the frame rate timer starts running, the decoded frame data to be rendered is taken out from the rendering buffer channel, and if the frame rate timer stops running, the process ends.

[0141] S409, taking out the decoded frame to be rendered from the rendering buffer channel.

[0142] S410: Input the decoded frame to be rendered into a renderer for rendering.

[0143] It is understandable that the rendered frame data will eventually be displayed on the screen, and when the rendered image starts to be played on the screen, the frame rate timer starts running. When the rendered image stops being played on the screen, the frame rate timer stops running.

[0144] Specifically, once the frame rate timer starts running, no matter whether the rendering buffer channel has decoded frame data to be rendered, the frame rate timer will continue to obtain frame data to be rendered from the rendering buffer channel.

[0145] The above technical solution pre-creates a decoding buffer channel and a rendering buffer channel, decomposes the acquired video stream to obtain video frames, and inputs the video frames into the pre-created decoding buffer channel. During decoding, video frames are taken out from the decoding buffer channel and input into the decoder for decoding to form decoded frames to be rendered. The decoded frames to be rendered are input into the pre-created rendering buffer channel, and then a frame rate timer is created. The frame rate timer takes out the decoded frames to be rendered from the rendering buffer channel according to the interval duration and inputs them into the renderer for rendering.

[0146] Figure 5 It is a structural schematic diagram of a video rendering device provided in an embodiment of the present application.

[0147] For example, Figure 5 As shown, the device 500 includes:

[0148] The first determining module 501 is configured to determine the number of frames to be rendered in response to satisfying a preset trigger condition.

[0149] The second determination module 502 is used to determine the interval duration according to the number of frames and the rated frame rate.

[0150] The rendering module 503 is used to take the moment when the current frame rendering is completed as the starting point, and render the next frame of the current frame after the interval time.

[0151] In a possible implementation, the first determination module is specifically used to: in response to satisfying a preset trigger condition, determine the number of frames to be rendered from a pre-created rendering buffer channel; wherein the rendering buffer channel is used to store decoded frames to be rendered.

[0152] In one possible implementation, the second determination module is specifically used to: if the number of frames to be rendered is less than the rated frame rate, calculate the proportion of the number of frames to be rendered to the rated frame rate; determine the interval duration based on the proportion; wherein the interval duration is negatively correlated with the proportion.

[0153] In one possible implementation, the second determination module is specifically used to: calculate the standard interval duration between rendering two frames at a rated frame rate; determine a correction ratio for the standard interval duration based on the proportion; wherein the correction ratio is 1 minus the difference between the proportion; and according to the correction ratio, correct the standard interval duration to obtain the interval duration.

[0154] In one possible implementation, the second determination module is specifically used to: calculate the product of the correction ratio and the standard interval duration; when the product is greater than or equal to a preset minimum duration, use the product as the interval duration; when the product is less than the minimum duration, use the minimum duration as the interval duration.

[0155] In a possible implementation, the second determination module is specifically configured to: if the number of frames is greater than or equal to the rated frame rate, determine a preset minimum duration as the interval duration.

[0156] In one possible implementation, the rendering module includes a decomposition unit and a decoding unit. The decomposition unit is specifically used to: store the video frames decomposed from the video stream in a pre-created decoding buffer channel; the decoding unit is used to obtain the video frames from the decoding buffer channel for decoding, obtain the decoded frames to be rendered, and store the decoded frames to be rendered in the rendering buffer channel.

[0157] In a possible implementation, the rendering module is specifically configured to: take the moment when the current frame rendering is completed as the starting point, and after an interval time, obtain the next frame of the current frame from the rendering buffer channel for rendering.

[0158] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0159] For example, Figure 6 As shown, the electronic device 600 includes: a memory 601 and a processor 602, wherein the memory 601 stores an executable program code 6011, and the processor 602 is used to call and execute the executable program code 6011 to perform a video rendering method.

[0160] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores an executable program code, and the processor is used to call and execute the executable program code to perform a video rendering method provided by an embodiment of the present application.

[0161] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0162] In the case of dividing each functional module according to each function, the device may also include a first determination module, a second determination module and a rendering module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0163] It should be understood that the device provided in this embodiment is used to execute the above-mentioned video rendering method, and thus can achieve the same effect as the above-mentioned implementation method.

[0164] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a vehicle, the processing module may be used to control and manage the actions of the vehicle. The storage module may be used to support the vehicle in executing related program codes, etc.

[0165] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits shown in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0166] In addition, the device provided in the embodiments of the present application may specifically be a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a video rendering method provided in the above embodiments.

[0167] This embodiment also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement a video rendering method provided in the above embodiment.

[0168] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement a video rendering method provided by the above-mentioned embodiment.

[0169] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0170] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0171] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0172] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video rendering method, characterized in that: The method comprises: In response to satisfying a preset trigger condition, determining the number of frames to be rendered; Determining the interval duration according to the number of frames and the rated frame rate; Taking the moment when the current frame rendering is completed as the starting point, the next frame of the current frame is rendered after the interval length.

2. The method according to claim 1, characterized in that Determining the interval duration according to the number of frames and the rated frame rate includes: If the number of frames is less than the rated frame rate, calculating the proportion of the number of frames to the rated frame rate; The interval duration is determined according to the proportion; wherein the interval duration is negatively correlated with the proportion.

3. The method according to claim 2, characterized in that The determining the interval duration according to the proportion includes: Calculate the standard interval time between rendering two frames at the rated frame rate; According to the proportion, determining a correction ratio for the standard interval duration; wherein the correction ratio is a difference between 1 and the proportion; The standard interval duration is corrected according to the correction ratio to obtain the interval duration.

4. The method according to claim 3, characterized in that The step of correcting the standard interval duration according to the correction ratio to obtain the interval duration includes: Calculating the product of the correction ratio and the standard interval duration; When the product is greater than or equal to the preset minimum duration, the product is used as the interval duration; When the product is smaller than the minimum duration, the minimum duration is used as the interval duration.

5. The method according to claim 1, characterized in that Determining the interval duration according to the number of frames and the rated frame rate includes: If the number of frames is greater than or equal to the rated frame rate, the preset minimum duration is determined as the interval duration.

6. The method according to claim 1, characterized in that The step of determining the number of frames to be rendered includes: Determining the number of frames to be rendered from a pre-created rendering buffer channel; wherein the rendering buffer channel is used to store decoded frames to be rendered; The step of taking the moment when the current frame rendering is completed as the starting point and rendering the next frame of the current frame after the interval time has elapsed includes: Taking the moment when the current frame rendering is completed as the starting point, after the interval length, the next frame of the current frame is obtained from the rendering buffer channel for rendering.

7. The method according to claim 6, characterized in that Before determining the number of frames to be rendered from the pre-created rendering buffer channel, the method further includes: The video frames decomposed from the video stream are stored in a pre-created decoding buffer channel; A video frame is obtained from the decoding buffer channel for decoding to obtain a decoded frame to be rendered, and the decoded frame to be rendered is stored in the rendering buffer channel.

8. A video rendering device, characterized in that: The device comprises: A first determination module, configured to determine the number of frames to be rendered in response to satisfying a preset trigger condition; A second determination module is used to determine the interval duration according to the number of frames and the rated frame rate; The rendering module is used to take the moment when the current frame rendering is completed as the starting point, and render the next frame of the current frame after the interval length.

9. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images