Screen control method based on YUV hard decoding and related equipment

Abstract

The invention provides a screen control method based on YUV hard decoding, and the method comprises the steps: receiving an external video control instruction, and determining corresponding video data; performing data conversion on video data corresponding to the control instruction through a preset hardware decoding module to obtain a video data stream, and writing the video data stream into a screen double buffer area which is used for alternately refreshing the data onto a screen; acquiring historical frame rate data and current frame rate data, and determining predicted frame rate data in combination with the historical frame rate data and the current frame rate data; and based on the predicted frame rate data, controlling the video data streams of the screen double buffer areas to be displayed. The problem that an existing screen display device cannot meet user interaction and animation refreshing requirements is solved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a screen control method based on YUV hard decoding and related equipment. Background Art

[0002] Currently, screen display devices are typically developed using embedded processors. When developed on low-end development boards, user interaction and high-refresh rates compete for chip resources, making it impossible to meet both user interaction and animation refresh requirements. In reality, the chip resource usage of high-refresh rates and user interaction is like a seesaw: higher chip resource usage on one side means lower chip resource usage on the other. Therefore, a screen control method that ensures smooth interaction is urgently needed to address the problem of existing screen display devices failing to meet both user interaction and animation refresh requirements. Summary of the Invention

[0003] An embodiment of the present invention provides a screen control method based on YUV hard decoding, aiming to provide a screen control method that ensures smooth interaction, so as to solve the problem that existing screen display devices cannot satisfy both user interaction and animation refresh. The present invention receives an external video control instruction, determines the corresponding video data, converts the video data corresponding to the control instruction through a preset hardware decoding module to obtain a video data stream, writes the video data stream into a screen double buffer, obtains historical frame rate data and current frame rate data, and determines predicted frame rate data in combination with the historical frame rate data and the current frame rate data. The predicted frame rate data is used to control the video data stream in the screen double buffer for display. The present invention solves the problem that existing screen display devices cannot satisfy both user interaction and animation refresh.

[0004] In a first aspect, an embodiment of the present invention provides a screen control method based on YUV hard decoding, the method comprising the following steps:

[0005] Receive external video control instructions and determine corresponding video data;

[0006] Performing data conversion on the video data corresponding to the control instruction through a preset hardware decoding module to obtain a video data stream, and writing the video data stream into a screen double buffer, wherein the screen double buffer is used to alternately refresh the data onto the screen;

[0007] Obtaining historical frame rate data and current frame rate data, and determining predicted frame rate data by combining the historical frame rate data and the current frame rate data;

[0008] Based on the predicted frame rate data, the video data stream of the screen double buffer is controlled to be displayed.

[0009] Optionally, obtaining current frame rate data includes:

[0010] Obtaining a first timestamp before decoding the video data and a second timestamp after decoding the video data;

[0011] Determining, based on the first timestamp and the second timestamp, the duration of multiple single frames of the video data;

[0012] Based on the multiple single frame times, current frame rate data is determined.

[0013] Optionally, determining the current frame rate data based on the multiple single-frame time consumptions includes:

[0014] Calculating an average of the time consumption of multiple single frames;

[0015] Based on the average value, the reciprocal of the average value is used as the current frame rate data.

[0016] Optionally, determining the current frame rate data based on the multiple single-frame time consumptions includes:

[0017] Determining weights corresponding to the time consumption of multiple single frames;

[0018] Based on the weights, a weighted average time consumption is obtained by weighted calculation, and the inverse of the weighted average time consumption is used as the current frame rate data.

[0019] Optionally, combining the historical frame rate data and the current frame rate data to determine predicted frame rate data includes:

[0020] Performing a linear conversion on the historical frame rate data and the current frame rate data to obtain a linear relationship between the historical frame rate data and the current frame rate data;

[0021] The linear relationship is predicted using a preset frame rate prediction model to determine predicted frame rate data.

[0022] Optionally, controlling the video data stream of the screen buffer for display based on the predicted frame rate data includes:

[0023] When the predicted frame rate is lower than a first frame rate threshold, a target discarded frame is obtained, the target discarded frame is eliminated, and the video data stream from which the target discarded frame is eliminated is displayed;

[0024] When the predicted frame rate is greater than the first frame rate threshold and less than a second frame rate threshold, controlling the video data stream of the screen buffer to be displayed according to the predicted frame rate;

[0025] When the predicted frame rate is greater than the second frame rate threshold, hardware resources are released through non-blocking delay.

[0026] Optionally, when the predicted frame rate is lower than a first frame rate threshold, obtaining a target discarded frame, removing the target discarded frame, and displaying the video data stream from which the target discarded frame is removed, includes:

[0027] When the predicted frame rate is lower than a first frame rate threshold, calculating the number of frames corresponding to the predicted frame rate and the number of frames corresponding to the current frame rate to determine a number of frame skips;

[0028] Performing inter-frame calculation on the video data stream to determine a key value for each frame;

[0029] and arranging the key values ​​in ascending order to obtain an ascending order result;

[0030] Based on the number of skipped frames and the ascending order result, the target discarded frames are determined, the target discarded frames are eliminated, and the video data stream from which the target discarded frames are eliminated is displayed.

[0031] In a second aspect, an embodiment of the present invention further provides a screen control device based on YUV hard decoding, the screen control device based on YUV hard decoding comprising:

[0032] A first determining module is used to receive an external video control instruction and determine corresponding video data;

[0033] a conversion module, configured to convert the video data corresponding to the control instruction through a preset hardware decoding module to obtain a video data stream, and write the video data stream into a screen double buffer, wherein the screen double buffer is configured to alternately refresh the data onto the screen;

[0034] A second determination module is configured to obtain historical frame rate data and current frame rate data, and determine predicted frame rate data by combining the historical frame rate data and the current frame rate data;

[0035] A control module is used to control the video data stream of the screen double buffer to display based on the predicted frame rate data.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and runnable on the processor, wherein when the processor executes the computer program, the steps of the screen control method based on YUV hard decoding provided in an embodiment of the present invention are implemented.

[0037] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the screen control method based on YUV hard decoding provided in the embodiment of the invention are implemented.

[0038] In an embodiment of the present invention, an external video control instruction is received to determine the corresponding video data; the video data corresponding to the control instruction is converted by a preset hardware decoding module to obtain a video data stream, and the video data stream is written into a screen double buffer, which is used to alternately refresh data on the screen; historical frame rate data and current frame rate data are obtained, and predicted frame rate data is determined by combining the historical frame rate data and the current frame rate data; based on the predicted frame rate data, the video data stream of the screen double buffer is controlled for display. The present invention receives an external video control instruction, determines the corresponding video data, converts the video data corresponding to the control instruction by a preset hardware decoding module to obtain a video data stream, and writes the video data stream into a screen double buffer; obtains historical frame rate data and current frame rate data, and determines predicted frame rate data by combining the historical frame rate data and the current frame rate data; and uses the predicted frame rate data to control the video data stream of the screen double buffer for display, thereby solving the problem that existing screen display devices cannot meet both user interaction and animation refresh requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 This is a flow chart of a screen control method based on YUV hard decoding provided by an embodiment of the present invention;

[0041] Figure 2 1 is a structural diagram of a screen control device based on YUV hard decoding provided by an embodiment of the present invention;

[0042] Figure 3 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0044] like Figure 1 As shown, Figure 1 1 is a flow chart of a screen control method based on YUV hard decoding provided by an embodiment of the present invention. The screen control method based on YUV hard decoding includes the following steps:

[0045] 101. Receive an external video control instruction and determine corresponding video data.

[0046] In an embodiment of the present invention, the screen control method based on YUV hard decoding can be applied to a smart device, which includes a screen display, a sensor, a microprocessor and other structures and electronic components. The screen display is used to display video images.

[0047] The above-mentioned external video control instructions can be commands received from outside the system and used to control video playback. The above-mentioned external can be user operations, networks, Bluetooth or other devices, for example, users can issue video playback commands through remote controls or touch screens.

[0048] The above-mentioned video data may be video content that needs to be played or processed according to the received control instruction.

[0049] It should be noted that, when an external video control instruction is received, the video data corresponding to the control instruction can be determined.

[0050] 102. Perform data conversion on the video data corresponding to the control instruction through a preset hardware decoding module to obtain a video data stream, and write the video data stream into a screen double buffer.

[0051] In the embodiments of the present invention, the preset hardware decoding module can be understood as a module pre-set by the system for decoding video data. The preset hardware decoding module can decode the input video data and convert it into image data that can be displayed on the screen. The preset hardware decoding module can be a GPU, DSP, etc.

[0052] The above-mentioned data conversion can be understood as a process of decoding the video data through a preset hardware decoding module and converting it into a video data stream. Specifically, the video data can be decoded into a JPEG format image through a preset hardware decoding module, and the JPEG format image can be converted into a YUV format data stream. The above-mentioned YUV format is a video data format, in which "Y" represents brightness information, "U" represents the blue chroma component, which represents the difference between blue and brightness, and "V" represents the red chroma component, which represents the difference between red and brightness. It is used to effectively reduce the amount of data by separating brightness and chroma signals, improve color encoding efficiency and be compatible with black and white display devices.

[0053] The above-mentioned video data stream can be understood as a YUV format video data stream obtained by converting the video data through a preset hardware decoding module.

[0054] The screen double buffer is used to alternately refresh data onto the screen. When data in one of the buffers is being refreshed onto the screen, the other buffer is used to receive data.

[0055] It should be noted that the above-mentioned screen double buffer can alternately refresh data to the screen, so that the display frame rate is doubled, which can avoid the screen flicker problem and improve the screen display effect.

[0056] 103. Obtain historical frame rate data and current frame rate data, and determine predicted frame rate data by combining the historical frame rate data and the current frame rate data.

[0057] In the embodiment of the present invention, the frame rate data may be understood as the number of image frames played per second.

[0058] The above-mentioned historical frame rate data can be understood as the frame rate values ​​at past time points. The historical frame rate data can be the frame rate data for a period of time before the current frame rate data, or the frame rate data for the same historical moment or time period as the current frame rate data; or it can include both the frame rate data for a period of time before the current frame rate data and the frame rate data for the same historical moment or time period as the current frame rate data.

[0059] The above current frame rate data can be understood as the current frame rate value.

[0060] The predicted frame rate data is frame rate data predicted based on historical and current frame rate data. Specifically, the predicted frame rate data can be obtained by analyzing the linear relationship between historical frame rate data and current frame rate data using a frame rate prediction model. The frame rate prediction model is trained using sample frame rate data and linear inertial annotated data corresponding to the sample frame rate data. The frame rate prediction model can be a frame rate prediction model built based on deep learning or machine learning, such as an ARIMA model.

[0061] 104. Based on the predicted frame rate data, control the video data stream of the screen double buffer for display.

[0062] In the embodiment of the present invention, the video data stream of the screen double buffer can be controlled according to the predicted frame rate data to be displayed on the screen with the predicted frame rate data, thereby realizing video playback.

[0063] It should be noted that when the predicted frame rate data is lower than 16Hz, some frames can be skipped to maintain the frame rate above 16Hz, and the video data after frame skipping is displayed on the screen; when 16Hz<predicted frame rate<24Hz, the video data stream of the screen double buffer is controlled to display the predicted frame rate data on the screen; when the predicted frame rate is greater than 24Hz, the hardware resources are released through non-blocking delay to achieve dynamic performance optimization.

[0064] In an embodiment of the present invention, an external video control instruction is received to determine the corresponding video data; the video data corresponding to the control instruction is converted by a preset hardware decoding module to obtain a video data stream, and the video data stream is written into a screen double buffer, which is used to alternately refresh data onto the screen; historical frame rate data and current frame rate data are obtained, and predicted frame rate data is determined by combining the historical frame rate data and the current frame rate data; based on the predicted frame rate data, the video data stream of the screen double buffer is controlled for display. The present invention receives an external video control instruction, determines the corresponding video data, converts the video data corresponding to the control instruction by a preset hardware decoding module to obtain a video data stream, and writes the video data stream into a screen double buffer, obtains historical frame rate data and current frame rate data, and determines predicted frame rate data by combining the historical frame rate data and the current frame rate data, and uses the predicted frame rate data to control the video data stream of the screen double buffer for display. The present invention solves the problem that existing screen display devices cannot meet both user interaction and animation refresh requirements.

[0065] It is understandable that in the specific implementation of this application, related data such as video data, frame rate data, instruction data, etc. are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data, as well as the training, deployment and calling of algorithm models, must comply with relevant laws, regulations and standards of relevant countries and regions.

[0066] Optionally, in the step of obtaining the current frame rate data, a first timestamp before the video data is decoded and a second timestamp after the video data is decoded can be obtained; based on the first timestamp and the second timestamp, the time consumption of multiple single frames of the video data is determined; based on the time consumption of multiple single frames, the current frame rate data is determined.

[0067] In the embodiment of the present invention, the above timestamp can be understood as a time representation of a time point.

[0068] The above-mentioned video data may be video data corresponding to a control instruction obtained according to a received control instruction, and the video data is video content that needs to be played or processed.

[0069] The first timestamp before the video data is decoded may be understood as the system time recorded when the preset hardware decoding model performs a decoding operation on the data video.

[0070] The second timestamp after the video data is decoded can be understood as the system time when the decoding operation of the data video is completed and recorded in the preset hardware decoding model.

[0071] The single-frame time consumption mentioned above can be understood as the difference between the second timestamp and the first timestamp, and the unit of the time consumption is second / frame.

[0072] Specifically, the average duration of multiple single frames may be calculated, and the reciprocal of the average duration may be used as the current frame rate data. The unit of the frame rate data is frame / second.

[0073] Optionally, in the step of determining the current frame rate data based on multiple single-frame time consumptions, an average value of the multiple single-frame time consumptions may be calculated; based on the average value, the reciprocal of the average value is used as the current frame rate data.

[0074] In the embodiment of the present invention, the above average value can be understood as the arithmetic mean of multiple single-frame time consumptions, which represents the sum of all single-frame time consumptions divided by the number of single-frame time consumptions.

[0075] It should be noted that, according to the formula frame rate data=1 / single-frame time consumption, the reciprocal of the average single-frame time consumption can be determined as the current frame rate data.

[0076] In one possible embodiment, for example, the time consumption of five single frames is respectively: 0.01 seconds / frame, 0.012 seconds / frame, 0.015 seconds / frame, 0.013 seconds / frame and 0.014 seconds / frame, and the average time consumption of the five single frames is calculated as (0.010+0.012+0.015+0.013+0.014) / 5=0.0136 frames / second. The reciprocal of the average value is the current frame rate data 1 / 0.0136≈73.5 frames / second. By calculating the average value, the impact of some frames taking a particularly long or short time on the overall frame rate can be reduced, making the frame rate more stable.

[0077] It should be noted that outlier filtering can be performed on multiple single-frame durations to eliminate unreasonable single-frame durations. For example, single-frame durations exceeding three times the standard deviation can be eliminated to ensure the correctness of the current frame rate data calculation.

[0078] Optionally, in the step of determining the current frame rate data based on multiple single-frame times, the weights corresponding to the multiple single-frame times can be determined; based on the weights, a weighted calculation is performed to obtain a weighted average time, and the inverse of the weighted average time is used as the current frame rate data.

[0079] In the embodiment of the present invention, the above-mentioned weight can be understood as the importance of an indicator or factor relative to a certain thing.

[0080] The weights corresponding to the time consumption of the above-mentioned multiple single frames can be determined according to the importance or complexity of different frames. For example, the weights can be assigned in a linearly increasing manner. Specifically, the weight of each frame increases linearly according to the distance from the latest frame, that is, the closer the frame is to the latest frame, the greater the weight; or the weights can be assigned in an exponentially increasing manner. Specifically, the weight of each frame increases exponentially according to the distance from the latest frame, that is, the closer the frame is to the latest frame, the greater the weight. The above-mentioned latest frame can be understood as the frame currently being displayed or about to be displayed.

[0081] Specifically, the weights corresponding to the time consumption of multiple single frames can be determined, and the weighted average of the weights corresponding to the time consumption of multiple single frames can be obtained to obtain the weighted average time consumption, and the inverse of the weighted average time consumption can be used as the current frame rate data.

[0082] Optionally, in the step of determining the predicted frame rate data by combining the historical frame rate data and the current frame rate data, the historical frame rate data and the current frame rate data can be linearly converted to obtain a linear relationship between the historical frame rate data and the current frame rate data; the linear relationship is predicted by a preset frame rate prediction model to determine the predicted frame rate data.

[0083] In the embodiment of the present invention, the above-mentioned historical frame rate data can be understood as the frame rate value at a past time point. The historical frame rate data can be the frame rate data of a period of time before the current frame rate data, or the frame rate data of the same historical moment or time period corresponding to the current frame rate data; or it can include both the frame rate data of a period of time before the current frame rate data and the frame rate data of the same historical moment or time period corresponding to the current frame rate data.

[0084] The above current frame rate data can be understood as the current frame rate value.

[0085] The above linear conversion can be understood as a process of linearly converting historical frame rate data and current frame rate data. Specifically, the historical frame rate data and the current frame rate data can be converted into a linearly related form through mathematical methods, and the linear relationship can be fitted using statistical methods such as the least squares method. The least squares method can be used to fit a linear equation y = mx + b, where y is the current frame rate data, x is the historical frame rate data, m is the slope, and b is the intercept.

[0086] In a possible embodiment, for example, the frame rate data of the past five frames (f1, f2, f3, f4, f5) and the frame rate data of the current frame (f6) can be used to perform a linear conversion on the frame rate data of the past five frames (f1, f2, f3, f4, f5) and the frame rate data of the current frame (f6), and the frame rate change between each frame (Δf = f2-f1, Δf2 = f3-f2, ..., Δf5 = f6-f5) can be calculated. The frame rate change is multiplied by the corresponding frame index to obtain a linear relationship: y = ax + b, where y is the current frame rate data, x is the historical frame rate data, a is the slope, and b is the intercept.

[0087] The linear relationship between the historical frame rate data and the current frame rate data can be understood as a relationship in which a constant proportional change exists between the historical frame rate data and the current frame rate data.

[0088] The frame rate prediction model can be one built based on deep learning or machine learning, such as an ARIMA model. This model combines autoregression (AR), differencing (I), and moving average (MA) to predict frame rate data based on a linear relationship between historical frame rate data and current frame rate data.

[0089] It should be noted that the historical frame rate data and the current frame rate data are combined and input into the ARIMA model algorithm to obtain the predicted frame rate data. Among them, the weighted average algorithm can be used to predict the frame rate, and the autoregressive integral moving average algorithm can be used, as shown below:

[0090] φ(B)Yt=θ(B)∈t

[0091] Among them, φ(B) is the autoregressive (AR) partial polynomial, which reflects the linear relationship between the current frame time and the historical frame time; Yt is the time series value, and the unit of the time consumption of the t-th frame is seconds; θ(B) is the moving average (MA) partial polynomial, which reflects the impact of the historical prediction error on the current frame; ∈t is the white noise error term, which represents unpredictable instantaneous interference such as user interaction and GPU scheduling; B is the lag operator.

[0092] When p > 0.05, residual interference exists. The p-value can be understood as a probability value for user testing, where residual interference primarily arises from interference during user interaction. The ARIMA model algorithm predicts the linear relationship between historical frame rate data and current frame rate data by first performing time series conversion, then performing d-order differencing, and finally fitting the ARIMA model to obtain the predicted frame rate data. The advantage of this ARIMA model algorithm is that it does not require parameter adjustment to adapt to new data; simply inputting real data yields direct results.

[0093] The above-mentioned preset frame rate prediction model is obtained by training the pre-trained frame rate prediction model through the training data set. After the training is completed, the preset frame rate prediction model is obtained. The above-mentioned training data set includes sample frame rate data and linear relationship annotation data corresponding to the sample frame rate data. The above-mentioned sample frame rate data includes sample historical frame rate data and sample current frame rate data. The above-mentioned sample historical frame rate data can be the frame rate data of a period of time before the sample current frame rate data, or the frame rate data of the same historical moment or the same time period corresponding to the sample current frame rate data; it can also include both the frame rate data of a period of time before the sample current frame rate data and the frame rate data of the same historical moment or the same time period corresponding to the historical current frame rate data.

[0094] The above linear relationship can be understood as the linear relationship between the sample historical frame rate data and the sample frame rate data, and can be represented by fitting the linear relationship y=ax+b using the least squares method, where y is the current frame rate data, x is the historical frame rate data, a is the slope, and b is the intercept.

[0095] The above-mentioned labeled data can be understood as the process of adding labels, annotations or tags to the sample frame rate data, and converting the sample frame rate data into machine-recognizable structured information, so that the model can recognize and process the sample frame rate data and labeled data.

[0096] The pre-trained frame rate prediction model may be a frame rate prediction model based on deep learning or machine learning, such as an ARIMA model.

[0097] This training can be supervised training, which uses a set of data with known labels to train a model. By optimizing model parameters, the model can predict the labels of new data or make decisions based on the characteristics of existing data. During training, the model parameters can be adjusted using a minimum loss function. Once training is complete, a preset frame rate prediction model is obtained. This loss function measures the difference between the model's prediction and the actual result and can be a mean square error loss function, a cross-entropy loss function, or other such function. The parameter adjustment can be performed by updating the model parameters using an optimization algorithm to minimize the loss function, thereby improving model performance. This optimization algorithm can be gradient descent, stochastic gradient descent, or other such algorithms.

[0098] The above-mentioned preset frame rate prediction model can combine historical frame rate data and current frame rate data to predict predicted frame rate data.

[0099] Optionally, in the step of controlling the video data stream of the screen buffer for display based on the predicted frame rate data, when the predicted frame rate is lower than a first frame rate threshold, a target discarded frame is obtained, the target discarded frame is eliminated, and the video data stream from which the target discarded frame is eliminated is displayed; when the predicted frame rate is greater than the first frame rate threshold and less than the second frame rate threshold, the video data stream of the screen buffer is controlled to be displayed according to the predicted frame rate; when the predicted frame rate is greater than the second frame rate threshold, the hardware resources are released through non-blocking delay.

[0100] In the embodiment of the present invention, the first frame rate threshold is a frame rate threshold preset by the system, and specifically may be 16 Hz.

[0101] The target discard frames may be frames that need to be discarded. For example, in a 100-frame video, if the instantaneous frame rates of the 50th, 60th, 70th, and 80th frames are lower than the first frame rate threshold, the 50th, 60th, 70th, and 80th frames may be marked as "target discard frames."

[0102] The above-mentioned elimination may be a process of removing the target discarded frames from the video data stream. The above-mentioned second frame rate threshold is a frame rate threshold preset by the system, and may be specifically 24 Hz.

[0103] The above non-blocking delay can be understood as allowing the program to continue executing other operations while waiting for a specific event to occur.

[0104] In one possible embodiment, when the predicted frame rate data is less than 16 Hz, the target discarded frame is obtained, the target discarded frame is eliminated, and the video data stream after eliminating the target discarded frame is displayed on the screen; when 16 Hz < predicted frame rate data < 24 Hz, the video data stream of the screen double buffer is controlled to be displayed on the screen with the predicted frame rate data; when the predicted frame rate is greater than 24 Hz, the hardware resources are released through non-blocking delay to achieve dynamic performance optimization.

[0105] It should be noted that the present invention uses a screen double buffer. When data in one buffer is being refreshed to the screen, the other buffer is used to receive data, which can avoid the occurrence of flickering and can adjust the length of the non-blocking delay as needed.

[0106] Optionally, when the predicted frame rate is lower than a first frame rate threshold, a target discarded frame is obtained, the target discarded frame is eliminated, and the video data stream from which the target discarded frame is eliminated is displayed. When the predicted frame rate is lower than the first frame rate threshold, the number of frames corresponding to the predicted frame rate and the number of frames corresponding to the current frame rate are calculated to determine the number of frame skips; inter-frame calculations are performed on the video data stream to determine the key value of each frame; and the key values ​​are arranged in ascending order to obtain an ascending arrangement result; based on the number of frame skips and the ascending arrangement result, the target discarded frame is determined, the target discarded frame is eliminated, and the video data stream from which the target discarded frame is eliminated is displayed.

[0107] In an embodiment of the present invention, the number of frames corresponding to the above-mentioned predicted frame rate can be understood as the number of frames rendered or displayed corresponding to the video data stream based on the predicted frame rate. For example, if the predicted frame rate is 30 frames / second, the number of frames rendered or displayed corresponding to the video stream is 30 frames of images, etc.

[0108] The number of frames corresponding to the current frame rate can be understood as the number of video frames currently displayed by the video data stream according to the current frame rate. For example, if the current frame rate is 60 frames per second, the number of frames rendered or displayed by the video data stream in one second is 60 frames.

[0109] The above-mentioned number of skipped frames can be understood as the number of picture frames that need to be skipped during the playback of the video data stream.

[0110] The above-mentioned inter-frame calculation can be understood as a calculation process of comparing the differences between adjacent frames in a video data stream.

[0111] The key value of each frame can be understood as the most important information or features in the frame image, such as the edge and texture of the image. The key value can be extracted by calculating features such as the gradient and color histogram of each frame.

[0112] The above ascending order arrangement can be understood as arranging the key values ​​of each frame in ascending order.

[0113] The above ascending order result may be a result obtained by arranging the key values ​​of each frame in ascending order.

[0114] Furthermore, the key value of the smallest number of skipped frames in the ascending order result may be determined as the target discarded frame, and the target discarded frame may be removed, and the video data stream after the target discarded frame is removed may be displayed.

[0115] In a possible embodiment, in a video sequence, each frame has a timestamp. When switching from one frame to the next, the key value between the two frames can be calculated, and the number of frames with the lowest key value and the number of frames skipped is determined as the target discarded frame, and the target discarded frame is discarded. Specifically, the frame with the lowest key value indicates that the change between the two frames is very small and can be skipped without affecting the video quality; if the change between the two frames with the lowest key value is also very small, the two frames with the lowest key value can be discarded, and so on. The present invention can dynamically adjust the difference between the number of frames corresponding to the predicted frame rate and the number of frames corresponding to the current frame rate. The difference between the number of frames corresponding to the predicted frame rate and the number of frames corresponding to the current frame rate is proportional to the number of frame skips. The larger the difference, the larger the number of frame skips.

[0116] In a possible embodiment, the present invention can also be based on the calculation between the predicted frame and the video data stream to determine the key value of each frame. The video data stream can also be segmented according to the average duration, a predicted frame is determined in each segment, and the predicted frame is calculated with the frames in the video data stream of each segment to determine the key value of each frame. It is understood that after obtaining the key value of each frame, the key values ​​can be sorted to obtain the sorting results, and frames with low key values ​​can be eliminated. Frames with low key values ​​indicate that the changes between the two frames are very small and can be skipped without affecting the video quality.

[0117] It should be noted that before skipping frames, an inter-frame calculation can be performed on the predicted frame and the current frame to obtain a key value for each frame. The frame with the lowest key value can be discarded to avoid affecting the output of key video information. Alternatively, the two frames with the lowest key values ​​can be discarded. Key frames can be calculated by calculating the difference between consecutive frames. The smaller the difference between consecutive frames, the lower the key value, and the larger the difference between consecutive frames, the higher the key value. It should be noted that the difference between consecutive frames can be calculated by calculating pixel difference, histogram difference, structural similarity, etc.

[0118] like Figure 2 As shown, an embodiment of the present invention provides a screen control device based on YUV hard decoding, and the screen control device based on YUV hard decoding includes:

[0119] A first determining module 201 is configured to receive an external video control instruction and determine corresponding video data;

[0120] The conversion module 202 is configured to convert the video data corresponding to the control instruction through a preset hardware decoding module to obtain a video data stream, and write the video data stream into a screen double buffer, wherein the screen double buffer is configured to alternately refresh the data onto the screen;

[0121] The second determination module 203 is configured to obtain historical frame rate data and current frame rate data, and determine predicted frame rate data by combining the historical frame rate data and the current frame rate data;

[0122] The control module 204 is configured to control the video data stream in the screen double buffer to be displayed based on the predicted frame rate data.

[0123] Optionally, the second determining module 203 includes:

[0124] An acquisition submodule, configured to acquire a first timestamp before decoding of the video data and a second timestamp after decoding of the video data;

[0125] A first determining submodule, configured to determine a time consumption of multiple single frames of the video data based on the first timestamp and the second timestamp;

[0126] The second determining submodule is configured to determine current frame rate data based on the time consumption of the plurality of single frames.

[0127] Optionally, the second determining submodule includes:

[0128] A calculation unit, configured to calculate an average of the time consumptions of a plurality of the single frames;

[0129] The first determining unit is configured to use the reciprocal of the average value as current frame rate data based on the average value.

[0130] Optionally, the second determining submodule includes:

[0131] A second determining unit is used to determine weights corresponding to the time consumptions of the plurality of single frames;

[0132] The third determining unit is configured to perform a weighted calculation based on the weight to obtain a weighted average time consumption, and then use the reciprocal of the weighted average time consumption as the current frame rate data.

[0133] Optionally, the second determining module 203 includes:

[0134] a conversion submodule, configured to perform linear conversion on the historical frame rate data and the current frame rate data to obtain a linear relationship between the historical frame rate data and the current frame rate data;

[0135] The first processing submodule is configured to perform prediction processing on the linear relationship using a preset frame rate prediction model to determine predicted frame rate data.

[0136] Optionally, the control module 204 includes:

[0137] A first display submodule is configured to obtain a target discarded frame when the predicted frame rate is lower than a first frame rate threshold, remove the target discarded frame, and display the video data stream from which the target discarded frame has been removed;

[0138] a second display submodule, configured to control the video data stream in the screen buffer to be displayed according to the predicted frame rate when the predicted frame rate is greater than the first frame rate threshold and less than a second frame rate threshold;

[0139] The second processing submodule is configured to release hardware resources through non-blocking delay when the predicted frame rate is greater than the second frame rate threshold.

[0140] Optionally, the first display submodule includes:

[0141] a first calculation unit, configured to calculate the number of frames corresponding to the predicted frame rate and the number of frames corresponding to the current frame rate to determine a number of frame skips when the predicted frame rate is lower than a first frame rate threshold;

[0142] A second calculation unit is used to perform inter-frame calculation on the video data stream to determine a key value of each frame;

[0143] The sorting unit arranges the key values ​​in ascending order to obtain an ascending order result;

[0144] The display unit is configured to determine the target discarded frame based on the number of skipped frames and the ascending order arrangement result, remove the target discarded frame, and display the video data stream from which the target discarded frame is removed.

[0145] like Figure 3 As shown, an embodiment of the present invention further provides an electronic device, including a processor, which can execute any of the above-mentioned product testing methods based on malicious master-slave architecture communication sub-systems.

[0146] Specifically, the system includes a processor 301, a memory 302, and a computer program stored in the memory 302 and capable of running on the processor 301 for executing a screen control method based on YUV hard decoding, wherein:

[0147] The processor 301 runs the computer program of the screen control method based on YUV hard decoding stored in the memory 302 and performs the following steps:

[0148] Receive external video control instructions and determine corresponding video data;

[0149] Performing data conversion on the video data corresponding to the control instruction through a preset hardware decoding module to obtain a video data stream, and writing the video data stream into a screen double buffer, wherein the screen double buffer is used to alternately refresh the data onto the screen;

[0150] Obtaining historical frame rate data and current frame rate data, and determining predicted frame rate data by combining the historical frame rate data and the current frame rate data;

[0151] Based on the predicted frame rate data, the video data stream of the screen double buffer is controlled to be displayed.

[0152] Optionally, the obtaining of current frame rate data performed by the processor 301 includes:

[0153] Obtaining a first timestamp before decoding the video data and a second timestamp after decoding the video data;

[0154] Determining, based on the first timestamp and the second timestamp, the duration of multiple single frames of the video data;

[0155] Based on the multiple single frame times, current frame rate data is determined.

[0156] Optionally, the processor 301 determines the current frame rate data based on the multiple single frame times, including:

[0157] Calculating an average of the time consumption of multiple single frames;

[0158] Based on the average value, the reciprocal of the average value is used as the current frame rate data.

[0159] Optionally, the processor 301 determines the current frame rate data based on the multiple single frame times, including:

[0160] Determining weights corresponding to the time consumption of multiple single frames;

[0161] Based on the weights, a weighted average time consumption is obtained by weighted calculation, and the inverse of the weighted average time consumption is used as the current frame rate data.

[0162] Optionally, the determining of predicted frame rate data based on the historical frame rate data and the current frame rate data performed by the processor 301 includes:

[0163] Performing a linear conversion on the historical frame rate data and the current frame rate data to obtain a linear relationship between the historical frame rate data and the current frame rate data;

[0164] The linear relationship is predicted using a preset frame rate prediction model to determine predicted frame rate data.

[0165] Optionally, the controlling of the video data stream of the screen buffer for display based on the predicted frame rate data performed by the processor 301 includes:

[0166] When the predicted frame rate is lower than a first frame rate threshold, a target discarded frame is obtained, the target discarded frame is eliminated, and the video data stream from which the target discarded frame is eliminated is displayed;

[0167] When the predicted frame rate is greater than the first frame rate threshold and less than a second frame rate threshold, controlling the video data stream of the screen buffer to be displayed according to the predicted frame rate;

[0168] When the predicted frame rate is greater than the second frame rate threshold, hardware resources are released through non-blocking delay.

[0169] Optionally, the processor 301 performs the following steps: obtaining a target discarded frame when the predicted frame rate is lower than a first frame rate threshold, removing the target discarded frame, and displaying the video data stream from which the target discarded frame is removed, including:

[0170] When the predicted frame rate is lower than a first frame rate threshold, calculating the number of frames corresponding to the predicted frame rate and the number of frames corresponding to the current frame rate to determine a number of frame skips;

[0171] Performing inter-frame calculation on the video data stream to determine a key value for each frame;

[0172] and arranging the key values ​​in ascending order to obtain an ascending order result;

[0173] Based on the number of skipped frames and the ascending order result, the target discarded frames are determined, the target discarded frames are eliminated, and the video data stream from which the target discarded frames are eliminated is displayed.

[0174] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the screen control method based on YUV hard decoding provided in the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0175] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0176] The above disclosure is merely a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A screen control method based on YUV hard decoding, characterized in that: The method comprises the following steps: Receive external video control instructions and determine corresponding video data; Performing data conversion on the video data corresponding to the control instruction through a preset hardware decoding module to obtain a video data stream, and writing the video data stream into a screen double buffer, wherein the screen double buffer is used to alternately refresh the data onto the screen; Obtaining historical frame rate data and current frame rate data, and determining predicted frame rate data by combining the historical frame rate data and the current frame rate data; Based on the predicted frame rate data, the video data stream of the screen double buffer is controlled to be displayed.

2. The screen control method based on YUV hard decoding according to claim 1, characterized in that: The obtaining of current frame rate data includes: Obtaining a first timestamp before decoding the video data and a second timestamp after decoding the video data; Determining, based on the first timestamp and the second timestamp, the duration of multiple single frames of the video data; Based on the multiple single frame times, current frame rate data is determined.

3. The screen control method based on YUV hard decoding according to claim 2, characterized in that: The determining of current frame rate data based on the multiple single frame times includes: Calculating an average of the time consumption of multiple single frames; Based on the average value, the reciprocal of the average value is used as the current frame rate data.

4. The screen control method based on YUV hard decoding according to claim 2, characterized in that: The determining of current frame rate data based on the multiple single frame times includes: Determining weights corresponding to the time consumption of multiple single frames; Based on the weights, a weighted average time consumption is obtained by weighted calculation, and the inverse of the weighted average time consumption is used as the current frame rate data.

5. The screen control method based on YUV hard decoding according to claim 4, characterized in that: The determining of predicted frame rate data by combining the historical frame rate data and the current frame rate data includes: Performing a linear conversion on the historical frame rate data and the current frame rate data to obtain a linear relationship between the historical frame rate data and the current frame rate data; The linear relationship is predicted using a preset frame rate prediction model to determine predicted frame rate data.

6. The screen control method based on YUV hard decoding according to claim 1, characterized in that: The controlling the video data stream of the screen buffer to display based on the predicted frame rate data includes: When the predicted frame rate is lower than a first frame rate threshold, a target discarded frame is obtained, the target discarded frame is eliminated, and the video data stream from which the target discarded frame is eliminated is displayed; When the predicted frame rate is greater than the first frame rate threshold and less than a second frame rate threshold, controlling the video data stream of the screen buffer to be displayed according to the predicted frame rate; When the predicted frame rate is greater than the second frame rate threshold, hardware resources are released through non-blocking delay.

7. The screen control method based on YUV hard decoding according to claim 6, characterized in that: When the predicted frame rate is lower than the first frame rate threshold, a target discarded frame is obtained, the target discarded frame is eliminated, and the video data stream from which the target discarded frame is eliminated is displayed, comprising: When the predicted frame rate is lower than a first frame rate threshold, calculating the number of frames corresponding to the predicted frame rate and the number of frames corresponding to the current frame rate to determine a number of frame skips; Performing inter-frame calculation on the video data stream to determine a key value for each frame; and arranging the key values ​​in ascending order to obtain an ascending order result; Based on the number of skipped frames and the ascending order result, the target discarded frames are determined, the target discarded frames are eliminated, and the video data stream from which the target discarded frames are eliminated is displayed.

8. A screen control device based on YUV hard decoding, characterized in that: The screen control device based on YUV hard decoding includes: A first determining module is used to receive an external video control instruction and determine corresponding video data; a conversion module, configured to convert the video data corresponding to the control instruction through a preset hardware decoding module to obtain a video data stream, and write the video data stream into a screen double buffer, wherein the screen double buffer is configured to alternately refresh the data onto the screen; A second determination module is configured to obtain historical frame rate data and current frame rate data, and determine predicted frame rate data by combining the historical frame rate data and the current frame rate data; A control module is used to control the video data stream of the screen double buffer to display based on the predicted frame rate data.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the screen control method based on YUV hard decoding are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the screen control method based on YUV hard decoding are implemented as described in any one of claims 1 to 7.

Images