A frame queue management method and system based on image quality monitoring

Abstract

The embodiment of the present application discloses a frame queue management method and system based on image quality monitoring. The technical solution provided by the embodiment of the present application records the time difference between each video frame sent into the frame queue and the previous video frame, and records the image quality monitoring calculation time of each video frame in the frame queue. Then, the average time difference is determined based on each input time difference, and the average calculation time is determined based on each image quality monitoring calculation time. Afterwards, the video frame input frequency of the frame queue is adjusted based on the comparison result of the average time difference and the average calculation time. The above-mentioned technical means are adopted to adjust the video frame input frequency of the frame queue to avoid the VMAF service thread occupying too many computing resources and affecting the source stream video transcoding efficiency, thereby ensuring the normal transcoding operation of the encoding thread. Without affecting the user's viewing experience, image quality monitoring calculations are performed as much as possible to optimize the image quality monitoring calculation effect.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video encoding technology, and in particular to a frame queue management method and system based on image quality monitoring. Background Art

[0002] At present, in the video transcoding system, the source stream video uploaded by the anchor needs to be transcoded to obtain a variety of transcoded videos with different bit rates and resolutions to meet the viewing needs in different scenarios. In order to be able to promptly detect abnormalities in the image quality of the transcoded video, the image quality of the transcoded video will be monitored to optimize the image quality of the transcoded video and improve the user experience. The current main transcoded video quality monitoring solution is VMAF (Video Multimethod Assessment Fusion), which uses a fusion of multiple evaluation methods to better adapt to the user's subjective feelings. When monitoring image quality based on VMAF, the real-time transcoded video frames are placed in the video frame queue, and the VMAF service thread extracts the video frames from the video frame queue to calculate the VMAF value of the transcoded video, and the corresponding VMAF value is used to accurately characterize the image quality of the transcoded video.

[0003] However, investing too much computing resources in the image quality monitoring process will affect the transcoding task of the main thread, resulting in a heavy transcoding task load and low source stream video transcoding efficiency. Summary of the invention

[0004] The embodiments of the present application provide a frame queue management method and system based on image quality monitoring, which can adaptively adjust the video frame queue for image quality monitoring calculation, avoid the VMAF service thread occupying too many computing resources and affecting the source stream video transcoding efficiency, and solve the technical problem that the existing image quality monitoring task causes excessive transcoding task load.

[0005] In a first aspect, an embodiment of the present application provides a frame queue management method based on image quality monitoring, comprising:

[0006] Record the time difference between each video frame sent into the frame queue and the previous video frame, and record the image quality monitoring calculation time of each video frame in the frame queue;

[0007] Determine an average time difference based on each input time difference, and determine an average calculation time based on each image quality monitoring calculation time;

[0008] The frequency of sending video frames into the frame queue is adjusted based on the comparison result of the average time difference and the average calculation time.

[0009] In a second aspect, an embodiment of the present application provides a frame queue management system based on image quality monitoring, including:

[0010] A recording module configured to record the time difference between each video frame sent into the frame queue and the previous video frame, and to record the image quality monitoring calculation time of each video frame in the frame queue;

[0011] A calculation module configured to determine an average time difference based on each input time difference, and determine an average calculation time based on each image quality monitoring calculation time;

[0012] The adjustment module is configured to adjust the frequency of video frame input into the frame queue based on the comparison result of the average time difference and the average calculation time.

[0013] In a third aspect, an embodiment of the present application provides a frame queue management device based on image quality monitoring, including:

[0014] memory and one or more processors;

[0015] The memory is configured to store one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the frame queue management method based on image quality monitoring as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are configured to execute the frame queue management method based on image quality monitoring as described in the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the frame queue management method based on image quality monitoring as described in the first aspect.

[0019] The embodiment of the present application records the time difference between each video frame sent into the frame queue and the previous video frame, and records the picture quality monitoring calculation time of each video frame in the frame queue. Then, the average time difference is determined based on each time difference, and the average calculation time is determined based on each picture quality monitoring calculation time. Afterwards, the video frame sending frequency of the frame queue is adjusted based on the comparison result of the average time difference and the average calculation time. Using the above technical means, the occupation of computing resources by the VMAF service thread is evaluated according to the comparison result of the picture quality monitoring calculation time and the video frame sending time of the frame queue. By adjusting the video frame sending frequency of the frame queue, the VMAF service thread is prevented from occupying too many computing resources and affecting the source stream video transcoding efficiency, thereby ensuring the normal transcoding operation of the encoding thread. Without affecting the user's viewing experience, the picture quality monitoring calculation is performed as much as possible to optimize the picture quality monitoring calculation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of a frame queue management method based on image quality monitoring provided in an embodiment of the present application;

[0021] Figure 2 This is a schematic diagram of adjusting the frame sending frequency of a frame queue according to an embodiment of the present application;

[0022] Figure 3 This is a flow chart of adjusting the frame sending frequency of a frame queue according to an embodiment of the present application;

[0023] Figure 4 It is a structural diagram of a frame queue management system based on image quality monitoring provided by an embodiment of the present application;

[0024] Figure 5 It is a structural diagram of a frame queue management device based on image quality monitoring provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0026] The present application provides a frame queue management method based on image quality monitoring, which aims to evaluate the occupation of computing resources by the VMAF service thread according to the comparison results of the image quality monitoring calculation time and the video frame input time of the frame queue, and then adjust the video frame input frequency of the frame queue to avoid the VMAF service thread from occupying too many computing resources and affecting the source stream video transcoding efficiency by determining that the current VMAF service thread will occupy too many computing resources for image quality monitoring calculation, so as to ensure the normal transcoding operation of the encoding thread. Compared with the traditional image quality monitoring calculation scheme, when performing image quality monitoring calculation, it generally directly puts the transcoded video frame directly into the frame queue, so that the VMAF service thread directly extracts the video frame from the video frame queue to calculate the VMAF value of the transcoded video. When the frame queue length is large, the VMAF service thread will use more computing resources for image quality monitoring calculation, which will lead to too many computing resources being invested in the image quality monitoring process, affecting the transcoding task of the transcoding thread, resulting in a large transcoding task load and low source stream video transcoding efficiency. Based on this, a frame queue management method based on image quality monitoring is provided in an embodiment of the present application to solve the technical problem that the existing image quality monitoring task causes excessive load on the transcoding task.

[0027] Example:

[0028] Figure 1 A flow chart of a frame queue management method based on image quality monitoring provided in an embodiment of the present application is given. The frame queue management method based on image quality monitoring provided in this embodiment can be executed by a frame queue management device based on image quality monitoring. The frame queue management device based on image quality monitoring can be implemented in software and / or hardware. The frame queue management device based on image quality monitoring can be composed of two or more physical entities, or can be composed of one physical entity. Generally speaking, the frame queue management device based on image quality monitoring can be a computing device such as a transcoding system backend, an encoding server, or a computer.

[0029] The following description is made by taking the frame queue management device based on image quality monitoring as an example of the subject of the frame queue management method based on image quality monitoring. Figure 1 , the frame queue management method based on image quality monitoring specifically includes:

[0030] S110, recording the time difference between each video frame sent into the frame queue and the previous video frame, and recording the image quality monitoring calculation time of each video frame in the frame queue.

[0031] When managing the frame queue, the embodiment of the present application compares the average time difference and average calculation time of the video frames sent into the frame queue, and judges whether the current image quality monitoring calculation speed meets the frequency of putting video frames into the video frame queue according to the comparison result. If so, it means that the current image quality monitoring calculation does not need to invest more computing resources; otherwise, more computing resources are needed. However, investing too many computing resources will affect the execution of the transcoding task of the system main thread (i.e., the transcoding thread), thereby affecting the efficiency of transcoded video stream distribution. Therefore, it is necessary to adaptively adjust the frequency of sending video frames into the frame queue to avoid excessive occupation of computing resources by the VMAF service thread.

[0032] In the video transcoding scenario, for the video source stream uploaded by the anchor in real time, after the encoding thread transcodes the transcoded video stream to obtain the transcoded video stream, the transcoded video stream is placed into the frame queue frame by frame in the order of the video frames, and then the system's VMAF service thread extracts the video frames in the frame queue for VMAF calculation.

[0033] Specifically, Figure 2 As shown, a schematic diagram of adjusting the frame queue frame sending frequency of an embodiment of the present application is provided. For the video frames that the encoding thread has completed encoding, they will be placed in the frame queue frame by frame, and then the VMAF service thread will extract them from the frame queue for VMAF calculation, and then return the VMAF calculation result (i.e., the image quality monitoring result) to the encoding thread. In this process, the VMAF service thread adaptively adjusts the frame sending frequency of the frame queue to perform image quality monitoring calculation as much as possible without affecting the user's viewing experience, thereby optimizing the image quality monitoring calculation effect.

[0034] The encoding thread cannot block the normal operation of the encoding thread when introducing video frames for VMAF calculation. It is used to submit the video frames to be calculated to the VMAF service thread, and then obtain the calculation results from the VMAF service thread to know the image quality of the transcoded video stream based on the calculation results. The VMAF service thread is a separate service thread, which is used to obtain the video frames to be calculated from the frame queue for VMAF calculation and submit the calculation results to the encoding thread. If there is no data in the frame queue, the VMAF service thread is suspended until a video frame that can be calculated arrives.

[0035] For the frame queue to be calculated by VMAF, the frequency of sending video frames needs to be adaptively adjusted to ensure that the frame queue is always maintained in a stable state and to avoid unlimited growth of the frame queue (for example, insufficient CPU computing resources cause the calculation frequency of the VMAF service thread to be lower than the frequency of sending video frames to the frame queue, which causes the growth of the frame queue).

[0036] The VMAF service thread records the time difference between each video frame sent into the frame queue and the previous video frame sent into the frame queue in real time, that is, the time difference between two adjacent video frames in the frame queue, and records the time difference between the i-th two adjacent video frames sent into the frame queue as I i On the other hand, it is necessary to record the image quality monitoring calculation time of each video frame in the frame queue, denoted as T i By recording the input time difference and the image quality monitoring calculation time, the frame queue frame sending frequency can be adjusted later.

[0037] S120, determining an average time difference based on each input time difference, and determining an average calculation time based on each image quality monitoring calculation time.

[0038] Furthermore, based on the real-time recorded input time difference and image quality monitoring calculation time, the embodiment of the present application calculates the average time difference of video frame input of the frame queue through each input time difference, and calculates the average calculation time of each video frame of the frame queue through each image quality monitoring calculation time. By comparing the average time difference and the average calculation time, the growth of the frame queue can be determined, and the frame queue input frequency can be adaptively adjusted.

[0039] Optionally, in order to avoid frequently adjusting the video frame input frequency of the frame queue according to the comparison results of the average time difference and the average calculation time, the embodiment of the present application sets a specified number, and when the number of video frames input into the frame queue is equal to the specified number, the average time difference is calculated based on each input time difference and the specified number, and the average calculation time is calculated based on each image quality monitoring calculation time and the specified number.

[0040] For example, after the anchor starts broadcasting, the video source stream generated in real time will be reported to the system for transcoding to obtain the corresponding transcoded video stream, and then the transcoded video stream will be put into the frame queue frame by frame for VMAF calculation. At this time, the statistics start from the first video frame put into the frame queue. When the number of video frames sent into the frame queue is equal to the specified number, the calculation of the average time difference and the average calculation time is triggered, and then the frequency of sending video frames into the frame queue is adjusted according to the calculation results.

[0041] Optionally, the embodiment of the present application sets the specified number to 10. When i=10, the average time difference and the average calculation time are respectively counted, and then the two are compared to adjust the frequency of video frame input to the frame queue.

[0042] Among them, when i=10, the calculation formula of the average time difference is:

[0043]

[0044] in, represents the average time difference calculated when i=10, I k It represents the input time difference calculated between every two adjacent video frames in the frame queue when i=10;

[0045] The average calculation time is calculated as:

[0046]

[0047] Among them, T 10 represents the average calculation time when i=10, T k It represents the image quality monitoring calculation time of each video frame in the frame queue when i=10.

[0048] Furthermore, when the number of video frames sent into the frame queue is greater than the specified number, the average time difference is calculated based on the average time difference calculated for the previous video frame and the input time difference calculated for the current video frame using a preset first influencing parameter, and the average calculation time is calculated based on the average calculation time calculated for the previous video frame and the image quality monitoring calculation time for the current video frame using a preset second influencing parameter.

[0049] Among them, when the number of video frames sent to the frame queue is greater than the specified number, the calculation formula of the average time difference is:

[0050]

[0051] in, represents the average time difference calculated corresponding to the current video frame, θ represents the first influencing parameter, represents the average time difference calculated from the previous video frame corresponding to the current video frame, I i+1 Indicates the input time difference calculated corresponding to the current video frame;

[0052] The average calculation time is calculated as:

[0053]

[0054] in, represents the average calculation time corresponding to the current video frame, φ represents the second influencing parameter, represents the average computation time of the previous video frame corresponding to the current video frame, T i+1 Indicates the quality monitoring calculation time of the current video frame.

[0055] According to actual tests, when the number of video frames sent to the frame queue is greater than the specified number, the average time difference corresponding to the current frame calculation is affected by the average time difference of the previous video frame calculation and the time difference between the current frame and the previous video frame. The average calculation time corresponding to the current frame calculation is affected by the average calculation time of the previous video frame calculation and the image quality monitoring time of the current frame. Based on this, the embodiment of the present application sets the corresponding first influencing parameter and the second influencing parameter, and constructs the above formulas (3) and (4) to accurately calculate the corresponding average time difference and average calculation time.

[0056] Optionally, the first influencing parameter θ and the second influencing parameter φ are set to 0.4. According to actual measurement requirements, the first influencing parameter θ and the second influencing parameter φ can be adaptively selected to take appropriate values, and no fixed restrictions are imposed herein.

[0057] S130 , adjusting the frequency of sending video frames into the frame queue based on the comparison result of the average time difference and the average calculation time.

[0058] Finally, based on the average time difference and average calculation time calculated above, by comparing the two, we can determine the comparison between the calculation frequency of the VMAF service thread and the frequency of the video frames sent into the frame queue, and then determine whether the image quality monitoring calculation of the current VMAF service thread will occupy the computing resources of the encoding thread and affect the normal operation of the encoding thread.

[0059] Optionally, in order to avoid frequent adjustment of the frame queue frame feeding frequency, the embodiment of the present application adjusts the video frame feeding frequency of the frame queue according to the comparison result of the average time difference and the average calculation time when the number of video frames fed into the frame queue is equal to the specified number. Afterwards, when the number of video frames fed into the frame queue is greater than the specified number, further select a corresponding time according to the number of video frames fed into the frame queue, and adjust the video frame feeding frequency according to the comparison result of the average time difference and the average calculation time.

[0060] Specifically, refer to Figure 3 The frame queue sending frame frequency adjustment process of the embodiment of the present application includes:

[0061] S1301, setting the product of the proportional coefficient and the average time difference as the time difference threshold;

[0062] S1302. When the average calculation time is greater than the time difference threshold, the frequency of sending video frames to the frame queue is lowered; when the average calculation time is less than or equal to the time difference threshold, the frequency of sending video frames to the frame queue is maintained unchanged, and the time difference threshold is less than the average time difference.

[0063] The time difference threshold limits the minimum value of the average calculation time. When the average calculation time is less than or equal to the time difference threshold, it indicates that the current system's computing resources meet the image quality monitoring calculation of the VMAF service thread and will not affect the operation of the encoding thread. At this time, the frequency of video frame input to the frame queue remains unchanged. When the average calculation time is greater than the time difference threshold, it indicates that the computing resources required for the image quality monitoring calculation of the current VMAF service thread exceed the standard and may occupy the computing resources of the encoding thread. At this time, the frequency of video frame input to the frame queue needs to be lowered to prevent the former VMAF service thread from excessively occupying system computing resources.

[0064] Furthermore, in the embodiment of the present application, when the average calculation time is greater than the time difference threshold and less than or equal to the average time difference, the video frame input frequency of the frame queue is lowered to a first set ratio of the initial input frequency; when the average calculation time is greater than the average time difference, the video frame input frequency of the frame queue is lowered to a second set ratio of the initial input frequency, and the first set ratio is greater than the second set ratio.

[0065] Exemplarily, when the number of video frames sent into the frame queue is equal to the specified number, the time difference threshold is determined to be α is the set ratio coefficient, and the value of this embodiment is 0.3. According to actual tests, the set ratio can be adaptively set. At this point, the computer load is considered sufficient to calculate VMAF, so no protective measures are needed. At this time, calculating VMAF will bring a greater load to the overall transcoding of the system encoding thread. At this time, the frequency of the video frame sent into the control frame queue is the first set ratio of the initial input frequency, and the first set ratio is 0.5. This means that the load of calculating VMAF at this time will seriously affect the operation of the system encoding thread. At this time, the frequency of the frame queue feeding the video frames is controlled to be the second set ratio of the initial feeding frequency, and the second set ratio is 0.1. According to actual adjustment needs, the first set ratio and the second set ratio can be adaptively set and are not fixed here.

[0066] Furthermore, when the number of video frames sent into the frame queue is an integer multiple of the target number, the frequency of sending the video frames into the frame queue is adjusted based on the comparison result of the average time difference and the average calculation time.

[0067] For example, when the number of video frames sent to the frame queue i>10, the average update time difference is At the same time, each time a frame of VMAF is calculated, the average calculation time is updated as

[0068]

[0069] Then, when the number of video frames sent into the frame queue is an integer multiple of the target number, the frequency of sending the video frames into the frame queue is adjusted by referring to the comparison method of the average time difference and the average calculation time. Optionally, the target number is 100, that is, whenever the number of video frames sent into the frame queue i%100==0 is satisfied, the comparison process of the average time difference and the average calculation time is repeated, and then the frequency of sending the video frames into the frame queue is adjusted according to the comparison result, so as to realize the complete process of adaptively adjusting the frequency of sending the video frames into the frame queue of the present application.

[0070] Optionally, in one embodiment, for the frame queue, the queue length of the frame queue is recorded in real time, and when the queue length exceeds a set length threshold, the video frame is stopped from being sent to the frame queue. It is understandable that the unlimited growth of the length of the frame queue will cause the system computing resources to be over-occupied by the VMAF service thread, seriously affecting the operation of the system encoding thread. Based on this, in the embodiment of the present application, during the transcoding process, as long as the length of the frame queue is greater than the length threshold at any time, the frame is stopped from being sent to the frame queue, so as to further ensure the normal transcoding operation of the encoding thread.

[0071] In the above, the time difference between each video frame sent into the frame queue and the previous video frame is recorded, and the image quality monitoring calculation time of each video frame in the frame queue is recorded. Then, the average time difference is determined based on each input time difference, and the average calculation time is determined based on each image quality monitoring calculation time. After that, the video frame input frequency of the frame queue is adjusted based on the comparison result of the average time difference and the average calculation time. Using the above technical means, the occupation of computing resources by the VMAF service thread is evaluated according to the comparison result of the image quality monitoring calculation time and the video frame input time of the frame queue. By adjusting the video frame input frequency of the frame queue, the VMAF service thread is prevented from occupying too many computing resources and affecting the source stream video transcoding efficiency, thereby ensuring the normal transcoding operation of the encoding thread. Without affecting the user's viewing experience, image quality monitoring calculations are performed as much as possible to optimize the image quality monitoring calculation effect.

[0072] Based on the above embodiments, Figure 4 This is a schematic diagram of the structure of a frame queue management system based on image quality monitoring provided by this application. Figure 4 The frame queue management system based on image quality monitoring provided in this embodiment specifically includes: a recording module 21, a calculation module 22 and an adjustment module 23.

[0073] Wherein, the recording module 21 is configured to record the time difference between each video frame sent into the frame queue and the previous video frame, and record the image quality monitoring calculation time of each video frame in the frame queue;

[0074] The calculation module 22 is configured to determine an average time difference based on each input time difference, and determine an average calculation time based on each image quality monitoring calculation time;

[0075] The adjustment module 23 is configured to adjust the frequency of video frame input into the frame queue based on the comparison result of the average time difference and the average calculation time.

[0076] Specifically, the calculation module 22 is configured to calculate the average time difference based on each input time difference and the specified number, and calculate the average calculation time based on each image quality monitoring calculation time and the specified number when the number of video frames sent into the frame queue is equal to the specified number; when the number of video frames sent into the frame queue is greater than the specified number, the average time difference is calculated based on the average time difference calculated for the previous video frame and the input time difference calculated for the current video frame, using a preset first influencing parameter, and the average calculation time is calculated based on the average calculation time calculated for the previous video frame and the image quality monitoring calculation time of the current video frame, using a preset second influencing parameter.

[0077] Specifically,

[0078] When the number of video frames sent to the frame queue is greater than the specified number, the average time difference is calculated as follows:

[0079]

[0080] in, represents the average time difference calculated corresponding to the current video frame, θ represents the first influencing parameter, represents the average time difference calculated from the previous video frame corresponding to the current video frame, I i+1 Indicates the input time difference calculated corresponding to the current video frame;

[0081] The average calculation time is calculated as:

[0082]

[0083]

[0084] The quality monitoring calculation time of the video frame.

[0085] Specifically, the adjustment module 23 is configured to use the product of the set proportional coefficient and the average time difference as the time difference threshold; when the average calculation time is greater than the time difference threshold, the frequency of video frame input into the frame queue is lowered; when the average calculation time is less than or equal to the time difference threshold, the frequency of video frame input into the frame queue is maintained unchanged, and the time difference threshold is less than the average time difference.

[0086] Among them, when the average calculation time is greater than the time difference threshold and less than or equal to the average time difference, the video frame input frequency of the frame queue is lowered to a first set ratio of the initial input frequency; when the average calculation time is greater than the average time difference, the video frame input frequency of the frame queue is lowered to a second set ratio of the initial input frequency, and the first set ratio is greater than the second set ratio.

[0087] When the number of video frames sent into the frame queue is an integer multiple of the target number, the frequency of sending the video frames into the frame queue is adjusted based on the comparison result of the average time difference and the average calculation time.

[0088] In addition, the frame queue management system based on image quality monitoring is also used to record the queue length of the frame queue in real time, and stop feeding video frames into the frame queue when the queue length exceeds a set length threshold.

[0089] In the above, the time difference between each video frame sent into the frame queue and the previous video frame is recorded, and the image quality monitoring calculation time of each video frame in the frame queue is recorded. Then, the average time difference is determined based on each input time difference, and the average calculation time is determined based on each image quality monitoring calculation time. After that, the video frame input frequency of the frame queue is adjusted based on the comparison result of the average time difference and the average calculation time. Using the above technical means, the occupation of computing resources by the VMAF service thread is evaluated according to the comparison result of the image quality monitoring calculation time and the video frame input time of the frame queue. By adjusting the video frame input frequency of the frame queue, the VMAF service thread is prevented from occupying too many computing resources and affecting the source stream video transcoding efficiency, thereby ensuring the normal transcoding operation of the encoding thread. Without affecting the user's viewing experience, image quality monitoring calculations are performed as much as possible to optimize the image quality monitoring calculation effect.

[0090] The frame queue management system based on image quality monitoring provided in the embodiment of the present application can be configured to execute the frame queue management method based on image quality monitoring provided in the above embodiment, and has corresponding functions and beneficial effects.

[0091] Based on the above practical example, the embodiment of the present application also provides a frame queue management device based on image quality monitoring, referring to Figure 5, the frame queue management device based on image quality monitoring includes: a processor 31, a memory 32, a communication module 33, an input device 34 and an output device 35. The memory 32, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the frame queue management method based on image quality monitoring described in any embodiment of the present application (for example, a recording module, a calculation module and an adjustment module in the frame queue management system based on image quality monitoring). The communication module is configured to perform data transmission. The processor 31 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory, that is, to realize the above-mentioned frame queue management method based on image quality monitoring. The input device 34 can be configured to receive input digital or character information, and generate key signal input related to user settings and function control of the device. The output device 35 may include a display device such as a display screen. The frame queue management device based on image quality monitoring provided above can be configured to execute the frame queue management method based on image quality monitoring provided in the above-mentioned embodiment, and has corresponding functions and beneficial effects.

[0092] On the basis of the above embodiments, the embodiments of the present application further provide a storage medium containing computer executable instructions, wherein the computer executable instructions are configured to execute a frame queue management method based on image quality monitoring when executed by a computer processor, and the storage medium may be any of various types of memory devices or storage devices. Of course, the storage medium containing computer executable instructions provided in the embodiments of the present application, whose computer executable instructions are not limited to the frame queue management method based on image quality monitoring as described above, can also execute the related operations in the frame queue management method based on image quality monitoring provided in any embodiment of the present application.

[0093] On the basis of the above embodiments, the embodiments of the present application also provide a computer program product. The essence of the technical solution of the present application or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium, including a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the frame queue management method based on image quality monitoring described in each embodiment of the present application.

Claims

1. A frame queue management method based on image quality monitoring, characterized in that: include: Recording the time difference between each video frame sent into the frame queue and the previous video frame, and recording the image quality monitoring calculation time of each video frame in the frame queue; Determine an average time difference based on each of the input time differences, and determine an average calculation time based on each of the image quality monitoring calculation times; The product of the set proportional coefficient and the average time difference is used as the time difference threshold; When the average calculation time is greater than the time difference threshold, the frequency of video frame input to the frame queue is lowered; when the average calculation time is less than or equal to the time difference threshold, the frequency of video frame input to the frame queue is maintained unchanged, and the time difference threshold is less than the average time difference.

2. The frame queue management method based on image quality monitoring according to claim 1, characterized in that: When the average calculation time is greater than the time difference threshold, lowering the frequency of sending video frames into the frame queue includes: When the average calculation time is greater than the time difference threshold and less than or equal to the average time difference, lowering the video frame input frequency of the frame queue to a first set ratio of the initial input frequency; When the average calculation time is greater than the average time difference, the video frame input frequency of the frame queue is lowered to a second set ratio of the initial input frequency, and the first set ratio is greater than the second set ratio.

3. The frame queue management method based on image quality monitoring according to claim 1, characterized in that: Also includes: The queue length of the frame queue is recorded in real time, and when the queue length exceeds a set length threshold, the video frames are stopped from being sent to the frame queue.

4. The frame queue management method based on image quality monitoring according to claim 1, characterized in that: The step of determining the average time difference based on each of the input time differences and determining the average calculation time based on each of the image quality monitoring calculation times includes: When the number of video frames sent into the frame queue is equal to the specified number, the average time difference is calculated based on each of the sending time differences and the specified number, and the average calculation time is calculated based on each of the image quality monitoring calculation times and the specified number; When the number of video frames sent into the frame queue is greater than the specified number, the average time difference is calculated based on the average time difference calculated for the previous video frame and the sending time difference calculated for the current video frame using a preset first influencing parameter, and the average calculation time is calculated based on the average calculation time calculated for the previous video frame and the image quality monitoring calculation time for the current video frame using a preset second influencing parameter.

5. The frame queue management method based on image quality monitoring according to claim 4, characterized in that: When the number of video frames sent to the frame queue is greater than the specified number, the calculation formula of the average time difference is: in, represents the average time difference calculated corresponding to the current video frame, θ represents the first influencing parameter, represents the average time difference calculated for the previous video frame corresponding to the current video frame, I i+1 Indicates the input time difference calculated corresponding to the current video frame; The calculation formula of the average calculation time is: in, represents the average calculation time corresponding to the current video frame, φ represents the second influencing parameter, represents the average calculation time of the previous video frame corresponding to the current video frame, T i+1 Indicates the image quality monitoring calculation time of the current video frame.

6. The frame queue management method based on image quality monitoring according to claim 4, characterized in that: When the number of video frames sent into the frame queue is greater than the specified number, adjusting the video frame sending frequency of the frame queue based on the comparison result of the average time difference and the average calculation time includes: When the number of video frames sent into the frame queue is an integer multiple of the target number, the frequency of sending the video frames into the frame queue is adjusted based on the comparison result of the average time difference and the average calculation time.

7. A frame queue management system based on image quality monitoring, characterized in that: include: A recording module configured to record the time difference between each video frame sent into the frame queue and the previous video frame, and record the image quality monitoring calculation time of each video frame in the frame queue; A calculation module, configured to determine an average time difference based on each of the input time differences, and to determine an average calculation time based on each of the image quality monitoring calculation times; an adjustment module, configured to use a product of a set proportional coefficient and the average time difference as a time difference threshold; When the average calculation time is greater than the time difference threshold, the frequency of video frame input to the frame queue is lowered; when the average calculation time is less than or equal to the time difference threshold, the frequency of video frame input to the frame queue is maintained unchanged, and the time difference threshold is less than the average time difference.

8. A frame queue management device based on image quality monitoring, characterized in that: include: memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the frame queue management method based on image quality monitoring as described in any one of claims 1-6.

9. A storage medium containing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are configured to execute the frame queue management method based on image quality monitoring as described in any one of claims 1-6.

10. A computer program product, characterized in that The computer program product includes instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the frame queue management method based on image quality monitoring as described in any one of claims 1-6.

Images