A method for image processing of a graphic card

Abstract

The present application relates to the field of image processing, and more particularly to a kind of image hierarchical processing method for graphic card, comprising: obtaining the two-dimensional image data corresponding to each video frame image in target video, whether image processing optimization is carried out for target video according to picture smoothness and graphic card load reference value;When image processing optimization is carried out, the initial rendering resolution of each video frame image is determined according to the complexity evaluation threshold of video frame image, and according to the number of mutation frame and mutation influence degree, each video frame image is uniformly combined or associated combined to obtain several video frame image combinations;According to the dynamic mutation coefficient and mutation duration of each video frame image combination, continuous interframe mixing or adjacent interframe mixing is carried out for each mutation frame to determine the actual rendering resolution of each mutation frame.The present application can improve the smoothness of image display when the load pressure of graphic card is large.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to an image hierarchical processing method for a graphics card. BACKGROUND

[0002] When processing complex image and video content, the limitations of traditional image processing methods gradually emerge. For example, when facing high-resolution, high-frame-rate videos, the load of the graphics card increases dramatically, often leading to a decrease in picture smoothness and frame rate fluctuations, which seriously affects the visual experience. At the same time, the complexity and change degree of different scenes differ greatly, and traditional methods are difficult to quickly adapt to these dynamic changes, resulting in poor image processing results, such as loss of details, rendering delays, and other problems. Therefore, how to improve the smoothness of image display when the load pressure of the graphics card is large is a problem that needs to be solved by those skilled in the art.

[0003] Chinese Patent Publication No. CN104318511A discloses a computer graphics card and its image processing method, which includes a first filtering module, a second filtering module, a third filtering module, and a first buffer zone, a second buffer zone, and a third buffer zone connected to the first filtering module, the second filtering module, and the third filtering module, respectively. The first buffer zone and the second buffer zone are commonly connected to the first GPU, the first GPU and the third buffer zone are commonly connected to the second GPU, the second GPU is connected to the third GPU, and the third GPU is connected to the fourth buffer zone. However, the above-mentioned scheme has the following problems: the flexibility and adaptability of image processing are poor, the image processing strategy cannot be dynamically adjusted according to real-time parameters, and the smoothness of image display is poor when the load pressure of the graphics card is large. SUMMARY

[0004] Therefore, the present application provides an image hierarchical processing method for a graphics card to overcome the problem of poor flexibility and adaptability of image processing in the prior art, which cannot dynamically adjust the image processing strategy according to real-time parameters, resulting in poor smoothness of image display when the load pressure of the graphics card is large.

[0005] To achieve the above-mentioned purpose, the present application provides an image hierarchical processing method for a graphics card, which comprises:

[0006] Obtaining two-dimensional image data corresponding to each video frame image in a target video, and determining whether to perform image processing optimization on the target video according to the picture smoothness and the graphics card load reference value;

[0007] When performing image processing optimization, determining the initial rendering resolution of each video frame image according to the complexity evaluation threshold of the video frame image, and performing uniform combination or associated combination on each video frame image according to the number of mutation frames and the mutation influence degree to obtain a plurality of video frame image combinations;

[0008] Based on the dynamic mutation coefficient and mutation duration of each video frame image combination, continuous inter-frame blending or adjacent inter-frame blending is performed on each mutated frame to determine the actual rendering resolution of each mutated frame.

[0009] The number of buffered frames is determined based on the evaluation reference value, and the change complexity is determined based on the rendering change degree corresponding to each mutated frame in the buffered frames. Based on the change complexity corresponding to each buffered frame and the loading deviation threshold, it is determined whether to adjust the graphics card refresh rate according to the evaluation threshold.

[0010] Under conditions of graphics card instability, the pre-adjusted video frame is determined based on the abnormal correlation, and the region rendering level or region rendering resolution is adjusted for the pre-adjusted video frame based on the abnormal coefficient.

[0011] Furthermore, if the smoothness of the video is less than the preset smoothness or the graphics card load reference value is greater than or equal to the preset graphics card load reference value, then image optimization processing will be performed on the target video.

[0012] If the smoothness of the video is greater than or equal to the preset smoothness and the graphics card load reference value is less than the preset graphics card load reference value, then no image optimization processing is required for the target video.

[0013] Furthermore, the initial rendering resolution of each video frame image is determined based on a complex evaluation threshold of the video frame image;

[0014] The initial rendering resolution of a single video frame is negatively correlated with the complexity evaluation threshold of that video frame.

[0015] Furthermore, if the number of mutated frames is greater than or equal to the preset number of mutated frames or the degree of mutation is greater than or equal to the preset degree of mutation, then the images of each video frame are associated and combined.

[0016] If the number of mutated frames is less than the preset number of mutated frames and the impact of the mutation is less than the preset impact of the mutation, then the images of each video frame are uniformly combined.

[0017] Furthermore, for combinations of images from a single video frame,

[0018] If the dynamic mutation coefficient is greater than or equal to the preset dynamic mutation coefficient or the mutation duration is greater than or equal to the preset mutation duration, then continuous inter-frame mixing is performed for each mutated frame in the video frame image combination.

[0019] If the dynamic mutation coefficient is less than the preset dynamic mutation coefficient and the mutation duration is less than the preset mutation duration, then neighboring frames are mixed for each mutated frame in the video frame image combination.

[0020] Furthermore, the number of buffer frames is determined based on the evaluation reference value;

[0021] The number of buffer frames is positively correlated with the evaluation reference value.

[0022] Furthermore, if the loading deviation threshold corresponding to each buffer frame is greater than the preset loading deviation threshold or the change complexity is greater than the preset change complexity, the refresh rate of the graphics card will be reduced according to the evaluation threshold.

[0023] The decrease in the graphics card refresh rate is positively correlated with the evaluation threshold.

[0024] Furthermore, for a single pre-adjusted video frame,

[0025] If the anomaly coefficient is less than the preset anomaly coefficient, then the region rendering level is set for the pre-adjusted video frame.

[0026] If the anomaly coefficient is greater than or equal to the preset anomaly coefficient, then the region rendering resolution is adjusted for the pre-adjusted video frame.

[0027] Furthermore, in the regional rendering level setting, the region is divided into several sub-regions based on the pixel fluctuation value of the pre-adjusted video frame, the number of rendering levels is determined based on the anomaly coefficient, the sub-regions corresponding to each rendering level are determined based on the regional pixel change value, and whether to increase the number of rendering levels is determined based on the loading deviation.

[0028] If the loading deviation is greater than the preset loading deviation, the number of rendering levels will be increased. The increase in the number of rendering levels is positively correlated with the loading deviation.

[0029] Furthermore, in the regional rendering resolution adjustment, the actual rendering resolution of each edge sub-region is reduced based on the abnormal coupling degree;

[0030] The reduction in the actual rendering resolution of each edge sub-region is positively correlated with the abnormal coupling degree.

[0031] Compared with the prior art, the beneficial effects of the present invention are that, in the technical solution of the present invention, the smoothness of the image during the display process and the current workload of the graphics card are effectively reflected by the screen smoothness and the graphics card load reference value. Then, based on the screen smoothness and the graphics card load reference value, it is determined whether to perform image processing optimization for the target video. By monitoring the screen smoothness, the user experience problems such as stuttering and tearing when watching videos are reduced, making the screen smoother. At the same time, by monitoring the graphics card load and adjusting it in time, system crashes or performance degradation caused by graphics card overload can be avoided.

[0032] Furthermore, in this invention, determining the initial rendering resolution of each video frame image based on the complexity evaluation threshold of the video frame image can reduce the rendering resolution of complex frames, thereby reducing the load on the graphics card and improving overall performance. By effectively reflecting the actual state of the mutated frames through the number of mutated frames and the degree of mutation impact, the corresponding or uniform combination can be performed, avoiding the resource waste or performance deficiency that may be caused by the fixed combination strategy, thereby improving the smoothness of image display.

[0033] Furthermore, this invention effectively reflects the frequency of scene changes and the degree of aggregation of mutated frames by using the dynamic mutation coefficient and mutation duration of the combined video frame images. Then, it adaptively performs continuous inter-frame mixing or adjacent inter-frame mixing according to the actual application scenario. Continuous inter-frame mixing uses information from multiple historical frames to smooth mutations, reduce screen tearing, and improve the naturalness of dynamic blur. Adjacent inter-frame mixing only refers to adjacent frames to reduce computational complexity and avoid excessive smoothing that leads to loss of detail. This reduces the abruptness of mutated frames while avoiding over-processing of smooth scenes, thereby improving the overall viewing comfort.

[0034] Furthermore, by dynamically adjusting the number of pre-loaded frames through evaluation reference values, this invention can reduce stuttering caused by rendering latency. Through dynamic buffering, rendering latency can be handled more smoothly, reducing frame rate fluctuations caused by excessively long single-frame rendering time. When the loading deviation threshold or change complexity of the buffered frames exceeds the preset value, the graphics card refresh rate is dynamically reduced according to the evaluation threshold, which can distribute the rendering pressure of a single frame, reduce stuttering or frame drops caused by computation timeout, and thus improve the image processing effect.

[0035] Furthermore, in this invention, the anomaly coefficient effectively reflects the degree of anomaly in the pre-adjusted video frame, and then the region rendering level is set or the region initial rendering resolution is adjusted according to the anomaly coefficient. Setting the region rendering level can prioritize the processing of high-fluctuation regions, and adjusting the region initial rendering resolution can dynamically reduce the overall resolution, which can accurately control the degree of degradation and thus improve the smoothness of the picture. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the image grading processing method for graphics cards according to the present invention;

[0037] Figure 2 This is a flowchart illustrating the process of determining whether to perform image processing optimization on a target video based on screen smoothness and graphics card load reference values ​​in this invention.

[0038] Figure 3 This is a flowchart illustrating the present invention's method of uniformly or correlatedly combining video frame images based on the number of mutated frames and the degree of mutation impact.

[0039] Figure 4This is a flowchart illustrating how the present invention performs continuous inter-frame mixing or adjacent inter-frame mixing for each mutated frame based on the dynamic mutation coefficient and mutation duration of each video frame image combination. Detailed Implementation

[0040] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0041] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0042] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0043] Please see Figures 1 to 4 As shown, this invention provides a method for predicting the effectiveness of biological wastewater treatment based on machine learning, comprising:

[0044] Obtain two-dimensional image data corresponding to each video frame in the target video, and determine whether to perform image processing optimization for the target video based on the smoothness of the picture and the reference value of the graphics card load.

[0045] When performing image processing optimization, the initial rendering resolution of each video frame image is determined based on the complex evaluation threshold of the video frame image, and the video frame images are uniformly or correlatedly combined according to the number of mutated frames and the degree of mutation influence to obtain a combination of several video frame images.

[0046] Based on the dynamic mutation coefficient and mutation duration of each video frame image combination, continuous inter-frame blending or adjacent inter-frame blending is performed on each mutated frame to determine the actual rendering resolution of each mutated frame.

[0047] The number of buffered frames is determined based on the evaluation reference value, and the change complexity is determined based on the rendering change degree corresponding to each mutated frame in the buffered frames. Based on the change complexity corresponding to each buffered frame and the loading deviation threshold, it is determined whether to adjust the graphics card refresh rate according to the evaluation threshold.

[0048] Under conditions of graphics card instability, the pre-adjusted video frame is determined based on the abnormal correlation, and the region rendering level or region rendering resolution is adjusted for the pre-adjusted video frame based on the abnormal coefficient.

[0049] The application scenario of this invention is image processing when displaying video frame images from a target video on a screen. The target video is the video that needs to be displayed on the screen. Each video frame image corresponds to one or two-dimensional image data. The two-dimensional image data corresponding to a single video frame image is the pixel value and position coordinates of each pixel point in the video frame image.

[0050] This invention includes several historical records, each recording at least one frame rate reference value, frame time fluctuation value, image smoothness, graphics card load reference value, and pixel fluctuation value during the image processing process of displaying video frames from the target video on the screen. Each historical record also has a corresponding pass / fail marker, indicating whether the image processing process of displaying video frames from the target video on the screen meets the user's requirements. These pass / fail markers can be manually recorded. It is understood that users can determine whether the image processing process of displaying video frames from the target video on the screen meets their requirements based on self-defined indicators. These self-defined indicators can be, but are not limited to, frame rate, and will not be elaborated upon here. Frame rate is measured using Fraps.

[0051] This invention sets target coefficients and relevant thresholds. The correspondence between the target coefficients and relevant thresholds is expressed by a weighting formula: Target Coefficient = Weighting Coefficient × Relevant Threshold. Specifically, this invention uses the initial rendering resolution, the number of mixed frames, the first adjustment coefficient, the number of buffer frames, the decrease in the graphics card refresh rate, the increase in the number of rendering levels, the value of 'a', the number of rendering levels, and the decrease in the actual rendering resolution as target coefficients. It uses the complexity evaluation threshold, combined anomaly degree, mutation reference value, evaluation reference value, evaluation threshold, loading deviation degree, pixel fluctuation value, anomaly coefficient, and anomaly coupling degree as relevant thresholds. It can be understood that all target coefficients have corresponding relevant thresholds. For example, the number of mixed frames corresponding to the target mutation frame and the video frame where the target mutation frame is located... There is a positive correlation between the combined anomaly degree corresponding to the combination, and the positive correlation between the number of mixed frames corresponding to the target mutation frame and the combined anomaly degree corresponding to the combination of video frame images where the target mutation frame is located is expressed by a weight formula. The value of the weight coefficient can be determined by the user's historical experience based on the degree of influence of the combined anomaly degree corresponding to the combination of video frame images where the target mutation frame is located on the number of mixed frames corresponding to the target mutation frame. Furthermore, the value of the weight coefficient can be optimized by combining the historical record of the image processing process when displaying video frame images in the target video on the screen multiple times with a multilayer perceptron. The value of the weight coefficient optimized by the multilayer perceptron is easy for those skilled in the art to understand and will not be elaborated here. The principle of the value of the weight coefficient corresponding to other target coefficients and related thresholds is the same and will not be elaborated here.

[0052] Specifically, if the smoothness of the video is less than the preset smoothness or the graphics card load reference value is greater than or equal to the preset graphics card load reference value, then image optimization processing will be performed on the target video.

[0053] If the smoothness of the video is greater than or equal to the preset smoothness and the graphics card load reference value is less than the preset graphics card load reference value, then no image optimization processing is required for the target video.

[0054] The present invention features a continuous monitoring cycle. At the end of each monitoring cycle, the smoothness of the screen and the reference value of the graphics card load are determined. The duration of the monitoring cycle can be set according to the user's needs. The greater the user's need for monitoring accuracy, the shorter the monitoring cycle duration. A value for the monitoring cycle duration is provided, which is 5 seconds.

[0055] The monitoring cycle that is preceding and adjacent to the current monitoring cycle is designated as the target monitoring cycle, and the end time of the target monitoring cycle is designated as the target time.

[0056] Graphics card load reference value = Current video memory used by the graphics card at the target time / Total video memory of the graphics card; The current video memory used by the graphics card is determined by NVIDIA SMI, which is a common technique used by those skilled in the art and will not be elaborated on in detail.

[0057] Smoothness of the image = Frame rate reference value / Preset frame rate reference value - Frame time fluctuation value / Preset frame time fluctuation value;

[0058] The frame rate reference value = the number of target video frame images / 5s. Each video frame image that is rendered within the target monitoring period is recorded as the target video frame image. The rendering process corresponding to a single video frame image is to construct a scene and set materials and textures for the two-dimensional image data corresponding to the video frame image. The process of scene construction and material and texture setting is a common technique used by those skilled in the art, and will not be elaborated on in detail.

[0059] The frame time fluctuation value is the standard deviation of the rendering time corresponding to each target video frame image. The number of target video frames images is measured by Fraps, and the rendering time corresponding to a single target video frame image is measured by CapFrameX. These are common techniques used by those skilled in the art, and will not be elaborated on in detail.

[0060] Users can determine the preset frame rate reference value and preset frame time fluctuation value according to the actual application scenario. The greater the user's need to improve the accuracy of the screen smoothness judgment, the smaller the preset frame rate reference value and preset frame time fluctuation value will be. The average value of the frame rate reference value and the average value of the frame time fluctuation value corresponding to the historical records that can meet the user's needs will be recorded as the preset frame rate reference value and preset frame time fluctuation value, respectively.

[0061] Users can determine the preset screen smoothness and preset graphics card load reference values ​​based on their actual application scenarios. The greater the user's demand for improved image display smoothness, the higher the preset screen smoothness value and the lower the preset graphics card load reference value. A method for determining the preset screen smoothness and preset graphics card load reference values ​​is provided, which detects the user's historical image optimization processing history for the target video, and records the average screen smoothness and the average graphics card load reference value corresponding to the historical records that meet the user's needs as the preset screen smoothness and preset graphics card load reference values, respectively.

[0062] Specifically, the initial rendering resolution of each video frame image is determined based on a complex evaluation threshold of the video frame image;

[0063] The initial rendering resolution of a single video frame is negatively correlated with the complexity evaluation threshold of that video frame.

[0064] Wherein, the complexity evaluation threshold corresponding to a single video frame image = graphics card load reference value / preset graphics card load reference value + (1 - pixel fluctuation value / preset pixel fluctuation value);

[0065] The pixel fluctuation value corresponding to a single video frame is the standard deviation of the pixel value corresponding to each pixel in the two-dimensional image data corresponding to that video frame; the preset pixel fluctuation value is the average of the pixel fluctuation values ​​corresponding to each video frame in each historical record.

[0066] The initial rendering resolution corresponding to a single video frame image = the number of pixels actually generated during the image rendering process of that video frame image / the number of pixels in the two-dimensional image data corresponding to that video frame image;

[0067] Understandably, the complexity evaluation threshold reflects the rendering difficulty and resource requirements of the current video frame image. By dynamically determining the initial rendering resolution of the video frame image through the complexity evaluation threshold, it is possible to make reasonable use of graphics card resources while ensuring image quality and avoiding resource waste or overload.

[0068] Specifically, if the number of mutated frames is greater than or equal to the preset number of mutated frames or the degree of mutation is greater than or equal to the preset degree of mutation, then the images of each video frame are associated and combined.

[0069] If the number of mutated frames is less than the preset number of mutated frames and the impact of the mutation is less than the preset impact of the mutation, then the images of each video frame are uniformly combined.

[0070] Among them, video frame images that have been rendered are recorded as rendered images, and video frame images that have not been rendered are recorded as images to be rendered;

[0071] For a single image to be rendered, the image to be rendered is recorded as the target image to be rendered. The video frame image that is before and adjacent to the target image to be rendered is recorded as the neighboring image corresponding to the target image to be rendered. If the rendering mutation value corresponding to the target image to be rendered is greater than the preset rendering mutation value, then the target image to be rendered is recorded as the mutation frame.

[0072] The rendering mutation value corresponding to the target image to be rendered = |the initial rendering resolution corresponding to the target image to be rendered - the initial rendering resolution of the neighboring images corresponding to the target image to be rendered|;

[0073] The number of mutated frames is the total number of mutated frames in the image to be rendered, and the mutation impact is the standard deviation of the rendering mutation value corresponding to each mutated frame.

[0074] Users can determine the preset number of mutation frames and preset mutation impact based on their actual application scenarios. The smaller the preset number of mutation frames and preset mutation impact, the greater the user's need for associating and combining images of each video frame. This provides a preset number of mutation frames and preset mutation impact, detects the user's historical records of associating and combining images of each video frame, and records the average number of mutation frames and the average mutation impact corresponding to the historical records that meet the user's needs as the preset number of mutation frames and preset mutation impact, respectively.

[0075] The process involves associating and combining video frame images, including: performing association analysis on each image to be rendered in chronological order from earliest to latest; when performing association analysis on a single image to be rendered, the image to be rendered is designated as the target image, and the other images to be rendered not included in the video frame image combination are designated as reference images; the rendering mutation values ​​corresponding to the reference images and the rendering mutation values ​​corresponding to the target images are accumulated sequentially in chronological order from earliest to latest until the total accumulated value is greater than a preset total accumulated value; then, the reference images before the reference image corresponding to the last accumulated rendering mutation value and the target image are designated as a video frame image combination, and the association analysis continues on the images to be rendered not included in the video frame image combination until all images to be rendered are included in the video frame image combination, at which point the association analysis stops.

[0076] The cumulative total is the sum of all rendering mutation values, which are calculated by adding the rendering mutation values ​​of the reference image and the target image in chronological order from earliest to latest.

[0077] The preset cumulative total value can be determined by the user based on the actual application scenario. The greater the user's need to improve image processing accuracy, the smaller the preset cumulative total value should be. A method for determining the preset cumulative total value is provided, which is the average of the cumulative total values ​​corresponding to each combination of video frame images in the historical record that can meet the user's needs. The cumulative total value corresponding to a single combination of video frame images is the sum of the rendering mutation values ​​corresponding to each video frame image in that combination.

[0078] The process involves uniformly combining images from each video frame, including: sequentially extracting images to be rendered in chronological order from earliest to latest, until the number of extracted images to be rendered reaches n. Then, the n extracted images to be rendered are recorded as a video frame image combination. The process continues to extract and combine images to be rendered from the remaining images to be rendered in the same way, until all images to be rendered have been extracted and combined. n is the smallest integer greater than or equal to n0, and n0 = 2 × the total number of images to be rendered / the number of mutation frames.

[0079] It is understandable that if the number of mutated frames is greater than or equal to the preset number of mutated frames or the degree of mutation is greater than or equal to the preset degree of mutation, it indicates that the mutation situation in the current video frame sequence is relatively significant. By combining mutated frames and their neighboring frames together, mutations can be better handled, discontinuities in the rendering process can be reduced, and the smoothness and consistency of the picture can be improved.

[0080] If the number of mutated frames is less than the preset number of mutated frames and the impact of the mutation is less than the preset impact of the mutation, it means that the mutation situation in the current video frame sequence is not significant. By evenly distributing the video frame images into different combinations, resources can be utilized more efficiently and rendering efficiency can be improved.

[0081] Specifically, for a combination of images from a single video frame,

[0082] If the dynamic mutation coefficient is greater than or equal to the preset dynamic mutation coefficient or the mutation duration is greater than or equal to the preset mutation duration, then continuous inter-frame mixing is performed for each mutated frame in the video frame image combination.

[0083] If the dynamic mutation coefficient is less than the preset dynamic mutation coefficient and the mutation duration is less than the preset mutation duration, then neighboring frames are mixed for each mutated frame in the video frame image combination.

[0084] Wherein, the dynamic mutation coefficient corresponding to a single video frame image combination = the number of mutated frames in the video frame image combination / the total number of video frame images in the video frame image combination;

[0085] The mutation duration corresponding to a single video frame image combination = (the number of mutation frames between the initial mutation frame and the final mutation frame of the video frame image combination + 2) / (the number of video frame images between the initial mutation frame and the final mutation frame of the video frame image combination + 2).

[0086] The first and last mutation frames appearing in the video frame image combination are recorded as the initial mutation frame and the final mutation frame, respectively, in chronological order from earliest to latest.

[0087] When performing continuous inter-frame mixing for a single mutation frame, the mutation frame is recorded as the target mutation frame. The number of mixed frames corresponding to the target mutation frame is determined based on the combination anomaly degree. The number of mixed frames corresponding to the target mutation frame is positively correlated with the combination anomaly degree corresponding to the combination of video frame images in which the target mutation frame is located.

[0088] Combinatorial anomaly = Dynamic mutation coefficient / Preset dynamic mutation coefficient + Mutation persistence / Preset mutation persistence;

[0089] The mixed frames corresponding to the target mutation frame are video frame images whose time is earlier than the time corresponding to the target mutation frame, selected in ascending order of the interval reference value with respect to the target mutation frame;

[0090] The formula for calculating the actual rendering resolution K corresponding to the target mutation frame is: α0 is the initial rendering resolution corresponding to the target mutation frame, k0 is the first adjustment coefficient, m is the number of mixed frames corresponding to the target mutation frame, and i is 1, 2, ..., m. i k is the sub-adjustment coefficient corresponding to the i-th hybrid frame of the target mutation frame. i The actual rendering resolution is the i-th blended frame corresponding to the target mutation frame.

[0091] The actual rendering resolution of video frames other than the mutation frame is the same as the initial rendering resolution.

[0092] The first adjustment coefficient corresponding to the target mutation frame is determined based on the mutation reference value. The first adjustment coefficient corresponding to the target mutation frame is negatively correlated with the mutation reference value corresponding to the target mutation frame. The value range of the first adjustment coefficient is [0,1].

[0093] The mutation reference value corresponding to a single mutation frame = the initial rendering resolution corresponding to the mutation frame / the average of the initial rendering resolutions of each video frame in the combination of video frame images corresponding to the mutation frame;

[0094] Second adjustment coefficient = 1 - First adjustment coefficient;

[0095] The sub-adjustment coefficient corresponding to a single hybrid frame = (weight coefficient / total weight coefficient) × second adjustment coefficient;

[0096] The weighting coefficient for a single mixed frame = similarity reference value / preset similarity reference value + (1 - interval reference value / preset interval reference value);

[0097] The method for determining the similarity reference value between any two video frame images is to record the pixels in the two video frame images that are located at the same position and have the same pixel value as the same pixel. The similarity reference value = the total number of the same pixels in a single video frame image / the total number of pixels in a single video frame image.

[0098] The reference value for the interval between any two video frames is the total number of video frames located between the two video frames.

[0099] Users can determine the preset similarity reference value and preset interval reference value according to the actual application scenario. The greater the user's need to improve the accuracy of the weight coefficient, the smaller the preset similarity reference value and preset interval reference value will be. One preset similarity reference value and preset interval reference value are provided: the preset similarity reference value is 50% and the preset interval reference value is 4.

[0100] The total weight coefficient corresponding to the target mutation frame is the sum of the weight coefficients of each mixed frame corresponding to the target mutation frame;

[0101] When performing inter-frame blending for each mutated frame in the video frame image combination, the actual rendering resolution of a single mutated frame = the initial rendering resolution of the mutated frame × the first adjustment coefficient + the actual rendering resolution of the neighboring frames of the mutated frame × the second adjustment coefficient.

[0102] The neighboring frames corresponding to a single mutation frame are video frame images that are adjacent to the mutation frame and occur earlier than the mutation frame.

[0103] It is understandable that if the dynamic mutation coefficient is greater than or equal to the preset dynamic mutation coefficient or the mutation duration is greater than or equal to the preset mutation duration, it means that the mutation is more frequent or the duration is longer. By performing continuous inter-frame blending, the mutation can be better handled, the discontinuity in the rendering process can be reduced, and the smoothness and consistency of the picture can be improved.

[0104] If the dynamic mutation coefficient is less than the preset dynamic mutation coefficient and the mutation duration is less than the preset mutation duration, it means that there are fewer mutations or the duration is shorter. By mixing adjacent frames, resources can be used more efficiently and rendering efficiency can be improved, especially when dealing with relatively static or little changing scenes.

[0105] Specifically, the number of buffered frames is determined based on an evaluation reference value;

[0106] The number of buffer frames is positively correlated with the evaluation reference value.

[0107] The evaluation reference value is calculated as follows: screen smoothness / preset screen smoothness + (1 - graphics card load reference value / preset graphics card load reference value).

[0108] A buffer frame is a video frame image that has been pre-rendered but has not yet been displayed on the screen; the number of buffer frames is the total number of buffer frames.

[0109] Understandably, by comprehensively evaluating the smoothness of the current screen and the graphics card load, dynamically adjusting the number of buffered frames to optimize the rendering process can ensure the smoothness and stability of the screen.

[0110] Specifically, if the loading deviation threshold corresponding to each buffer frame is greater than the preset loading deviation threshold or the change complexity is greater than the preset change complexity, the refresh rate of the graphics card will be reduced according to the evaluation threshold.

[0111] The decrease in the graphics card refresh rate is positively correlated with the evaluation threshold.

[0112] If the loading deviation threshold corresponding to each buffer frame is less than or equal to the preset loading deviation threshold and the change complexity is less than or equal to the preset change complexity, then there is no need to adjust the graphics card refresh rate according to the evaluation threshold.

[0113] The loading deviation threshold is the average of the sub-deviation thresholds corresponding to each buffer frame;

[0114] The sub-deviation threshold for a single buffer frame = the loading time of that buffer frame - the average loading time of each buffer frame in the historical records that can meet user needs;

[0115] The loading time for a single buffer frame is the time taken to render and process that buffer frame;

[0116] The change complexity is the average of the rendering change degree corresponding to each mutated frame in each buffer frame;

[0117] Evaluation threshold = Load deviation threshold / Preset load deviation threshold + Change complexity / Preset change complexity;

[0118] The values ​​of the preset loading deviation threshold and the preset change complexity can be determined by the user according to the actual application scenario. The greater the user's need to improve the accuracy of the evaluation threshold, the higher the value of the preset loading deviation threshold and the preset change complexity. A method for determining the preset loading deviation threshold and the preset change complexity is provided, which takes the average value of the loading deviation threshold and the average value of the change complexity corresponding to the historical records that can meet the user's needs, and records them as the preset loading deviation threshold and the preset change complexity respectively.

[0119] The graphics card refresh rate is the number of times per second the monitor can update the image after the graphics card sends the image data to the monitor. The unit is Hz. The initial graphics card refresh rate is 144Hz, which means that the monitor can refresh the image 144 times per second.

[0120] Understandably, the loading deviation threshold effectively reflects the efficiency and stability of the current rendering process. If the loading deviation threshold is high, the graphics card refresh rate can be appropriately reduced to reduce the rendering pressure.

[0121] Specifically, for a single pre-adjusted video frame,

[0122] If the anomaly coefficient is less than the preset anomaly coefficient, then the region rendering level is set for the pre-adjusted video frame.

[0123] If the anomaly coefficient is greater than or equal to the preset anomaly coefficient, then the region rendering resolution is adjusted for the pre-adjusted video frame.

[0124] Among them, the unstable condition of the graphics card adjustment is that after the refresh rate of the graphics card is reduced according to the evaluation threshold, the unstable coefficient is greater than the preset unstable coefficient.

[0125] Instability coefficient = Number of video frames with screen tearing displayed on the screen within 5 minutes after adjusting the refresh rate of the graphics card according to the evaluation threshold / Number of video frames displayed on the screen within 5 minutes after adjusting the refresh rate of the graphics card according to the evaluation threshold; Since video frames with screen tearing will produce obvious discontinuous edges in the vertical direction, video frames with screen tearing are determined by edge detection algorithm. In addition, the contents that are easy for those skilled in the art to understand will not be elaborated.

[0126] The user can determine the value of the preset instability coefficient according to the actual application scenario. The greater the user's demand for improving the image processing effect, the smaller the value of the preset instability coefficient. One preset instability coefficient value is provided, which is 10%.

[0127] Abnormal video frames are recorded as those that appear abnormal within 5 minutes of the video frame images displayed after the graphics card refresh rate is reduced according to the evaluation threshold.

[0128] The anomaly correlation degree corresponding to a single video frame image is the maximum value among the sub-anomaly correlation degrees between that video frame image and each abnormal video frame image. The sub-anomaly correlation degree between a single video frame image and a single abnormal video frame image = 1 / (the absolute value of the difference in rendering change degree between the two video frame images + 1).

[0129] The rendering variation of a single video frame image = |the actual rendering resolution of the video frame image -the actual rendering resolution of the video frame image that is adjacent to the video frame image and earlier in time|;

[0130] When determining the pre-adjusted video frame based on the abnormal correlation degree, the image to be rendered with an abnormal correlation degree greater than the preset abnormal correlation degree is recorded as the pre-adjusted video frame.

[0131] The value of the preset abnormal correlation degree can be determined by the user according to the actual application scenario. The greater the user's demand for improving the image processing effect, the smaller the value of the preset abnormal correlation degree. A method for determining the value of the preset abnormal correlation degree is provided, which is to record the average value of the abnormal correlation degree corresponding to each pre-adjusted video frame in the historical record that can meet the user's needs as the preset abnormal correlation degree.

[0132] The anomaly coefficient corresponding to a single pre-adjusted video frame is the tear edge length corresponding to the anomaly video frame with the highest correlation to the sub-anomaly corresponding to that pre-adjusted video frame; the tear edge length is obtained by extracting the contour of the tear edge through an edge detection algorithm, and then the number of pixels contained in the contour of the tear edge is recorded as the tear edge length.

[0133] The value of the preset anomaly coefficient can be determined by the user according to the actual application scenario. The larger the value of the preset anomaly coefficient, the greater the user's need for setting the region rendering level for the pre-adjusted video frame. A preset anomaly coefficient value is provided, which is the average value of the anomaly coefficients corresponding to each pre-adjusted video frame in the history of region rendering level setting that can meet the user's needs.

[0134] Understandably, if the anomaly coefficient is less than the preset anomaly coefficient, the anomaly degree of the pre-adjusted video frame is considered to be small. The regional rendering level setting can be dynamically adjusted according to the real-time rendering needs to ensure efficient use of resources. When dealing with complex scenes, the rendering level of non-critical areas can be automatically reduced to improve the smoothness of the picture.

[0135] If the anomaly coefficient is greater than or equal to the preset anomaly coefficient, the anomaly degree of the pre-adjusted video frame is considered to be large, and it needs to be handled by adjusting the initial rendering resolution of the region. Using different rendering resolutions for certain regions of the pre-adjusted video frame can reduce rendering time and improve the smoothness of the picture.

[0136] Specifically, in the regional rendering level setting, the region is divided into several sub-regions based on the pixel fluctuation value of the pre-adjusted video frame, the number of rendering levels is determined based on the anomaly coefficient, the sub-regions corresponding to each rendering level are determined based on the regional pixel change value, and whether to increase the number of rendering levels is determined based on the loading deviation.

[0137] If the loading deviation is greater than the preset loading deviation, the number of rendering levels will be increased. The increase in the number of rendering levels is positively correlated with the loading deviation.

[0138] It should be noted that if the loading deviation is less than or equal to the preset loading deviation, there is no need to adjust the number of rendering levels.

[0139] The video frame is divided into several sub-regions based on the pixel fluctuation value of the pre-adjusted video frame. The sub-regions are divided into a sub-regions with the same area and shape. Each sub-region is rectangular. The value of a is positively correlated with the pixel fluctuation value.

[0140] The number of rendering levels is determined based on the anomaly coefficient. The number of rendering levels corresponding to a single pre-adjusted video frame is positively correlated with the anomaly coefficient corresponding to that pre-adjusted video frame. The number of sub-regions corresponding to a single rendering level = a / number of rendering levels. For a single rendering level, the sub-regions corresponding to that rendering level are selected sequentially in descending order of the region's pixel change values.

[0141] The pixel change value corresponding to a single sub-region is the standard deviation of the pixel values ​​corresponding to each pixel in that sub-region.

[0142] Load deviation = Load level fluctuation - Preset load level fluctuation;

[0143] Load level volatility is the standard deviation of the level reference value corresponding to each rendering level. The level reference value corresponding to a single rendering level is the average value of the region pixel change value corresponding to each sub-region of that rendering level. The preset load level volatility value can be determined by the user according to the actual application scenario. The system detects historical records that have not been adjusted to increase the number of rendering levels, and records the average value of the load level volatility corresponding to the historical records that meet the user's needs as the preset load level volatility.

[0144] The number of rendering levels is a batch that is rendered simultaneously by dividing a single pre-adjusted video frame. The sub-regions corresponding to each rendering level are rendered sequentially in descending order of the level reference value.

[0145] Specifically, in the regional rendering resolution adjustment, the actual rendering resolution of each edge sub-region is reduced based on the abnormal coupling degree.

[0146] The reduction in the actual rendering resolution of each edge sub-region is positively correlated with the abnormal coupling degree.

[0147] Among them, the edge sub-region is the sub-region whose region pixel change value is less than the preset region pixel change value;

[0148] The value of the preset region pixel change value can be determined by the user according to the actual application scenario. The greater the user's demand for improving image processing quality, the smaller the value of the preset region pixel change value should be. A method for determining the preset region pixel change value is provided, which detects the historical records of regional rendering resolution adjustment and records the average value of the region pixel change value corresponding to each edge sub-region in the historical records that can meet the user's needs as the preset region pixel change value.

[0149] The abnormal coupling degree corresponding to a single pre-adjusted video frame = abnormal coefficient / (number of edge sub-regions / total number of sub-regions).

[0150] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. An image grading processing method for graphics cards, characterized in that, include: Obtain two-dimensional image data corresponding to each video frame in the target video, and determine whether to perform image processing optimization for the target video based on the smoothness of the picture and the reference value of the graphics card load. When performing image processing optimization, the initial rendering resolution of each video frame image is determined based on the complex evaluation threshold of the video frame image, and the video frame images are uniformly or correlatedly combined according to the number of mutated frames and the degree of mutation influence to obtain a combination of several video frame images. Wherein, the mutation impact is the standard deviation of the rendering mutation value corresponding to each mutation frame, and the rendering mutation value corresponding to the target image to be rendered is = |the initial rendering resolution corresponding to the target image to be rendered - the initial rendering resolution of the neighboring images corresponding to the target image to be rendered|. Based on the dynamic mutation coefficient and mutation duration of each video frame image combination, continuous inter-frame blending or adjacent inter-frame blending is performed on each mutated frame to determine the actual rendering resolution of each mutated frame. The number of buffered frames is determined based on the evaluation reference value, and the change complexity is determined based on the rendering change degree corresponding to each mutated frame in the buffered frames. Based on the change complexity corresponding to each buffered frame and the loading deviation threshold, it is determined whether to adjust the graphics card refresh rate according to the evaluation threshold. Under conditions of graphics card instability, the pre-adjusted video frame is determined based on the abnormal correlation, and the region rendering level or region rendering resolution is adjusted for the pre-adjusted video frame based on the abnormal coefficient.

2. The image grading processing method for a graphics card according to claim 1, characterized in that, If the smoothness of the video is less than the preset smoothness or the graphics card load reference value is greater than or equal to the preset graphics card load reference value, then image optimization processing will be performed on the target video. If the smoothness of the video is greater than or equal to the preset smoothness and the graphics card load reference value is less than the preset graphics card load reference value, then no image optimization processing is required for the target video.

3. The image grading processing method for a graphics card according to claim 2, characterized in that, The initial rendering resolution of each video frame image is determined based on a complex evaluation threshold. The initial rendering resolution of a single video frame is negatively correlated with the complexity evaluation threshold of that video frame.

4. The image grading processing method for a graphics card according to claim 3, characterized in that, If the number of mutated frames is greater than or equal to the preset number of mutated frames or the degree of mutation is greater than or equal to the preset degree of mutation, then the images of each video frame are associated and combined. If the number of mutated frames is less than the preset number of mutated frames and the impact of the mutation is less than the preset impact of the mutation, then the images of each video frame are uniformly combined.

5. The image grading processing method for a graphics card according to claim 1, characterized in that, For a single video frame image combination, If the dynamic mutation coefficient is greater than or equal to the preset dynamic mutation coefficient or the mutation duration is greater than or equal to the preset mutation duration, then continuous inter-frame mixing is performed for each mutated frame in the video frame image combination. If the dynamic mutation coefficient is less than the preset dynamic mutation coefficient and the mutation duration is less than the preset mutation duration, then neighboring frames are mixed for each mutated frame in the video frame image combination.

6. The image grading processing method for a graphics card according to claim 1, characterized in that, The number of buffer frames is determined based on the evaluation reference value; The number of buffer frames is positively correlated with the evaluation reference value.

7. The image grading processing method for a graphics card according to claim 1, characterized in that, If the loading deviation threshold for each buffered frame is greater than the preset loading deviation threshold or the change complexity is greater than the preset change complexity, the refresh rate of the graphics card will be reduced based on the evaluation threshold. The decrease in the graphics card refresh rate is positively correlated with the evaluation threshold.

8. The image grading processing method for a graphics card according to claim 1, characterized in that, For a single pre-adjusted video frame, If the anomaly coefficient is less than the preset anomaly coefficient, then the region rendering level is set for the pre-adjusted video frame. If the anomaly coefficient is greater than or equal to the preset anomaly coefficient, then the region rendering resolution is adjusted for the pre-adjusted video frame.

9. The image grading processing method for a graphics card according to claim 7, characterized in that, In the regional rendering level settings, the region is divided into several sub-regions based on the pixel fluctuation value of the pre-adjusted video frame. The number of rendering levels is determined based on the anomaly coefficient, and the sub-regions corresponding to each rendering level are determined based on the regional pixel change value. The loading deviation is used to determine whether to increase the number of rendering levels. If the loading deviation is greater than the preset loading deviation, the number of rendering levels will be increased. The increase in the number of rendering levels is positively correlated with the loading deviation.

10. The image grading processing method for a graphics card according to claim 7, characterized in that, In the regional rendering resolution adjustment, the actual rendering resolution of each edge sub-region is reduced based on the abnormal coupling degree. The reduction in the actual rendering resolution of each edge sub-region is positively correlated with the abnormal coupling degree.

Images