A video conversion method and related device

Abstract

The present invention provides a video conversion method and related devices, which relate to the field of computer vision. Determine the brightness amplitude channel image corresponding to the current frame image in the first type of video, perform a global enhancement operation on it to obtain a global enhanced image, perform edge-preserving filtering on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement image, perform de-flickering on the local highlight enhancement image to obtain a de-flickering local highlight enhancement image, enhance the de-flickering local highlight enhancement image to obtain a local enhanced image, obtain an enhanced image based on the global enhanced image and the local enhanced image, perform a highlight suppression operation on the enhanced image to obtain a video frame of the second type of video, and achieve the goal of converting the first type of video to the second type of video. In addition, the flickering problem caused by enhancement is solved by de-flickering, and the problem of excessive brightness and glare caused by local enhancement is avoided by highlight suppression operation, thereby improving the viewing experience.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and more particularly to a video conversion method and related devices. Background Art

[0002] As people's requirements for video quality gradually increase, various types of videos continue to emerge, such as HDR (High Dynamic Range) technology, SDR (Standard Dynamic Range), etc.

[0003] In practical applications, it may be necessary to convert one type of video into another type of video to improve the viewing experience of the video. So how to convert one type of video into another type of video has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the present invention provides a video conversion method and related devices to solve the problem of converting one type of video into another type of video.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A video conversion method, comprising:

[0007] Determine a brightness amplitude channel image corresponding to a current frame image in a first type of video;

[0008] Performing a global enhancement operation on the brightness amplitude channel image to obtain a global enhanced image;

[0009] Performing edge-preserving filtering processing on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement image, and performing de-flickering processing on the local highlight enhancement image to obtain a de-flickering local highlight enhancement image;

[0010] enhancing the de-flickering local highlight enhancement image to obtain a local enhanced image, and obtaining an enhanced image based on the global enhanced image and the local enhanced image;

[0011] A highlight suppression operation is performed on the enhanced image to obtain a video frame of a second type of video.

[0012] Optionally, performing a global enhancement operation on the brightness amplitude channel image to obtain a global enhanced image includes:

[0013] Performing scene analysis on the brightness amplitude channel image to obtain an amplitude intensity histogram;

[0014] Determining a global enhancement parameter corresponding to the amplitude intensity histogram;

[0015] Constructing a global enhancement curve corresponding to the global enhancement parameter;

[0016] The global enhancement curve is used to perform global enhancement processing on the brightness amplitude channel image to obtain a global enhanced image.

[0017] Optionally, edge-preserving filtering is performed on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement map, including:

[0018] Determine a partial area in the brightness amplitude channel image where the brightness component is greater than a preset threshold as a local highlight area;

[0019] An edge-preserving filtering process is performed on the local highlight area to obtain a local highlight enhancement map.

[0020] Optionally, performing a de-flickering process on the local highlight enhancement image to obtain a de-flickering local highlight enhancement image includes:

[0021] Obtain the local highlight enhancement image corresponding to the previous frame image and the local highlight enhancement image after flicker removal;

[0022] Downsampling and binarizing the local highlight enhancement image corresponding to the current frame image and the local highlight enhancement image corresponding to the previous frame image respectively to obtain a first enhancement image and a second enhancement image, and downsampling the de-flickering local highlight enhancement image corresponding to the previous frame image to obtain a historical local highlight enhancement image corresponding to the previous frame image;

[0023] Determine an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image based on the first enhancement image, the second enhancement image, and the historical local highlight enhancement image corresponding to the previous frame image;

[0024] For the local highlight enhancement map corresponding to the current frame image, the actual mean variance is used as the distribution and normalized to the target mean variance to obtain the local highlight enhancement map after flicker removal.

[0025] Optionally, determining an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image based on the first enhancement image, the second enhancement image, and a historical local highlight enhancement image corresponding to the previous frame image includes:

[0026] Calculating an intersection area of ​​the first enhancement map and the second enhancement map;

[0027] Determine basic contour units corresponding to the first enhancement map and the second enhancement map respectively;

[0028] Matching the basic contour units corresponding to the first enhancement graph and the basic contour units corresponding to the second enhancement graph with the basic contour units in the intersection area respectively, to obtain the common contour units of the first enhancement graph and the second enhancement graph;

[0029] Determine matching contour units of the first enhancement image and the second enhancement image based on a comparison result of coordinates in the basic contour units in the intersection area and coordinates in the common contour units;

[0030] Performing pixel filling operations on matching contour units in the first enhanced image and the second enhanced image respectively to obtain a matching contour binary image corresponding to the first enhanced image and a matching contour binary image corresponding to the second enhanced image;

[0031] Calculate a first mean variance of the amplitude of a region corresponding to the matching contour binary image corresponding to the first enhanced image in the downsampled local highlight enhanced image corresponding to the current frame image;

[0032] Calculate a second mean variance of the amplitude of the area corresponding to the matching contour binary image corresponding to the second enhanced image in the historical local highlight enhanced image corresponding to the previous frame image;

[0033] Based on the first mean variance and the second mean variance, an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image are determined.

[0034] Optionally, after calculating the second mean variance of the amplitude of the area corresponding to the matching contour binary image corresponding to the second enhanced image in the historical local highlight enhanced image corresponding to the previous frame image, the method further includes:

[0035] If the first mean variance satisfies the correction rule, a correction operation is performed on the first mean variance to obtain a corrected first mean variance, and if the second mean variance satisfies the correction rule, a correction operation is performed on the second mean variance to obtain a corrected second mean variance;

[0036] Based on the corrected first mean variance and the corrected second mean variance, an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image are determined.

[0037] Optionally, obtaining an enhanced image based on the global enhanced image and the local enhanced image includes:

[0038] A pixel-by-pixel dot product operation is performed on the global enhanced image and the local enhanced image to obtain an enhanced image.

[0039] Optionally, performing a highlight suppression operation on the enhanced image to obtain a video frame of a second type of video includes:

[0040] Generate a highlight suppression map corresponding to a target fixed area in the enhanced image; the target fixed area is an area where specified data exists;

[0041] Performing a linear low-pass filtering operation on the highlight suppression map to obtain a smoothed highlight suppression map;

[0042] Mapping the smoothed highlight suppression map into a preset range to obtain a mapped highlight suppression map;

[0043] Performing a pixel-by-pixel dot product operation on the mapped highlight suppression map and the enhanced image to obtain a target video frame;

[0044] The target video frame is converted into a video frame of a second type of video.

[0045] A video conversion device, comprising:

[0046] An image determination module, used to determine a brightness amplitude channel image corresponding to a current frame image in a first type of video;

[0047] A global enhancement module, used for performing a global enhancement operation on the brightness amplitude channel image to obtain a global enhanced image;

[0048] An image processing module, configured to perform edge-preserving filtering on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement image, and perform de-flickering processing on the local highlight enhancement image to obtain a de-flickering local highlight enhancement image;

[0049] A local enhancement module, used for enhancing the local highlight enhancement image after flicker removal to obtain a local enhanced image, and obtaining an enhanced image based on the global enhanced image and the local enhanced image;

[0050] The highlight suppression module is used to perform a highlight suppression operation on the enhanced image to obtain a video frame of a second type of video.

[0051] An electronic device comprises at least one processor and a memory connected to the processor, wherein:

[0052] The memory is used to store computer programs;

[0053] The processor is used to execute the computer program so that the electronic device can implement the above-mentioned video conversion method.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention provides a video conversion method and related devices. In the present invention, the brightness amplitude channel image corresponding to the current frame image in the first type of video is determined, the brightness amplitude channel image is globally enhanced to obtain a globally enhanced image, the local highlight area in the brightness amplitude channel image is edge-preserving filtered to obtain a local highlight enhancement image, the local highlight enhancement image is de-flickered to obtain a de-flickered local highlight enhancement image, the de-flickered local highlight enhancement image is enhanced to obtain a local enhanced image, based on the global enhanced image and the local enhanced image, an enhanced image is obtained, and a highlight suppression operation is performed on the enhanced image to obtain a video frame of a second type of video, thereby achieving the goal of converting the first type of video into the second type of video. In addition, in the present invention, the flicker problem caused by enhancement can be effectively solved by de-flickering. In addition, by performing a highlight suppression operation on the enhanced image, the problem of excessive brightness and glare caused by local enhancement can be avoided, and the viewing experience of the converted second type of video can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0057] Figure 1 A flowchart of a video conversion method provided by an embodiment of the present invention;

[0058] Figure 2 A flowchart of a global enhancement method provided by an embodiment of the present invention;

[0059] Figure 3 A flowchart of a flicker removal method provided by an embodiment of the present invention;

[0060] Figure 4 A flow chart of a variance calculation method provided by an embodiment of the present invention;

[0061] Figure 5 A flowchart of another flicker removal method provided by an embodiment of the present invention;

[0062] Figure 6 A flowchart of another video conversion method provided by an embodiment of the present invention;

[0063] Figure 7 A schematic diagram of the structure of a video conversion device provided by an embodiment of the present invention;

[0064] Figure 8A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] As people's requirements for video quality gradually increase, various types of videos continue to emerge, such as HDR (High Dynamic Range) technology. With the support of HDR technology and HDR display devices, HDR videos have better brightness, contrast and color than SDR (Standard Dynamic Range) videos, significantly improving the video viewing experience.

[0067] In practical applications, it may be necessary to convert one type of video into another type of video to improve the viewing experience of the video. For example, due to the cost of shooting HDR videos, the current video format is still mainly SDR video. So, how to convert SDR videos into HDR videos at low cost to improve the viewing experience of the video has become a technical problem that needs to be solved urgently. It should be noted that the specific implementation can be introduced later by taking the conversion of SDR videos into HDR videos as an example, but the present invention is not limited to the scenario of converting SDR videos into HDR videos, but can also be other ordinary videos converted into HDR videos or SDR videos.

[0068] With the development of data-driven deep learning technology, it has surpassed traditional processing methods in more and more fields. However, due to the lack of high-quality HDR data for different scenes, it is difficult for the SDR to HDR algorithm based on deep learning to achieve the best effect and generalization. In addition, the operating efficiency of the deep learning method is higher on the GPU (graphics processing unit), but lower on the CPU (Central Processing Unit), which is not suitable for scenes that use the CPU but have high real-time requirements.

[0069] In addition, traditional SDR to HDR algorithms can also be used for video conversion. However, traditional SDR to HDR algorithms mostly use fixed curves or parameters to convert SDR videos, which makes the algorithm's scene adaptability poor.

[0070] At the same time, since one of the biggest advantages of HDR video over SDR video is that it has a larger brightness range, how to enhance the local highlight details of SDR video has become one of the most important steps in converting SDR to HDR. At present, most methods enhance local highlights based on edge-preserving filtering and implement real-time applications through fast edge-preserving filtering algorithms. However, most of these algorithms are designed for natural images. When subtitles appear in SDR video, due to the change in pixel distribution between frames, some edge-preserving filters based on fast algorithms will flicker after enhancement.

[0071] In addition, the brightness of the text area may be over-enhanced, causing glare, which seriously affects the HDR video viewing experience.

[0072] To this end, in the present invention, when performing global enhancement, a scene analysis is performed on the Y channel image in the linear space to obtain the Y channel component amplitude intensity histogram statistics in the linear space, and then a global enhancement operation is performed, taking into account different needs in different scenes, and then based on the global brightness and color optimization algorithm of the scene information, the absolute brightness of the HDR video can be adaptively adjusted, making up for the lack of scene adaptability of traditional algorithms.

[0073] In addition, in the present invention, a local highlight enhancement algorithm based on nonlinear filtering is used, and assembly and multi-thread acceleration are used, so that the algorithm can run in real time on a CPU with limited computing power and with high efficiency.

[0074] In addition, in the present invention, an inter-frame flicker removal algorithm based on contour matching is adopted to solve the flicker problem caused by nonlinear filtering in some scenes.

[0075] In addition, the present invention proposes a subtitle and station logo area brightness protection algorithm, which effectively solves the problem of excessive brightness and glare in the subtitle and station logo area, ensures the high quality of HDR video, and improves the viewing experience of HDR video.

[0076] In addition, the present invention can implement the above algorithm through software, without the need for additional hardware configuration, and the cost is low.

[0077] Based on the above content, an embodiment of the present invention provides a video conversion method, and the execution subject may be an electronic device such as a controller and a server.

[0078] Reference Figure 1 , the video conversion method may include:

[0079] S11. Determine a brightness amplitude channel image corresponding to a current frame image in a first type of video.

[0080] In this embodiment, the object to be processed is a first type of video, and the first type of video may be an SDR video. Taking an SDR video as an example, the SDR video includes multiple frame images, and the frame image processed in the current cycle is referred to as a current frame image, and the frame image processed in the previous cycle is referred to as a previous frame image.

[0081] Since the brightness of the frame image in the SDR video needs to be processed, and the brightness amplitude channel image, that is, the Y channel image, represents the intensity amplitude of the brightness component in the linear space, it is necessary to determine the Y channel image corresponding to the current frame image in the SDR video.

[0082] Specifically, FFmpeg (Fast Forward Mpeg) is used to decode the SDR video in the BT.601 or BT.709 color gamut into an RGB image in a non-linear space, and a gamma transform is performed on the RGB image to obtain an RGB image in a linear space and a Y channel image in a linear space.

[0083] Among them, the RGB image in the linear space represents the intensity amplitude of the red, green, and blue components in the linear space respectively, and the Y channel image in the linear space represents the intensity amplitude of the brightness component in the linear space. The specific calculation formula is as follows:

[0084]

[0085]

[0086] Among them, rgb represents the RGB image in nonlinear space, Represents an RGB image in linear space, , , Represents the components of its red, green, and blue channels. , and is the coefficient, Represents the Y channel image in linear space. When the SDR video is in BT.601 color space, When the SDR video is in BT.709 color space .

[0087] S12. Perform a global enhancement operation on the brightness amplitude channel image to obtain a global enhanced image.

[0088] In this embodiment, the overall brightness, contrast, and color of the image are adaptively optimized through a global enhancement operation, that is, the global light and color effects of the converted HDR video are optimized to prevent extreme effects of being too dark or glaring in different scenes, and the saturation is adaptively adjusted according to the brightness change to prevent oversaturation or undersaturation.

[0089] S13, performing edge-preserving filtering processing on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement image, and performing de-flickering processing on the local highlight enhancement image to obtain a de-flickering local highlight enhancement image.

[0090] After global enhancement, in order to effectively preserve the edge information in the image during the filtering process, an edge-preserving filtering operation is performed. When performing the edge-preserving filtering operation, the local highlight area in the Y channel image is first determined, and then an edge-preserving filtering algorithm, such as the bilateral grid filtering algorithm, is used to perform edge-preserving filtering processing to obtain a local highlight enhancement image.

[0091] The bilateral grid filtering algorithm is a fast edge-preserving filtering algorithm, which may cause the pixel value of the highlight in the frame image with subtitles to become higher, so that when jumping from the frame image without subtitles to the frame image with subtitles, there will be a flickering problem after the local enhancement process, and the user experience is poor. Therefore, in this embodiment, the local highlight enhancement image is de-flickered to avoid the impact of the flickering problem on the user experience.

[0092] When performing flicker removal, an inter-frame flicker removal algorithm based on contour matching is used to effectively solve the flicker problem caused by enhancement.

[0093] S14, enhancing the de-flickering local highlight enhancement image to obtain a local enhanced image, and obtaining an enhanced image based on the global enhanced image and the local enhanced image.

[0094] In this embodiment, the HDR local highlight enhancement algorithm is used to adaptively enhance the local highlight details in the local highlight enhancement image after flickering, thereby obtaining a local enhanced image.

[0095] Then, a pixel-by-pixel dot product operation is performed on the global enhanced image and the local enhanced image to obtain an enhanced image. Here, since global enhancement and local enhancement are considered at the same time, the enhanced image obtained will have a better enhancement effect.

[0096] S15. Perform a highlight suppression operation on the enhanced image to obtain a video frame of a second type of video.

[0097] In this embodiment, a highlight suppression strategy is proposed to address the problem of excessive brightness and glare in areas such as video subtitles and station logos due to local enhancement. The highlight suppression strategy is used to perform highlight suppression operations on the enhanced image, effectively improving the viewing experience of the second type of video after conversion.

[0098] The video frames of the second type of video finally obtained can constitute the second type of video. The second type of video can be, for example, an HDR video, and the user can play the HDR video on a corresponding HDR display device.

[0099] In this embodiment, the brightness amplitude channel image corresponding to the current frame image in the first type of video is determined, a global enhancement operation is performed on the brightness amplitude channel image to obtain a global enhanced image, an edge-preserving filtering process is performed on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement map, a de-flickering process is performed on the local highlight enhancement map to obtain a de-flickering local highlight enhancement map, the de-flickering local highlight enhancement map is enhanced to obtain a local enhanced image, an enhanced image is obtained based on the global enhanced image and the local enhanced image, a highlight suppression operation is performed on the enhanced image to obtain a video frame of the second type of video, thereby achieving the goal of converting the first type of video to the second type of video. In addition, in the present invention, the flickering problem caused by enhancement can be effectively solved by de-flickering. In addition, by performing a highlight suppression operation on the enhanced image, problems such as excessive brightness and glare caused by local enhancement can be avoided, and the viewing experience of the converted second type of video can be effectively improved.

[0100] The above embodiment mentions that a global enhancement operation is performed on the brightness amplitude channel image to obtain a global enhanced image. Now, its specific implementation is introduced. Figure 2 “Performing a global enhancement operation on the brightness amplitude channel image to obtain a global enhanced image” may include:

[0101] S21. Perform scene analysis on the brightness amplitude channel image to obtain an amplitude intensity histogram.

[0102] In practical applications, different scenes have different requirements for HDR video, which results in different conversion methods for SDR video. In the embodiments of the present invention, through scene analysis operations, the requirements for HDR video in different scenes are taken into account to meet the purpose of obtaining the required HDR video in different scenes.

[0103] Specifically, a scene analysis is performed on the Y channel image in the linear space to obtain an amplitude intensity histogram of the Y channel brightness component in the linear space. The more distributions on the right side of the amplitude intensity histogram, the larger the amplitude, and vice versa.

[0104] S22: Determine a global enhancement parameter corresponding to the amplitude intensity histogram.

[0105] Specifically, if the video is converted into an HDR video, the global enhancement parameter in this embodiment is the HDE global enhancement parameter. The HDR global enhancement table is indexed using the histogram statistical distribution of the amplitude intensity histogram to obtain the HDR global enhancement parameter.

[0106] The HDR global enhancement table is obtained by pre-modeling different types of HDR videos. The HDR global enhancement parameters in the HDR global enhancement table include . Subsequently, the global enhancement curve can be constructed through the HDR global enhancement parameters (specifically, it can be such as the HDR global enhancement curve) to achieve the Y channel image in the linear space The enhancement of the global enhanced Y channel image is obtained .

[0107] in, Control the HDR global enhancement curve to adjust dark area details, mid-bright details and global highlight gain respectively. The larger it is, the more dark and medium-bright details it corresponds to, and the greater the global highlight gain, and vice versa. , The smaller, The larger it is, the smaller the dark area details and the global highlight gain will be, and the larger the mid-light details will be, and vice versa.

[0108] Control the curvature of the HDR global enhancement curve in the dark and bright areas respectively. The smaller it is, the greater the rate of change of the dark area, and vice versa. The smaller it is, the smaller the rate of change of the bright area is, and vice versa. The smaller it is, the more the amplitude intensity histogram is distributed on the right side. The bigger the smaller, and vice versa.

[0109] Respectively control the inflection point of the HDR global enhancement curve in the medium-bright range and the offset coefficient of the inflection point. When the scene is brighter, the more the amplitude intensity histogram is distributed on the right side, the need to reduce global highlight enhancement to prevent the image from being too bright, while reducing dark enhancement to prevent the overall artistic expression of the image from being destroyed, and optimize the overall contrast by appropriately improving the medium-bright details, reducing the dark area change rate, and increasing the bright area change rate. In actual applications, when the amplitude intensity histogram is distributed more on the left side, When the amplitude intensity histogram is mostly distributed on the right side, begins to play a role in further regulating t. The adjustment of the dark area brightness can achieve more precise control of the dark area brightness. When the scene is darker, it is necessary to enhance the dark area and appropriately enhance the global highlights to prevent the converted image from being too dark, resulting in loss of dark area details. At the same time, the overall contrast is optimized by increasing the dark area change rate and reducing the bright area change rate. Controls the absolute brightness of the enhanced HDR. When the amplitude intensity histogram is more distributed on the left side, The bigger it is, the smaller it is.

[0110] According to the above principle, the HDR global enhancement table is indexed using the histogram statistical distribution to obtain the HDR global enhancement parameters.

[0111] S23. Construct a global enhancement curve corresponding to the HDR global enhancement parameter.

[0112] In this embodiment, taking the second type of video as HDR as an example, after obtaining the HDR global enhancement parameter, the HDR global enhancement parameter will correspondingly control the corresponding content in the HDR global enhancement curve, such as The HDR global enhancement curve is controlled to regulate the dark area details, medium-bright details and global highlight gain respectively. Therefore, the HDR global enhancement curve can be obtained by using the correspondence between the HDR global enhancement parameters and the HDR global enhancement curve.

[0113] S24. Perform global enhancement processing on the brightness amplitude channel image using the global enhancement curve to obtain a globally enhanced image.

[0114] Specifically, after obtaining the HDR global enhancement curve, the HDR global enhancement curve can be used to implement the Y channel image in the linear space. The enhancement of the Y channel image is obtained globally. , where the global enhanced Y channel image This is the global enhanced image in this embodiment.

[0115] Furthermore, you can also get the global enhanced Y channel image And the Y channel image in linear space Rate of change , used for subsequent adaptive adjustment of HDR color saturation.

[0116] Finally, the Y channel image is globally enhanced Map to range to meet the absolute brightness space requirements of HDR video. , .

[0117] It should be noted that the HDR global enhancement parameters in this embodiment are only one embodiment of the present invention. In addition, more HDR enhancement tables can be modeled to achieve smoother control of the HDR global enhancement parameters. In addition, more HDR global enhancement parameters can be constructed to achieve better global enhancement effects and improve the adaptability to scenes.

[0118] In this embodiment, global brightness and color optimization is performed based on scene information, and the absolute brightness of the second type of video can be adaptively adjusted, which makes up for the lack of scene adaptability of traditional algorithms.

[0119] In another implementation of the present invention, when edge-preserving filtering is performed on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement map, the following steps are specifically included:

[0120] 1) Determine a partial area in the brightness amplitude channel image whose brightness component is greater than a preset threshold as a local highlight area.

[0121] Specifically, a basic threshold value needs to be preset and configured, and the basic threshold value is set based on experience or obtained through experimental statistics. The preset threshold value in this embodiment is the threshold value converted from the basic threshold value to the linear space through gamma transformation.

[0122] If it is a Y channel image in linear space In the example, if the brightness component of a certain area is greater than the preset threshold, it means that the current brightness is relatively high, and the area is the local highlight area in this embodiment.

[0123] 2) Performing edge-preserving filtering on the local highlight area to obtain a local highlight enhancement map.

[0124] Specifically, the edge-preserving filtering is implemented by using the above-mentioned bilateral grid filtering algorithm or other fast edge-preserving filtering algorithms.

[0125] In order to improve processing efficiency, the bilateral grid filter can be assembled and multi-threaded optimized so that it can run efficiently and in real time on the CPU.

[0126] After the edge-preserving filtering algorithm is determined, the edge-preserving filtering process is performed using the algorithm to guide the detail enhancement of local highlights, so that the edge information in the image can be effectively preserved during the filtering process.

[0127] In another embodiment of the present invention, since subtitles may suddenly appear in two consecutive video frames in an SDR video, the amplitude intensity distribution of the Y channel image in the linear space is changed, so that the amplitude intensity of the local highlight enhancement image obtained after edge-preserving filtering of two consecutive frames of images with subtitles and images without subtitles is suddenly changed, resulting in a difference in brightness between the two frames after local highlight enhancement, thereby causing inter-frame flicker. Therefore, it is necessary to further perform a de-flickering process on the local highlight enhancement image obtained from two consecutive frames.

[0128] Specifically, refer to Figure 3 “Performing a de-flickering process on the local highlight enhancement image to obtain a de-flickering local highlight enhancement image” may include:

[0129] S31, obtaining a local highlight enhancement image corresponding to a previous frame of image and a local highlight enhancement image after flicker removal.

[0130] Since each frame image in the SDR video is processed in sequence according to a cycle, the frame image processed in the previous cycle is called the previous frame image.

[0131] In this embodiment, since it is necessary to perform a de-flickering process on the local highlight enhancement images obtained from two consecutive frames, it is necessary to obtain the local highlight enhancement image corresponding to the previous frame image and the local highlight enhancement image after de-flickering.

[0132] make They respectively represent the local highlight enhancement image of the current frame image, the local highlight enhancement image of the previous frame image, and the local highlight enhancement image of the previous frame image after flickering is removed.

[0133] S32. Down-sample and binarize the local highlight enhancement image corresponding to the current frame image and the local highlight enhancement image corresponding to the previous frame image, respectively, to obtain a first enhancement image and a second enhancement image, and down-sample the de-flickering local highlight enhancement image corresponding to the previous frame image to obtain a historical local highlight enhancement image corresponding to the previous frame image.

[0134] In this embodiment, the local highlight enhancement image corresponding to the current frame image and the local highlight enhancement image corresponding to the previous frame image are downsampled respectively, and the local highlight enhancement image after flickering corresponding to the previous frame image is downsampled. Downsampling can reduce the resolution to increase the subsequent processing speed.

[0135] In addition, it is necessary to Binarization is performed to enhance the outline of the image. It should be noted that only The result after binarization, therefore, can be only Binarization is performed, where the downsampled Binarization is performed to obtain the first enhanced image. Binarization is performed to obtain a second enhanced image.

[0136] S33. Determine an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image based on the first enhancement image, the second enhancement image, and the historical local highlight enhancement image corresponding to the previous frame image.

[0137] In this embodiment, after the first enhancement image, the second enhancement image, and the historical local highlight enhancement image corresponding to the previous frame image are determined in step S32, the actual mean variance and the target mean variance of the local highlight enhancement image corresponding to the current frame image can be determined using these three data. Figure 4 , the specific implementation may include the following steps:

[0138] S41: Calculate an intersection area of ​​the first enhancement map and the second enhancement map.

[0139] Specifically, the binarized The intersection area The intersection area in this embodiment refers to the area without subtitles in common between the previous frame image and the current frame image. This is the first enhanced image, after binarization This is the second enhanced image.

[0140] In practical applications, for example, the content of the previous frame image is two people eating and the two people are not talking, and the content of the current frame image is two people eating and one of them is explaining. At this time, there are subtitles in the current frame image, then the intersection area That is, the same area without subtitles in the two frame images.

[0141] S42: Determine basic contour units corresponding to the first enhancement map and the second enhancement map respectively.

[0142] Specifically, calculate the binarized The outermost contour and coordinates are obtained, and the minimum enclosing rectangle of each outermost contour is obtained. Each independent outermost contour and its corresponding minimum enclosing rectangle constitute a basic contour unit. There will be multiple basic contour units in each.

[0143] S43: Match the basic contour units corresponding to the first enhancement graph and the basic contour units corresponding to the second enhancement graph with the basic contour units in the intersection area respectively to obtain the common contour units of the first enhancement graph and the second enhancement graph.

[0144] Specifically, the binarized The basic contour units are respectively The basic contour units are matched to find the binarized The common basic contour unit is called the common contour unit.

[0145] In the matching process, each time the binarized and A basic contour unit and All the basic contour units in are matched, because and A basic contour unit in may be Multiple basic contour units in the match, so it is necessary to All basic contour elements in the The basic contour units that have been matched in . And if the binarized or The basic contour unit and The basic contour units in or The minimum bounding rectangle of the basic contour unit must contain The minimum bounding rectangle of the basic contour unit is matched. This method can quickly exclude the basic contour units that do not meet the conditions.

[0146] In this embodiment, the intersection area is used , find the binarized The common contour unit matched in The same area is found for the subsequent de-blinking operation.

[0147] S44: Determine matching contour units of the first enhancement image and the second enhancement image based on a comparison result of coordinates in the basic contour units in the intersection area and coordinates in the common contour units.

[0148] In practical applications, the binarized The following common contour elements can be matched:

[0149] 1. After binarization A basic contour unit in Multiple basic contour units in form a common contour unit.

[0150] 2. After binarization A basic contour unit in Multiple basic contour units in form a common contour unit.

[0151] That is, there is a problem of one-to-many matching. To solve this problem, the embodiment of the present invention performs further screening operations based on coordinates. Specifically, by judging Are the coordinates of the basic contour units in the binarized or In the basic contour unit, that is, to determine whether it is located in the common contour unit. If the condition is met, it means that the two basic contour units match, that is, the corresponding common contour unit is correct and needs to be retained. Finally, if The basic contour unit and The basic contour unit matches, The corresponding common contour unit is a matching contour unit shared by both.

[0152] S45 , performing pixel filling operations on the matching contour units in the first enhanced image and the second enhanced image respectively to obtain a matching contour binary image corresponding to the first enhanced image and a matching contour binary image corresponding to the second enhanced image.

[0153] Specifically, after getting the binarization After all the matching contour units are processed, the binarized Fill the matching contour unit to obtain the matching contour binary image .

[0154] Among them, matching contour binary image is the matching contour binary image corresponding to the first enhanced image, matching contour binary image It is the matching contour binary image corresponding to the second enhanced image.

[0155] In this embodiment, pixel filling operation is performed on the matching contour units respectively because there is only one contour in the matching contour unit and there is no corresponding pixel value. Therefore, pixel values ​​of 1 or 255 can be added to the pixels therein to complete the pixel filling operation.

[0156] S46, calculating a first mean variance of the amplitude of a region corresponding to the matching contour binary image corresponding to the first enhanced image in the downsampled local highlight enhanced image corresponding to the current frame image.

[0157] Specifically, calculate the above down-sampling exist The mean variance of the amplitude at . That is the first mean variance in this embodiment.

[0158] Specifically, from the downsampled local highlight enhancement map corresponding to the current frame image, that is, the downsampled In, find exist , get the amplitude in the area, and then calculate the mean variance of the amplitude , and get the first mean variance.

[0159] S47, calculating a second mean variance of the amplitude of the area corresponding to the matching contour binary image corresponding to the second enhanced image in the historical local highlight enhanced image corresponding to the previous frame image.

[0160] Specifically, calculate the downsampled exist The mean variance of the amplitude at , and get the second mean variance.

[0161] The specific implementation of step S47 is the same as step S46, please refer to the corresponding instructions above.

[0162] S48. Determine an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image based on the first mean variance and the second mean variance.

[0163] Specifically, after obtaining the first mean variance and the second mean variance Then, the actual mean variance of the local highlight enhancement image corresponding to the current frame image can be calculated. and the target mean variance .

[0164] Among them, the actual mean variance Refers to the actual mean variance of the local highlight enhancement image corresponding to the current frame image, the target mean variance It refers to the theoretical mean variance of the local highlight enhancement map corresponding to the current frame image.

[0165] If the current frame image is matched for the first time, the actual mean variance and target mean variance To initialize, , .

[0166] If it is not the first match, the actual mean variance Variance with target mean By the coefficient of variation To regulate. , whose size is determined by The intersection-over-combination ratio and the aforementioned downsampling The mean-variance change ratio of is jointly determined. The more hours, The closer , The closer , on the contrary, the closer it is to its original value.

[0167] Based on this embodiment, in another implementation of the present invention, there may be a first mean variance calculated and the second mean variance Inaccurate problem, at this time, the first mean variance needs to be and the second mean variance Perform corrective actions.

[0168] First mean variance For example, in order to verify the first mean variance Whether it is accurate, the corresponding correction rules (also called error correction conditions) are set. The specific contents of the correction rules are:

[0169] If the above downsampling exist The sum of the amplitudes at is greater than the above downsampling exist The sum of the amplitudes at The number of non-zero amplitudes is greater than that after downsampling The number of amplitudes that are not 0, but Less than , then Make corrections.

[0170] In this embodiment, if the correction rule is satisfied, it means that there are a large number of amplitudes close to 0, which affects the expected mean variance. At this time, the data is unreasonable and needs to be corrected or rectified.

[0171] Then, after calculating the second mean variance of the amplitude of the area corresponding to the matching contour binary image corresponding to the second enhancement image in the historical local highlight enhancement image corresponding to the previous frame image, if the first mean variance satisfies the correction rule, the first mean variance is corrected to obtain the corrected first mean variance, and, if the second mean variance satisfies the correction rule, the second mean variance is corrected to obtain the corrected second mean variance.

[0172] First mean variance Take this as an example to introduce the correction or rectification process:

[0173] Set multiple given thresholds th, such as th is respectively Every value in .

[0174] During correction, by filtering out the downsampled exist When the amplitude is less than the given threshold th, the mean variance is recalculated. After the mean variance corresponding to each given threshold th is calculated, the average of these mean variances is taken as the new .

[0175] Second mean variance The same method should be used for correction.

[0176] After obtaining the corrected first mean variance and the corrected second mean variance, determining the actual mean variance of the local highlight enhancement image corresponding to the current frame image based on the corrected first mean variance and the corrected second mean variance and the target mean variance , its specific implementation refers to the above corresponding instructions.

[0177] It should be noted that if the first mean variance and the second mean variance If only one of them needs to be corrected, then only the one that needs to be corrected is corrected, and then the actual mean variance is recalculated and the target mean variance .

[0178] S34. For the local highlight enhancement map corresponding to the current frame image, the actual mean variance is used as distribution, and normalized to the target mean variance to obtain the local highlight enhancement map after flicker removal.

[0179] Specifically, by As a distribution, normalized to the target mean variance , and get the local highlight enhancement image after flicker removal .

[0180] In this embodiment, by As a distribution, normalized to the target mean variance , so that the local highlight enhancement map of the current frame image The amplitude intensity and the local highlight enhancement map of the previous frame image The amplitude intensity has the same distribution, thus achieving the purpose of flicker removal.

[0181] It should be noted that the overall process of de-flickering in the present invention can also refer to Figure 5 , Figure 3 The approximate implementation logic of flicker removal is given. Figure 5 The detailed process of deflickering is given.

[0182] In this embodiment, in view of the flicker problem occurring during the local enhancement process, an inter-frame flicker removal algorithm based on contour matching is proposed, which effectively solves the flicker problem caused by the enhancement.

[0183] In another implementation of the present invention, after obtaining the de-flickering local highlight enhancement image, an enhancement operation may be performed on the de-flickering local highlight enhancement image to obtain a local enhanced image. It can be further enhanced to obtain a local highlight enhancement image that better meets actual needs. Perform scene analysis to obtain its amplitude histogram distribution, use the amplitude histogram to index the local highlight enhancement table, and obtain the local highlight enhancement parameters, where the implementation principle and significance of the local highlight enhancement parameters are the same as the above-mentioned HDR global enhancement parameters. The HDR global enhancement table is obtained by modeling different types of HDR videos, and similarly, the local highlight enhancement table is also obtained by modeling different types of HDR videos.

[0184] The local highlight enhancement curve is obtained by using the local highlight enhancement parameters to achieve the local highlight enhancement image after flicker removal. Optimize to get the final local highlight enhancement map , which is the above-mentioned local enhanced image. The amplitude is normalized to range and compare it with the global enhanced Y channel image , that is, the above-mentioned global enhanced image is dot-producted pixel by pixel to obtain the local enhanced Y channel image , which is the enhanced image mentioned above.

[0185] In this embodiment, the local highlight details can be adaptively enhanced through the HDR local highlight enhancement algorithm.

[0186] In another implementation of the present invention, after local highlight enhancement, the brightness of video subtitles, station logos and other areas may be too high to cause glare. Considering that subtitles and station logos are generally in fixed areas of the video. Therefore, a highlight suppression operation is performed on the target fixed area in the enhanced image to obtain a video frame of the second type of video. The specific steps are as follows:

[0187] 1) Generate a highlight suppression map corresponding to the target fixed area in the enhanced image.

[0188] Specifically, the target fixed area is an area where designated data exists. The designated data in this embodiment is data such as subtitles and station logos that are fixedly displayed in the frame image.

[0189] Generate highlight suppression maps for these target fixed areas In practical applications, the highlight suppression map In the data, the pixels in the specified data area are 1, and the pixels in other areas are 0.

[0190] 2) Performing a linear low-pass filtering operation on the highlight suppression map to obtain a smoothed highlight suppression map.

[0191] Specifically, a linear low-pass filter is used to Processed to smooth its edges.

[0192] 3) Mapping the smoothed highlight suppression map into a preset range to obtain a mapped highlight suppression map.

[0193] Specifically, the smoothed Map to In the range, , Size according to the above Adaptive adjustment, The bigger The smaller it is, the bigger it is.

[0194] 4) Performing a pixel-by-pixel dot product operation on the mapped highlight suppression map and the enhanced image to obtain a target video frame.

[0195] Specifically, the mapped highlight suppression map, i.e., the mapped With the local enhanced Y channel image , that is, the above-mentioned enhanced image is subjected to pixel-by-pixel dot product operation to obtain the final local enhanced Y channel image , that is, the target video frame. By performing pixel-by-pixel dot product operations, the highlights of the subtitle and logo areas in the enhanced image can be suppressed, thereby avoiding the glare problem caused by excessive brightness.

[0196] 5) Converting the target video frame into a video frame of a second type of video.

[0197] Specifically, according to , get the RGB image in linear space Augmented image on 3 channels:

[0198]

[0199] in, Represents the enhanced image of the red channel in linear space, Represents the Y channel image in linear space, , represents the component of the red channel, Represents the saturation coefficient, whose value is determined by the above rate of change Adaptive adjustment, when The smaller, The bigger it is, the smaller it is. , Adopt and In the same way,

[0200] Finally, , Mapped to the BT.2020 color space and encoded using the PQ OETF transmission curve specified in the HDR10 standard to obtain the final video frame in the HDR space.

[0201] In one embodiment of the present invention, the above-mentioned local highlight enhancement image after flicker removal The enhancement and highlight suppression steps can be combined into one step. In addition, the encoding process of the PQ OETF transmission curve can be optimized by table lookup instead of pixel-by-pixel calculation, further improving the processing speed of the algorithm.

[0202] In this embodiment, in order to solve the problem that the video subtitles, station logos and other areas are too bright and dazzling due to local enhancement, a corresponding highlight suppression strategy is used to suppress the highlight, which effectively improves the viewing experience of the second type of video after conversion.

[0203] As can be seen from the above, through the video conversion method of the present invention, SDR video can be efficiently converted to HDR video on the CPU, and the visual effect of the converted HDR video is significantly improved. The overall implementation process can be referred to Figure 6 .

[0204] First, FFmpeg is used to perform color gamut space conversion and mapping on the input SDR video, wherein the color gamut space conversion and mapping refers to the above-mentioned conversion into an RGB image in a linear space and a Y channel image in a linear space.

[0205] Then, the pre-modeled HDR global enhancement table is indexed according to the scene information to obtain the HDR global enhancement parameters, and the HDR global enhancement curve is obtained using the HDR global enhancement parameters. The HDR global enhancement curve is used to optimize and adjust the global light and shadow to prevent extreme effects such as being too dark or dazzling in different scenes.

[0206] Subsequently, the proposed de-flickering algorithm based on contour matching is used to de-flicker the local highlight enhancement image obtained by guided filtering. Specifically, referring to the de-flickering process described above, the flickering problem caused by enhancement is effectively solved.

[0207] Then, the pre-modeled local highlight enhancement table is indexed according to the amplitude distribution of the local highlight enhancement map after flicker removal, and the local highlight enhancement map is further optimized to adaptively enhance the local highlight details.

[0208] Subsequently, the globally enhanced Y channel image is locally enhanced and the subtitle and station logo area is protected. After obtaining the corresponding RGB enhanced image, the saturation is adaptively adjusted to prevent oversaturation or undersaturation.

[0209] Finally, the RGB enhanced image is mapped to the final HDR format.

[0210] It can be seen from the above that the embodiment of the present invention effectively improves the viewing experience of the converted HDR video through the combination of the above steps.

[0211] Based on the above-mentioned embodiment of the video conversion method, another embodiment of the present invention provides a video conversion device, referring to Figure 7 ,include:

[0212] An image determination module 11 is used to determine a brightness amplitude channel image corresponding to a current frame image in a first type of video;

[0213] A global enhancement module 12 is used to perform a global enhancement operation on the brightness amplitude channel image to obtain a global enhanced image;

[0214] The image processing module 13 is used to perform edge-preserving filtering on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement image, and perform de-flickering processing on the local highlight enhancement image to obtain a de-flickering local highlight enhancement image;

[0215] A local enhancement module 14, configured to enhance the local highlight enhancement image after flicker removal to obtain a local enhanced image, and to obtain an enhanced image based on the global enhanced image and the local enhanced image;

[0216] The highlight suppression module 15 is used to perform a highlight suppression operation on the enhanced image to obtain a video frame of a second type of video.

[0217] In an optional implementation, the global enhancement module 12 includes:

[0218] A scene analysis submodule, used for performing scene analysis on the brightness amplitude channel image to obtain an amplitude intensity histogram;

[0219] A parameter determination submodule, used to determine the HDR global enhancement parameter corresponding to the amplitude intensity histogram;

[0220] A curve construction submodule, used to construct a global enhancement curve corresponding to the HDR global enhancement parameter;

[0221] The global enhancement submodule is used to perform global enhancement processing on the brightness amplitude channel image using the global enhancement curve to obtain a globally enhanced image.

[0222] In an optional implementation, the image processing module 13 includes:

[0223] A region determination submodule, used for determining a partial region in the brightness amplitude channel image whose brightness component is greater than a preset threshold as a local highlight region;

[0224] The filtering processing submodule is used to perform edge-preserving filtering on the local highlight area to obtain a local highlight enhancement image.

[0225] In an optional implementation, the image processing module 13 includes:

[0226] The enhancement image acquisition submodule is used to acquire the local highlight enhancement image corresponding to the previous frame image and the local highlight enhancement image after flickering removal;

[0227] A preprocessing submodule, for downsampling and binarizing the local highlight enhancement image corresponding to the current frame image and the local highlight enhancement image corresponding to the previous frame image, respectively, to obtain a first enhancement image and a second enhancement image, and for downsampling the de-flickering local highlight enhancement image corresponding to the previous frame image, to obtain a historical local highlight enhancement image corresponding to the previous frame image;

[0228] a variance determination submodule, configured to determine an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image based on the first enhancement image, the second enhancement image, and the historical local highlight enhancement image corresponding to the previous frame image;

[0229] The de-flickering submodule is used to normalize the local highlight enhancement map corresponding to the current frame image to the target mean variance using the actual mean variance as distribution to obtain the local highlight enhancement map after de-flickering.

[0230] In an optional implementation, the variance determination submodule includes:

[0231] An intersection calculation unit, used to calculate an intersection area of ​​the first enhancement map and the second enhancement map;

[0232] a data determination unit, configured to determine basic contour units corresponding to the first enhancement map and the second enhancement map respectively;

[0233] a matching unit, configured to match the basic contour units corresponding to the first enhancement image and the basic contour units corresponding to the second enhancement image with the basic contour units in the intersection area respectively, to obtain the common contour units of the first enhancement image and the second enhancement image;

[0234] a comparison unit, configured to determine matching contour units of the first enhancement image and the second enhancement image based on a comparison result of coordinates in the basic contour units in the intersection area and coordinates in the common contour units;

[0235] A pixel filling unit, used to perform pixel filling operations on matching contour units in the first enhancement image and the second enhancement image, respectively, to obtain a matching contour binary image corresponding to the first enhancement image and a matching contour binary image corresponding to the second enhancement image;

[0236] A first calculation unit is used to calculate a first mean variance of the amplitude of an area corresponding to the matching contour binary image corresponding to the first enhancement image in the downsampled local highlight enhancement image corresponding to the current frame image;

[0237] A second calculation unit is used to calculate a second mean variance of the amplitude of an area corresponding to the matching contour binary image corresponding to the second enhancement image in the historical local highlight enhancement image corresponding to the previous frame image;

[0238] A variance determination unit is used to determine an actual mean variance and a target mean variance of the local highlight enhancement map corresponding to the current frame image based on the first mean variance and the second mean variance.

[0239] In an optional implementation, the variance determination submodule further includes:

[0240] a correction unit, configured to, if the first mean variance satisfies a correction rule, perform a correction operation on the first mean variance to obtain a corrected first mean variance, and, if the second mean variance satisfies the correction rule, perform a correction operation on the second mean variance to obtain a corrected second mean variance;

[0241] The variance determination unit is further used to determine the actual mean variance and the target mean variance of the local highlight enhancement image corresponding to the current frame image based on the corrected first mean variance and the corrected second mean variance.

[0242] In an optional implementation, the local enhancement module 14 includes:

[0243] The dot product submodule is used to perform a pixel-by-pixel dot product operation on the global enhanced image and the local enhanced image to obtain an enhanced image.

[0244] In an optional implementation, the highlight suppression module 15 includes:

[0245] A map generation submodule is used to generate a highlight suppression map corresponding to a target fixed area in the enhanced image; the target fixed area is an area where specified data exists;

[0246] A filtering submodule, used for performing a linear low-pass filtering operation on the highlight suppression map to obtain a smoothed highlight suppression map;

[0247] A mapping submodule, used for mapping the smoothed highlight suppression map into a preset range to obtain a mapped highlight suppression map;

[0248] An image processing submodule, used for performing a pixel-by-pixel dot product operation on the mapped highlight suppression map and the enhanced image to obtain a target video frame;

[0249] The conversion submodule is used to convert the target video frame into a video frame of a second type of video.

[0250] In this embodiment, the brightness amplitude channel image corresponding to the current frame image in the first type of video is determined, a global enhancement operation is performed on the brightness amplitude channel image to obtain a global enhanced image, an edge-preserving filtering process is performed on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement map, a de-flickering process is performed on the local highlight enhancement map to obtain a de-flickering local highlight enhancement map, the de-flickering local highlight enhancement map is enhanced to obtain a local enhanced image, an enhanced image is obtained based on the global enhanced image and the local enhanced image, a highlight suppression operation is performed on the enhanced image to obtain a video frame of the second type of video, thereby achieving the goal of converting the first type of video to the second type of video. In addition, in the present invention, the flickering problem caused by enhancement can be effectively solved by de-flickering. In addition, by performing a highlight suppression operation on the enhanced image, problems such as excessive brightness and glare caused by local enhancement can be avoided, and the viewing experience of the converted second type of video can be effectively improved.

[0251] It should be noted that, for the working process of each module, sub-module and unit in this embodiment, please refer to the corresponding description in the above embodiment, which will not be repeated here.

[0252] The present application also provides an electronic device in an embodiment. Figure 8 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 8 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0253] like Figure 8As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603 to implement the above-mentioned video conversion method. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0254] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0255] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any video conversion method provided in the embodiment of the present application.

[0256] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any video conversion method provided in the embodiment of the present application.

[0257] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A video conversion method, characterized in that: include: Determine a brightness amplitude channel image corresponding to a current frame image in a first type of video; Performing a global enhancement operation on the brightness amplitude channel image to obtain a global enhanced image; Performing edge-preserving filtering processing on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement image, and performing de-flickering processing on the local highlight enhancement image to obtain a de-flickering local highlight enhancement image; enhancing the de-flickering local highlight enhancement image to obtain a local enhanced image, and obtaining an enhanced image based on the global enhanced image and the local enhanced image; Performing a highlight suppression operation on the enhanced image to obtain a video frame of a second type of video; The de-flickering process is performed on the local highlight enhancement image to obtain the de-flickering local highlight enhancement image, including: Obtain the local highlight enhancement image corresponding to the previous frame image and the local highlight enhancement image after flicker removal; Downsampling and binarizing the local highlight enhancement image corresponding to the current frame image and the local highlight enhancement image corresponding to the previous frame image respectively to obtain a first enhancement image and a second enhancement image, and downsampling the de-flickering local highlight enhancement image corresponding to the previous frame image to obtain a historical local highlight enhancement image corresponding to the previous frame image; Determine an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image based on the first enhancement image, the second enhancement image, and the historical local highlight enhancement image corresponding to the previous frame image; For the local highlight enhancement map corresponding to the current frame image, the actual mean variance is used as distribution, normalized to the target mean variance, to obtain the local highlight enhancement map after flicker removal; Wherein, based on the first enhancement map, the second enhancement map and the historical local highlight enhancement map corresponding to the previous frame image, determining the actual mean variance and the target mean variance of the local highlight enhancement map corresponding to the current frame image includes: Calculating an intersection area of ​​the first enhancement map and the second enhancement map; Determine basic contour units corresponding to the first enhancement map and the second enhancement map respectively; Matching the basic contour units corresponding to the first enhancement graph and the basic contour units corresponding to the second enhancement graph with the basic contour units in the intersection area respectively, to obtain the common contour units of the first enhancement graph and the second enhancement graph; Determine matching contour units of the first enhancement image and the second enhancement image based on a comparison result of coordinates in the basic contour units in the intersection area and coordinates in the common contour units; Performing pixel filling operations on matching contour units in the first enhanced image and the second enhanced image respectively to obtain a matching contour binary image corresponding to the first enhanced image and a matching contour binary image corresponding to the second enhanced image; Calculate a first mean variance of the amplitude of a region corresponding to the matching contour binary image corresponding to the first enhanced image in the downsampled local highlight enhanced image corresponding to the current frame image; Calculate a second mean variance of the amplitude of the area corresponding to the matching contour binary image corresponding to the second enhanced image in the historical local highlight enhanced image corresponding to the previous frame image; Based on the first mean variance and the second mean variance, an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image are determined.

2. The video conversion method according to claim 1, characterized in that: Performing a global enhancement operation on the brightness amplitude channel image to obtain a global enhanced image includes: Performing scene analysis on the brightness amplitude channel image to obtain an amplitude intensity histogram; Determining a global enhancement parameter corresponding to the amplitude intensity histogram; Constructing a global enhancement curve corresponding to the global enhancement parameter; The global enhancement curve is used to perform global enhancement processing on the brightness amplitude channel image to obtain a global enhanced image.

3. The video conversion method according to claim 1, characterized in that: Performing edge-preserving filtering on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement map, including: Determine a partial area in the brightness amplitude channel image where the brightness component is greater than a preset threshold as a local highlight area; An edge-preserving filtering process is performed on the local highlight area to obtain a local highlight enhancement map.

4. The video conversion method according to claim 1, characterized in that: After calculating the second mean variance of the amplitude of the area corresponding to the matching contour binary image corresponding to the second enhanced image in the historical local highlight enhanced image corresponding to the previous frame image, the method further includes: If the first mean variance satisfies the correction rule, a correction operation is performed on the first mean variance to obtain a corrected first mean variance, and if the second mean variance satisfies the correction rule, a correction operation is performed on the second mean variance to obtain a corrected second mean variance; Based on the corrected first mean variance and the corrected second mean variance, an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image are determined.

5. The video conversion method according to claim 1, characterized in that: Obtaining an enhanced image based on the global enhanced image and the local enhanced image, comprising: A pixel-by-pixel dot product operation is performed on the global enhanced image and the local enhanced image to obtain an enhanced image.

6. The video conversion method according to claim 1, characterized in that: Performing a highlight suppression operation on the enhanced image to obtain a video frame of a second type of video includes: Generate a highlight suppression map corresponding to a target fixed area in the enhanced image; the target fixed area is an area where specified data exists; Performing a linear low-pass filtering operation on the highlight suppression map to obtain a smoothed highlight suppression map; Mapping the smoothed highlight suppression map into a preset range to obtain a mapped highlight suppression map; Performing a pixel-by-pixel dot product operation on the mapped highlight suppression map and the enhanced image to obtain a target video frame; The target video frame is converted into a video frame of a second type of video.

7. A video conversion device, characterized in that: include: An image determination module, used to determine a brightness amplitude channel image corresponding to a current frame image in a first type of video; A global enhancement module, used for performing a global enhancement operation on the brightness amplitude channel image to obtain a global enhanced image; An image processing module, configured to perform edge-preserving filtering on the local highlight area in the brightness amplitude channel image to obtain a local highlight enhancement image, and perform de-flickering processing on the local highlight enhancement image to obtain a de-flickering local highlight enhancement image; A local enhancement module, used for enhancing the local highlight enhancement image after flicker removal to obtain a local enhanced image, and obtaining an enhanced image based on the global enhanced image and the local enhanced image; A highlight suppression module, configured to perform a highlight suppression operation on the enhanced image to obtain a video frame of a second type of video; The image processing module performs a de-flickering process on the local highlight enhancement image to obtain a de-flickering local highlight enhancement image, including: Obtain the local highlight enhancement image corresponding to the previous frame image and the local highlight enhancement image after flicker removal; Downsampling and binarizing the local highlight enhancement image corresponding to the current frame image and the local highlight enhancement image corresponding to the previous frame image respectively to obtain a first enhancement image and a second enhancement image, and downsampling the de-flickering local highlight enhancement image corresponding to the previous frame image to obtain a historical local highlight enhancement image corresponding to the previous frame image; Determine an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image based on the first enhancement image, the second enhancement image, and the historical local highlight enhancement image corresponding to the previous frame image; For the local highlight enhancement map corresponding to the current frame image, the actual mean variance is used as distribution, normalized to the target mean variance, to obtain the local highlight enhancement map after flicker removal; Wherein, based on the first enhancement map, the second enhancement map and the historical local highlight enhancement map corresponding to the previous frame image, determining the actual mean variance and the target mean variance of the local highlight enhancement map corresponding to the current frame image includes: Calculating an intersection area of ​​the first enhancement map and the second enhancement map; Determine basic contour units corresponding to the first enhancement map and the second enhancement map respectively; Matching the basic contour units corresponding to the first enhancement graph and the basic contour units corresponding to the second enhancement graph with the basic contour units in the intersection area respectively, to obtain the common contour units of the first enhancement graph and the second enhancement graph; Determine matching contour units of the first enhancement image and the second enhancement image based on a comparison result of coordinates in the basic contour units in the intersection area and coordinates in the common contour units; Performing pixel filling operations on matching contour units in the first enhanced image and the second enhanced image respectively to obtain a matching contour binary image corresponding to the first enhanced image and a matching contour binary image corresponding to the second enhanced image; Calculate a first mean variance of the amplitude of a region corresponding to the matching contour binary image corresponding to the first enhanced image in the downsampled local highlight enhanced image corresponding to the current frame image; Calculate a second mean variance of the amplitude of the area corresponding to the matching contour binary image corresponding to the second enhanced image in the historical local highlight enhanced image corresponding to the previous frame image; Based on the first mean variance and the second mean variance, an actual mean variance and a target mean variance of the local highlight enhancement image corresponding to the current frame image are determined.

8. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the video conversion method as described in any one of claims 1 to 6.

Images