High-speed enhancement method for super-large-resolution image

Abstract

The invention relates to the technical field of image correction, and particularly discloses a high-speed enhancement method for an ultra-large-resolution image, and the method comprises the steps: obtaining an ultra-large-resolution image in a YUV format, and carrying out the extraction and storage of a Y component in the ultra-large-resolution image; a content self-adaptive image segmentation method is executed, through the steps of brightness-based region contrast parallel enhancement, Y component image overall equalization, UV component image self-adaptive compensation and the like, the contrast of a sub-image can be subjected to self-adaptive enhancement firstly, so that image details are improved, and then the image details are improved through Y component image overall equalization. According to the method and the device, seams among the sub-block images after contrast enhancement can be eliminated, the quality of the adjusted image is further improved, and finally, details such as edges and textures in the image can be clearer through UV component processing, so that the quality enhancement of the super-large-resolution image is realized.

Description

Technical Field

[0001] The invention relates to the technical field of image correction, and in particular to a method for high-speed enhancement of an ultra-large resolution image. Background Art

[0002] Ultra-high resolution images are widely used in security monitoring, virtual reality, medicine, remote sensing technology and many other fields. However, in the process of obtaining ultra-high resolution images, there are problems such as uneven image brightness and color, low contrast, unclear detail texture, high and complex noise distribution, and color deviation. Therefore, ultra-high resolution images need to be corrected.

[0003] Traditional correction methods are usually global or local image enhancement methods based on traditional underlying image processing, such as global or local histogram equalization, converting color space and separately processing image information such as chromaticity and brightness, or image enhancement methods based on physical models. Image enhancement methods based on physical models can achieve a balance in multiple tasks while taking into account dynamic range compression, edge enhancement, brightness and contrast enhancement to achieve a balance, thereby achieving the effect of improving image quality.

[0004] However, traditional image correction methods are generally global or local image enhancement methods based on traditional underlying image processing or image enhancement methods based on physical models. Although both methods can improve image quality, their processing methods are simple and can only enhance specific features of the image, which can easily cause color distortion or over-correction, reduce image perception, and thus affect image quality. Summary of the invention

[0005] The purpose of the present invention is to provide a high-speed enhancement method for ultra-high resolution images to solve the following technical problems:

[0006] How to improve the quality of super-resolution images after rectification.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A high-speed enhancement method for ultra-high resolution images, the method comprising:

[0009] S1: Obtain an ultra-high resolution image in YUV format, and extract and store the Y component in the ultra-high resolution image;

[0010] S2: Perform content-adaptive image segmentation method;

[0011] S3: Get the Y component block result;

[0012] S4: Brightness-based regional contrast parallel enhancement;

[0013] S5: Overall equalization of Y component image;

[0014] S6: Adaptive compensation of UV component image;

[0015] S7: Output the enhanced super-resolution image.

[0016] Furthermore, the process of extracting and storing the Y component in the ultra-high resolution image in S1 includes:

[0017] S11: firstly, a frame of ultra-high resolution image at the current moment is acquired through an image acquisition device;

[0018] S12: converting the obtained ultra-high resolution image into a YUV format image, particularly using an NV12 image format in a YUV420 format;

[0019] S13: Acquire the Y component in the YUV format image, and extract and store the Y component in the ultra-high resolution image in the YUV image format acquired at the current moment.

[0020] Furthermore, the content-adaptive image segmentation method in S2 includes:

[0021] The target position is extracted according to the image brightness information by executing the content-adaptive image segmentation method. Since the size of each pixel on the Y component image represents the brightness of the actual target, the target position is extracted according to the image brightness information and recorded into a brightness information table. The Y component is then segmented into three categories: over-bright targets, normal brightness targets, and darker targets according to the size of the Y component pixel values.

[0022] Furthermore, the process of obtaining the Y component segmentation result in S3 includes:

[0023] After locating and segmenting the target position according to brightness on the Y component image, the segmentation circumscribed rectangle is calculated, and the Y component image is divided into blocks according to the minimum rectangle. Now the Y component image is composed of some rectangular sub-images containing over-bright targets, normal brightness targets and darker targets.

[0024] Furthermore, the process of enhancing the regional contrast in parallel based on brightness in S4 includes:

[0025] For the Y component image after segmentation, the thread library is used to process the sub-images in parallel through multiple threads. The brightness-based regional contrast parallel enhancement method simultaneously enhances the over-bright target, normal brightness target and dark target sub-images, and the processing of the segmented sub-images does not affect each other.

[0026] Furthermore, the process of the brightness-based regional contrast parallel enhancement in S4 further includes:

[0027] First, each sub-image of the Y component image after segmentation is smoothed to obtain the low-frequency part, and then the image obtained by the previous operation is subtracted from the original image to obtain the high-frequency part. Finally, the obtained image is enhanced, and the low-frequency part and the high-frequency part are recombined to obtain the output result image of the algorithm, thereby achieving the purpose of adaptive contrast enhancement of each sub-image.

[0028] Furthermore, the overall equalization process of the Y component image in S5 includes:

[0029] The Y component image is operated using an equalization method according to the grayscale image to equalize the Y component image as a whole and eliminate or weaken the influence of obvious gaps between each image block.

[0030] Furthermore, the UV component image adaptive compensation process in S6 is:

[0031] After operating the Y component image, the Y component image of the YUV image brightness component is processed to maintain the naturalness of the overall image color, adaptively compensate the UV component, and improve the overall color.

[0032] Beneficial effects of the present invention:

[0033] (1) The present invention can firstly adaptively enhance the contrast of the sub-images through the steps of brightness-based regional contrast parallel enhancement, Y component image overall equalization, and UV component image adaptive compensation, thereby improving the image details. Then, by overall equalization of the Y component image, the seams between the sub-block images after contrast enhancement can be eliminated, further improving the image quality after adjustment. Finally, by processing the UV component, the details such as the edges and textures in the image can be made clearer, thereby achieving quality enhancement of the ultra-high resolution image.

[0034] (2) The present invention suppresses the strong light of the over-bright target sub-image, reduces the target pixel value, enhances the contrast, adaptively enhances the contrast of the target sub-image with normal brightness, enhances the low illumination of the darker target sub-image, increases the target pixel value, enhances the contrast, and then adaptively offsets and compensates according to the specific brightness of the actual ambient light, readjusts the pixel value of the image, and improves the target details. Thereafter, the output result diagram of the algorithm is obtained by recombining the low-frequency and high-frequency images, and the contrast of the processed sub-image is adaptively enhanced and the averaging operation is performed, so that the image details can be enhanced.

[0035] (3) The present invention operates the Y component image by using an equalization method in combination with a grayscale image. At this time, the Y component image is overall equalized, which can eliminate or reduce the influence of the seams, thereby improving the image quality after the adjustment, and further improving the clarity and quality of the large-resolution image.

[0036] (4) By processing the UV component, the present invention can accurately adjust the color saturation, hue and other attributes of the image, making the image color more realistic and natural, significantly improving the color performance of the image, improving the visual quality of the image, and making the details such as the edges and textures in the image clearer, thereby improving the recognition and readability of the image, and further improving the quality of the adjusted large-resolution image. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present invention will be further described below in conjunction with the accompanying drawings.

[0038] Figure 1 It is a flow chart of a high-speed enhancement method for ultra-high resolution images in the present invention;

[0039] Figure 2 It is a flow chart of the process of extracting and storing the Y component in the ultra-high resolution image in the present invention;

[0040] Figure 3 is an example original image of image segmentation in the present invention;

[0041] Figure 4 is a segmentation map after image segmentation in the present invention;

[0042] Figure 5 is the image block diagram in the present invention;

[0043] Figure 6 It is the comparison before and after the image adjustment in the present invention Figure 1 ;

[0044] Figure 7 This is the comparison of the image before and after adjustment in the present invention. Figure 2 . DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] See also Figure 1 As shown, in one embodiment, the present application provides a method for high-speed enhancement of an ultra-large resolution image, the method comprising:

[0047] S1: Obtain an ultra-high resolution image in YUV format, and extract and store the Y component in the ultra-high resolution image;

[0048] S2: Perform content-adaptive image segmentation method;

[0049] S3: Get the Y component block result;

[0050] S4: Brightness-based regional contrast parallel enhancement;

[0051] S5: Overall equalization of Y component image;

[0052] S6: Adaptive compensation of UV component image;

[0053] S7: output enhanced super-resolution image;

[0054] Through the above technical solution, this example provides a high-speed enhancement method for ultra-high resolution images. First, an ultra-high resolution image in YUV format is obtained, and the Y component in the ultra-high resolution image is extracted and stored. Then, a content-adaptive image segmentation method is performed to obtain a Y component segmentation result. Then, the regional contrast based on brightness is enhanced in parallel and the Y component image is overall equalized. Finally, the enhanced ultra-high resolution image can be output through adaptive compensation of the UV component image.

[0055] Through such settings, when the quality of the ultra-high resolution image is enhanced, the contrast of the sub-image can be adaptively enhanced first through the steps of parallel enhancement of regional contrast based on brightness, overall equalization of Y component image and adaptive compensation of UV component image, so as to improve the image details. Then, through the overall equalization of the Y component image, the seams between the sub-block images after the contrast enhancement can be eliminated, and the image quality after the adjustment is further improved. Finally, through the processing of the UV component, the details such as the edge and texture in the image can be made clearer, so as to achieve the quality enhancement of the ultra-high resolution image.

[0056] See also Figure 2 As shown, the process of extracting and storing the Y component in the ultra-high resolution image in S1 includes:

[0057] S11: firstly, a frame of ultra-high resolution image at the current moment is acquired through an image acquisition device;

[0058] S12: converting the obtained ultra-high resolution image into a YUV format image, particularly using an NV12 image format in a YUV420 format;

[0059] S13: Acquire the Y component in the YUV format image, and extract and store the Y component in the ultra-high resolution image in the YUV image format acquired at the current moment;

[0060] Through the above technical solution, this embodiment provides a process for extracting and storing the Y component in the ultra-high resolution image. First, a frame of ultra-high resolution image at the current moment is obtained through an image acquisition device, and the obtained ultra-high resolution image is converted into a YUV format image, especially the NV12 image format in the YUV420 format. Finally, the Y component in the YUV format image is obtained, and the Y component in the ultra-high resolution image in the YUV image format obtained at the current moment is extracted and stored;

[0061] The invention of the YUV format was due to the transition period between color television and black-and-white television. The earliest conception of color television was to use the three primary colors of RGB for simultaneous transmission. This design method was three times the original black-and-white bandwidth and was not a very good design at the time. RGB focuses on the human eye's perception of color, while YUV focuses on the visual sensitivity to brightness. Y represents brightness and UV represents chroma, which are represented by Cr and Cb respectively. Records in the YUV format are usually presented in the format of Y:UV. In this method, the ultra-high-resolution image obtained by the image acquisition device is usually a decoded YUV format image, and the NV12 image format in the YUV420 format is particularly used. In this way, while ensuring the image quality, the memory size used is greatly reduced, which is convenient for transmission and operation. At this time, the Y component, that is, the brightness component, is saved separately, that is, the image containing brightness information, which can also be understood as an ultra-high-resolution grayscale image.

[0062] The content-adaptive image segmentation method in S2 includes:

[0063] The target position is extracted according to the image brightness information by executing the content-adaptive image segmentation method. Since the size of each pixel on the Y component image represents the brightness of the actual target, the target position is extracted according to the image brightness information and recorded into a brightness information table. The Y component is then segmented into three categories: over-bright targets, normal brightness targets, and darker targets according to the Y component pixel value.

[0064] Through the above technical solution, since the Y component itself represents the brightness component, the Y component can be divided into overbright targets, normal brightness targets and darker targets according to the Y component pixel value. For example, taking the pixel value 0-255 as an example, the pixel value 200-255 is divided into overbright targets, the pixel value 100-200 is divided into normal brightness targets, and the pixel value 0-100 is divided into darker targets. Figure 3 and Figure 4 As shown, Figure 4 It's the original picture. Figure 4 is the segmentation map, and the bright target is Figure 4 Medium white area, normal brightness target is Figure 4 Medium gray area, darker targets are Figure 4In the black area, in the subsequent processing, due to the difference in imaging caused by different brightness, different enhancement methods are used for over-bright targets, normal brightness targets and darker targets respectively to prevent over-compensation and distortion caused by excessive processing of different types of targets, and at the same time reduce the amount of processing calculations.

[0065] See also Figure 4 and Figure 5 As shown, the process of obtaining the Y component block result in S3 includes:

[0066] After locating and segmenting the target position according to brightness on the Y component image, the segmentation circumscribed rectangle is calculated, and the Y component image is divided into blocks according to the minimum rectangle. Now the Y component image is composed of some rectangular sub-images containing over-bright targets, normal brightness targets and darker targets;

[0067] Through the above technical scheme, for the position information map after the Y component image is located and segmented according to the brightness information, the segmentation circumscribed rectangle is calculated, and the Y component image is divided into blocks as a whole according to the minimum rectangle. That is, after locating and segmenting the target position on the Y component image according to the brightness, the segmentation circumscribed rectangle is calculated, and the Y component image is divided into blocks as a whole according to the minimum rectangle. Now the Y component image is composed of some rectangular sub-images containing over-bright targets, normal brightness targets and darker targets, which is convenient for subsequent further processing.

[0068] The process of parallel enhancement of regional contrast based on brightness in S4 includes:

[0069] For the Y component image after segmentation, the thread library is used to process the sub-images in parallel through multiple threads. The brightness-based regional contrast parallel enhancement method is used to enhance the over-bright target, normal brightness target and dark target sub-images respectively, and the processing of the segmented sub-images does not affect each other.

[0070] Through the above technical solution, the following operations are first performed on each pixel in the over-bright target and dark target sub-images;

[0071] The pixel value of the processed image is calculated by the formula dst(i, j)=|src(i, j)*α+β|;

[0072] Among them, i and j are any points in the image, src(i, j) is the original image pixel value at position (i, j) in an image, α and β are both constant coefficients, where a constant coefficient means that the coefficient of a variable remains unchanged in a function or equation. Here, each pixel in the current frame of the image is multiplied by a fixed coefficient α, plus a fixed coefficient β, where α represents the scaling factor of the image pixel value, the type is floating point type, the default is 1.0, which means no scaling, 0.25 means the pixel value is reduced by 4 times, and 4.0 means the pixel value is enlarged by 4 times; β represents the offset constant of the image pixel value, the type is floating point type, the default is 0.0, which means no offset, -20.0 means the pixel value is reduced by 20, the image is darker, and 20.0 means the pixel value is increased by 20, the image is brighter; and ensure that the processed values ​​are normalized to 0-255, and then the contrast of the over-bright target, normal brightness target and darker target sub-images is adaptively enhanced at the same time.

[0073] The process of the brightness-based regional contrast parallel enhancement in S4 further includes:

[0074] First, each sub-image of the Y component image after block division is smoothed to obtain the low-frequency part, and then the image obtained by the previous operation is subtracted from the original image to obtain the high-frequency part. Finally, the obtained image is enhanced, and the low-frequency part and the high-frequency part are recombined to obtain the output result image of the algorithm, so as to achieve the purpose of adaptive contrast enhancement of each sub-image.

[0075] Through the above technical solution, the process of parallel enhancement of regional contrast based on brightness includes: using the relevant information of the local histogram to map the data through adaptive contrast enhancement, and using the unsharp masking technology to effectively enhance the gradient of the low-contrast image. First, the image is divided into two parts. The first part is the low-frequency part obtained after smoothing the original image; the second part is the image obtained by subtracting the first part from the original image to obtain the high-frequency part. Then, the image obtained from the second part is enhanced, and the images of the first and second parts are recombined to obtain the output result diagram of the algorithm, which specifically includes the following two steps;

[0076] First, assume that x(i, j) is the pixel value at position (i, j) in the image, the low-frequency part is obtained by finding the local average value, and the template size is (2n+1)×(2n+1);

[0077] By formula Calculate the average value of the pixel values ​​in an area with a window size of (2n+1)×(2n+1);

[0078] Where n is a positive integer, and k∈[in, i+n], and l∈[jn, j+n];

[0079] And through the formula Calculate the variance of the pixel value x(i, j) at position (i, j) in an image;

[0080] Then the high frequency part is enhanced, and the standard deviation is used as the gain value, and the mean value is m x (i, j) is approximately considered as the background part, then x(k, l)-m x (i, j) is the high-frequency detail part, and the gain product is performed on the high frequency;

[0081] By formula Calculate and obtain the pixel value f(i, j) after the high-frequency part of the pixel value x(i, j) at position (i, j) in an image is enhanced;

[0082] Where D is a constant, which can be the global average value of the image, the global mean square error of the image, or a controllable offset can be added as an adjustment;

[0083] Through such settings, these sub-images are processed in multi-threaded parallel by using the thread library, and different enhancement methods are used for over-bright target, normal brightness target and darker target sub-images, namely, the brightness-based regional contrast parallel enhancement method, which suppresses the strong light of the over-bright target sub-image, reduces the target pixel value, and enhances the contrast, adaptively enhances the contrast of the normal brightness target sub-image, enhances the low illumination of the darker target sub-image, increases the target pixel value, and enhances the contrast, and then adaptively offsets and compensates according to the specific brightness of the actual ambient light, readjusts the pixel value of the image, and enhances the target details. After that, the output result diagram of the algorithm is obtained by recombining the low-frequency part and the high-frequency part of the image, and the processed sub-image can be adaptively contrast enhanced and averaged, so that the image details can be enhanced.

[0084] The overall equalization process of the Y component image in S5 includes:

[0085] According to the grayscale image, the Y component image is operated by using an equalization method to make the Y component image overall equalized, eliminating or weakening the influence of obvious gaps between each image block;

[0086] Through the above technical solution, this example provides an overall equalization process for the Y component image. After the Y component image is divided into blocks and the contrast of the over-bright target, normal brightness target and dark target sub-images is adaptively enhanced, each target sub-image is enhanced. However, there will be obvious gaps at the seams of the sub-images. This part is calculated by averaging. Take the grayscale image as an example:

[0087] First, count the number of pixels N(i) at each gray level of the original image. The value range of i is [0, max-1], and max is the total number of gray levels in the image. The number of gray levels is related to the image bit depth. If the image is 8-bit deep, there are 2^8=256 gray levels in total.

[0088] Then the probability of occurrence of a pixel with gray level i is calculated by the formula P(i)=N(i) / M;

[0089] Where M is the number of pixels in the image;

[0090] Then through the formula Calculate the cumulative distribution function of P(i);

[0091] And finally calculate the average value of the mixed part:

[0092]

[0093] Among them, f(min) is the minimum value of the cumulative distribution function, M and N represent the number of pixels of the length and width of the image respectively, round is the rounding function, and f(max) is the maximum value of the cumulative distribution function;

[0094] Through the above technical solution, the Y component image is operated by using an equalization method in combination with the grayscale image. At this time, the Y component image is overall equalized, which can eliminate or weaken the influence of the seams, thereby improving the image quality after the adjustment, and further improving the clarity and quality of the large-resolution image.

[0095] The UV component image adaptive compensation process in S6 is:

[0096] After operating the Y component image, the Y component image of the YUV image brightness component is processed to maintain the naturalness of the overall image color, and the UV component is adaptively compensated to improve the overall color;

[0097] Through the above technical solution, this example provides an adaptive compensation process for UV component images. After operating the Y component image, the overall details of the image are enhanced, but the image color will become lighter as a whole, and the color is not bright. At this time, it is necessary to adaptively compensate the UV component to improve the overall color. The processing of the UV component can be adaptively corrected according to the following formula;

[0098] By formula and Calculate and obtain the enhanced U component pixel value at position (i, j) and the enhanced V component pixel value at position (i, j) respectively;

[0099] Among them, Y dst(k, l) is the enhanced Y component image of position (k, l) within the n×n domain of position (i, j), Y src (k, l) is the Y component image of the position (k, l) before enhancement within the n×n domain of the position (i, j);

[0100] Through such settings, through the processing of UV components, the color saturation, hue and other attributes of the image can be accurately adjusted to make the image color more realistic and natural, significantly improve the color performance of the image, improve the visual quality of the image, and make the details such as edges and textures in the image clearer, thereby improving the recognition and readability of the image, and then improving the quality of the adjusted large-resolution image.

[0101] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A high-speed enhancement method for ultra-high resolution images, characterized in that: The method comprises: S1: Obtain an ultra-high resolution image in YUV format, and extract and store the Y component in the ultra-high resolution image; S2: Perform content-adaptive image segmentation method; S3: Get the Y component block result; S4: Brightness-based regional contrast parallel enhancement; S5: Overall equalization of Y component image; S6: Adaptive compensation of UV component image; S7: Output the enhanced super-resolution image.

2. The method for high-speed enhancement of ultra-high resolution images according to claim 1, characterized in that: The process of extracting and storing the Y component in the ultra-high resolution image in S1 includes: S11: firstly, a frame of ultra-high resolution image at the current moment is acquired through an image acquisition device; S12: converting the obtained ultra-high resolution image into a YUV format image, particularly using an NV12 image format in a YUV420 format; S13: Acquire the Y component in the YUV format image, and extract and store the Y component in the ultra-high resolution image in the YUV image format acquired at the current moment.

3. The method for high-speed enhancement of ultra-high resolution images according to claim 1, characterized in that: The content-adaptive image segmentation method in S2 includes: The target position is extracted according to the image brightness information by executing the content-adaptive image segmentation method. Since the size of each pixel on the Y component image represents the brightness of the actual target, the target position is extracted according to the image brightness information and recorded into a brightness information table. The Y component is then segmented into three categories: over-bright targets, normal brightness targets, and darker targets according to the size of the Y component pixel values.

4. The method for high-speed enhancement of ultra-high resolution images according to claim 1, characterized in that: The process of obtaining the Y component block result in S3 includes: After locating and segmenting the target position according to brightness on the Y component image, the segmentation circumscribed rectangle is calculated, and the Y component image is divided into blocks according to the minimum rectangle. Now the Y component image is composed of some rectangular sub-images containing over-bright targets, normal brightness targets and darker targets.

5. The method for high-speed enhancement of ultra-high resolution images according to claim 1, characterized in that: The process of parallel enhancement of regional contrast based on brightness in S4 includes: For the Y component image after segmentation, the thread library is used to process the sub-images in parallel through multiple threads. The brightness-based regional contrast parallel enhancement method simultaneously enhances the over-bright target, normal brightness target and dark target sub-images, and the processing of the segmented sub-images does not affect each other.

6. The method for high-speed enhancement of ultra-high resolution images according to claim 1, characterized in that: The process of the brightness-based regional contrast parallel enhancement in S4 further includes: First, each sub-image of the Y component image after segmentation is smoothed to obtain the low-frequency part, and then the image obtained by the previous operation is subtracted from the original image to obtain the high-frequency part. Finally, the obtained image is enhanced, and the low-frequency part and the high-frequency part are recombined to obtain the output result image of the algorithm, thereby achieving the purpose of adaptive contrast enhancement of each sub-image.

7. The method for high-speed enhancement of ultra-high resolution images according to claim 1, characterized in that: The overall equalization process of the Y component image in S5 includes: The Y component image is operated using an equalization method according to the grayscale image to equalize the Y component image as a whole and eliminate or weaken the influence of obvious gaps between each image block.

8. The method for high-speed enhancement of ultra-high resolution images according to claim 1, characterized in that: The UV component image adaptive compensation process in S6 is: After operating the Y component image, the Y component image of the YUV image brightness component is processed to maintain the naturalness of the overall image color, adaptively compensate the UV component, and improve the overall color.

Images