Image signal processing method, system and pan / tilt device

Abstract

The present invention discloses an image signal processing method, system and pan / tilt device, and relates to the technical field of image processing. A first image is obtained by performing white balance processing on an image to be processed; a second image is obtained by performing brightness enhancement on the first image; a third image and a fourth image are obtained respectively by performing gamma correction and sharpening processing on the second image; and the third image and the fourth image are fused to obtain a final high-quality image. The color deviation is reduced by white balance processing; the contrast of the image is improved by brightness enhancement; the phenomenon of underexposure or overexposure can be repaired by gamma correction; and the detail level and clarity of the image can be enhanced by sharpening processing. By fusing the images after gamma correction and sharpening processing, the negative effects caused by separate processing can be avoided, and the brightness and details can be balanced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image signal processing method, system and pan / tilt device. Background Art

[0002] Poor image quality at night is a common and complex problem in surveillance systems, mainly due to many factors. At night, due to the lack of natural light, various artificial light sources become the main source of image brightness. These artificial light sources include street lighting, car headlights, billboards, store windows, etc. These light sources are unevenly distributed in space, and the intensity varies greatly, resulting in highly uneven background light in night images. Some areas may be too bright, while other places are too dark. This uneven distribution of light and large color temperature differences cause color cast and distortion in the overall perception of the image. In addition, the quality of surveillance images is low due to the foggy environment at night, affecting the clarity and usability of the image.

[0003] Patent CN118608441A discloses a night-light image enhancement method and device based on color shift correction, which performs image enhancement through a deep learning model. However, this method requires a large amount of computing resources for training and reasoning, and this method mainly focuses on compensation of brightness and color, and does not effectively process other aspects of the image (such as texture, details or noise). In nighttime monitoring, noise and lack of details are common problems, and simply correcting color shift may not solve all problems in the image, especially the processing of noise control and detail enhancement. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of low nighttime image quality mentioned in the above background technology, and to propose an image signal processing method, system and pan / tilt device.

[0005] According to a first aspect of the present invention, there is provided an image signal processing method, the method comprising:

[0006] Acquire an image to be processed; the image to be processed is an RGB image;

[0007] Performing white balance processing on the image to be processed to obtain a first image;

[0008] Performing brightness enhancement on the first image to obtain a second image;

[0009] performing gamma correction and sharpening processing on the second image to obtain a third image and a fourth image respectively;

[0010] The third image and the fourth image are fused to obtain a final high-quality image.

[0011] Optionally, performing white balance processing on the image to be processed to obtain the first image includes:

[0012] Dividing the image to be processed into a plurality of sub-blocks;

[0013] Convert each sub-block into a grayscale image and perform normalization within the block to obtain a weight coefficient matrix for each sub-block;

[0014] According to the weight coefficient matrix of each sub-block, the weighted mean and standard deviation of each RGB channel of each sub-block are calculated;

[0015] According to the weighted mean and standard deviation of multiple sub-blocks, the weighted intensity of the RGB channels of the image to be processed is calculated:

[0016] ;

[0017] in, , is the channel identifier; represents the weighted strength; k and r are sub-block identifiers; p is the number of sub-blocks; is the weighted mean; s is the standard deviation;

[0018] According to the weighted intensity of each channel, the color correction coefficient is obtained:

[0019] ;

[0020] According to the color correction coefficient, the RGB channel values ​​of the image to be processed are adjusted to obtain a first image.

[0021] Optionally, the performing brightness enhancement on the first image to obtain the second image includes:

[0022] Convert the first image to HSV space to obtain the original hue component, saturation component and lightness component;

[0023] Perform histogram equalization on the original brightness component to obtain an enhanced brightness component;

[0024] The enhanced lightness component is combined with the original hue component and saturation component to obtain a new HSV image, and the new HSV image is converted back to an RGB image to obtain a second image.

[0025] Optionally, performing gamma correction and sharpening processing on the second image to obtain a third image and a fourth image respectively includes:

[0026] performing adaptive gamma correction on the second image to obtain a third image;

[0027] According to a preset Gaussian filter, blurring the second image to obtain a blurred image;

[0028] Calculating the difference between the second image and the blurred image, and performing standardization processing to obtain a detail image;

[0029] The detail image is added back to the second image to obtain a fourth image.

[0030] Optionally, fusing the third image and the fourth image to obtain a final high-quality image includes:

[0031] Performing Laplace weight calculation on a target image to obtain a first weight map corresponding to the target image; the target image is any one of the third image and the fourth image;

[0032] Calculate the saliency weight of the target image to obtain its corresponding second weight map;

[0033] According to the first weight map and the second weight map corresponding to the third image and the fourth image, the corresponding fusion weight map is obtained:

[0034] ;

[0035] Wherein, K and i are image identifiers, when equal to 1, it corresponds to the third image; when equal to 2, it corresponds to the fourth image; is the first weight graph; is the second weight graph; is the normalized fusion weight map; T is the temporary weight map generated by the calculation process;

[0036] According to the fusion weight map, the third image and the fourth image are weighted fused to obtain a final high-quality image.

[0037] Optionally, performing Laplace weight calculation on the target image to obtain a first weight map corresponding thereto includes:

[0038] Convert the target image to a grayscale image;

[0039] Performing a convolution operation on the grayscale image according to a preset Laplacian operator to obtain a Laplacian image;

[0040] An absolute value of the Laplacian image is taken to obtain a first weight map.

[0041] Optionally, the performing saliency weight calculation on the target image to obtain the corresponding second weight map includes:

[0042] Convert the target image to the LAB color space to obtain the fifth image, and calculate the mean of each channel thereof;

[0043] According to the difference between each pixel in the fifth image and the mean value of the whole image, the second weight map is calculated:

[0044] ;

[0045] in, is a location identifier; is the second weight graph; are the three channels of the fifth image; Represents the channel mean.

[0046] Optionally, performing weighted fusion on the third image and the fourth image according to the fusion weight map to obtain a final high-quality image includes:

[0047] According to the fusion weight map, the third image and the fourth image are subjected to multi-scale weighted fusion using the Gaussian pyramid and the Laplacian pyramid to obtain a final high-quality image.

[0048] A second aspect of the present invention provides an image signal processing system, the system comprising:

[0049] A data acquisition module, used to acquire an image to be processed; the image to be processed is an RGB image;

[0050] A color correction module, used for performing white balance processing on the image to be processed to obtain a first image;

[0051] A brightness enhancement module, used to enhance the brightness of the first image to obtain a second image;

[0052] The detail enhancement module is used to perform gamma correction and sharpening processing on the second image to obtain a third image and a fourth image respectively; and fuse the third image with the fourth image to obtain a final target image.

[0053] According to a third aspect of the present invention, a pan / tilt device is provided, on which a computer program is embedded. When the program is executed, any image signal processing method provided in the embodiments of the present invention can be implemented.

[0054] Beneficial effects of the present invention:

[0055] The present invention proposes an image signal processing method, which includes: acquiring an image to be processed; the image to be processed is an RGB image; performing white balance processing on the image to be processed to obtain a first image; performing brightness enhancement on the first image to obtain a second image; performing gamma correction and sharpening processing on the second image to obtain a third image and a fourth image respectively; and fusing the third image and the fourth image to obtain a final high-quality image.

[0056] Through white balance processing, color deviation can be reduced; through brightness enhancement, the contrast of the image can be improved; through gamma correction, underexposure or overexposure can be repaired; through sharpening processing, the detail level and clarity of the image can be enhanced. By fusing the images after gamma correction and sharpening processing, the negative effects of separate processing can be avoided and the brightness and details can be balanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The present invention will be further described below in conjunction with the accompanying drawings.

[0058] Figure 1 A flowchart of an image signal processing method is provided for an embodiment of the present invention;

[0059] Figure 2 An architecture diagram of an image signal processing system is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] The embodiment of the present invention provides an image signal processing method. Figure 1 , Figure 1 A flowchart of an image signal processing method provided by an embodiment of the present invention. The method comprises the following steps:

[0062] S101, obtaining an image to be processed.

[0063] S102, performing white balance processing on the image to be processed to obtain a first image.

[0064] S103: Perform brightness enhancement on the first image to obtain a second image.

[0065] S104, performing gamma correction and sharpening processing on the second image to obtain a third image and a fourth image respectively.

[0066] S105: Fusing the third image and the fourth image to obtain a final high-quality image.

[0067] Among them, the image to be processed is an RGB image.

[0068] Based on an image signal processing method provided by an embodiment of the present invention, color deviation is reduced by white balance processing; image contrast is improved by brightness enhancement; underexposure or overexposure can be repaired by gamma correction; and image detail level and clarity can be enhanced by sharpening processing. By fusing the images after gamma correction and sharpening processing, the negative effects caused by separate processing can be avoided, and brightness and details can be balanced.

[0069] In one implementation, after gamma correction, the brightness and contrast of the image are improved, but details may be lost. Sharpening can enhance these details and make the image clearer. When the two are combined, the benefits of brightness adjustment can be retained while recovering the details brought by sharpening.

[0070] In one embodiment, step S102 includes:

[0071] Step 1: Divide the image to be processed into multiple sub-blocks.

[0072] Step 2: Convert each sub-block into a grayscale image and perform normalization within the block to obtain the weight coefficient matrix of each sub-block:

[0073] ;

[0074] in, is the weight coefficient matrix; k is the sub-block identifier; m and n are the sizes of the sub-blocks, indicating that the k-th sub-block has m rows and n columns of pixels; is a grayscale image; and Is the pixel position identifier.

[0075] Step 3: According to the weight coefficient matrix of each sub-block, the weighted mean and standard deviation of each RGB channel of each sub-block are calculated:

[0076] ;

[0077] in, , is the channel identifier; is the weighted mean of the k-th sub-block channel C; The standard deviation of the k-th sub-block channel C; is the kth sub-block at position The C channel value of

[0078] Step 4: Calculate the weighted intensity of the RGB channels of the image to be processed based on the weighted mean and standard deviation of multiple sub-blocks:

[0079] ;

[0080] in, , is the channel identifier; represents the weighted strength; k and r are sub-block identifiers; p is the number of sub-blocks; is the weighted mean; s is the standard deviation.

[0081] Step 5: Get the color correction coefficient based on the weighted intensity of each channel:

[0082] ;

[0083] Step six: adjust the RGB channel values ​​of the image to be processed according to the color correction coefficient to obtain a first image.

[0084] In one implementation, the specific method of converting each sub-block into a grayscale image may be: ; Where R, G, and B are the red, green, and blue channel values ​​of the sub-block respectively.

[0085] In one implementation, the image can be divided into 12 sub-blocks on average. By calculating the weighted mean and standard deviation of each sub-block, the color and brightness distribution of the local area can be more accurately reflected. By dividing the image into sub-blocks, the error accumulation caused by global image processing is reduced, and the color correction coefficient is adaptively calculated according to the local features of the image, which improves the robustness of the algorithm under different image conditions and can process images with complex lighting conditions or hue changes. In different shooting environments, this method can effectively optimize the color to keep the image color natural and consistent.

[0086] In one embodiment, step S103 includes:

[0087] Step 1: Convert the first image to the HSV space to obtain the original hue component, saturation component and lightness component.

[0088] Step 2: Perform histogram equalization on the original brightness component to obtain an enhanced brightness component.

[0089] Step three, combining the enhanced lightness component with the original hue component and saturation component to obtain a new HSV image, and converting the new HSV image back to an RGB image to obtain a second image.

[0090] In one implementation, after converting the image to the HSV color space, the three components of hue, saturation, and brightness are separated, so that the brightness enhancement process of the image can be independent of the color adjustment. This method avoids the color distortion that may be caused when performing brightness enhancement in the RGB space, ensuring that the color remains natural and accurate.

[0091] In one implementation, by performing histogram equalization on the luminance component, the brightness distribution can be stretched to improve the details and visibility in the image.

[0092] In one embodiment, step S104 includes:

[0093] Performing adaptive gamma correction on the second image to obtain a third image;

[0094] The second image is sharpened using a standardized sharpening mask to obtain a fourth image; specifically:

[0095] Step 1: According to the preset Gaussian filter , for the second image Perform blur processing to obtain a blurred image .

[0096] Step 2: Calculate the difference between the second image and the blurred image , and perform standardization to obtain a detailed image Among them, standardization The difference image can be normalized by linear stretching.

[0097] Step 3: Add the detail image back to the second image to obtain the fourth image .

[0098] In one implementation, adaptive gamma correction may employ adaptive gamma calculation based on local brightness distribution, by adopting a lower gamma value in a low brightness area to increase brightness, and adopting a higher gamma value in a high brightness area to reduce overexposure and optimize details in dark and bright areas.

[0099] In one implementation, the standard deviation of the Gaussian filter can be set to 20, which can effectively capture large-scale edge information in the image while smoothing details as much as possible to avoid excessive noise or tiny details affecting the sharpening effect.

[0100] In one implementation, the linear stretching calculation formula is:

[0101] ;

[0102] The preset range can be set ; Standardization can avoid over-enhancement or over-compression of detail images, thus ensuring a more uniform and natural enhancement effect. and are the minimum and maximum pixel values ​​in the graph DT, respectively. Is the pixel position identifier.

[0103] In one embodiment, step S105 includes:

[0104] Step 1, performing Laplace weight calculation on the target image to obtain a first weight map corresponding thereto; the target image is any one of the third image and the fourth image;

[0105] Step 2: Calculate the saliency weight of the target image to obtain its corresponding second weight map;

[0106] Step 3: Obtain the corresponding fusion weight map according to the first weight map and the second weight map corresponding to the third image and the fourth image:

[0107] ;

[0108] Wherein, K and i are image identifiers, when equal to 1, it corresponds to the third image; when equal to 2, it corresponds to the fourth image; is the first weight graph; is the second weight graph; is the normalized fusion weight map; T is the temporary weight map generated by the calculation process;

[0109] Step 4: Perform weighted fusion on the third image and the fourth image according to the fusion weight map to obtain a final high-quality image.

[0110] In one implementation, performing Laplace weight calculation on the target image to obtain the first weight map corresponding thereto includes:

[0111] Step 1: Convert the target image to a grayscale image.

[0112] Step 2: Perform convolution operation on the grayscale image according to the preset Laplacian operator to obtain the Laplacian image.

[0113] Step 3: Take the absolute value of the Laplacian image to obtain the first weight map.

[0114] Among them, the convolution kernel of the Laplacian operator can be set to: [[0, 1, 0], [1, -4, 1], [0, 1, 0]].

[0115] The Laplacian operator can help extract high-frequency information in an image and emphasize the edges, details, and contours in the image. During the fusion process, using the Laplacian weight map helps maintain the structural information of the image and avoid loss of details. This is especially important for parts of the image with important details (such as object contours, edges, etc.).

[0116] In one implementation, performing saliency weight calculation on the target image to obtain the corresponding second weight map includes:

[0117] Step 1: Convert the target image to the LAB color space to obtain the fifth image, and calculate the mean of each channel thereof.

[0118] Step 2: Calculate the second weight map based on the difference between each pixel in the fifth image and the mean value of the entire image:

[0119] ;

[0120] in, is a location identifier; is the second weight graph; are the three channels of the fifth image; Represents the channel mean.

[0121] Saliency detection can weight image regions based on their visual saliency, which helps ensure that the visual focus part of the image remains prominent after fusion and is not overwhelmed by the influence of other regions.

[0122] In one implementation, by combining Laplacian weights and saliency weights, important details and areas in an image can be more intelligently identified and enhanced, thereby retaining more details, clarity, and visual appeal during fusion.

[0123] In one embodiment, weighted fusion is performed on the third image and the fourth image according to the fusion weight map to obtain a final high-quality image, including:

[0124] According to the fusion weight map, the third image and the fourth image are subjected to multi-scale weighted fusion using the Gaussian pyramid and the Laplacian pyramid to obtain the final high-quality image:

[0125] ;

[0126] in, represents the third image, represents the fourth image, It is the Gaussian pyramid decomposition of the fused weight map; is the Laplacian pyramid decomposition of the input image; Represents the level of the pyramid. The output image is obtained by upsampling the lowest level image and then adding it to the higher level images until the first level is reached and added to the output. The number of levels of the pyramid can be 3.

[0127] Through multi-level and multi-scale weighted fusion, different detail levels of the image (from coarse to fine) can be appropriately enhanced, so that each area of ​​the image can be finely processed. High-frequency details (such as edges, textures) and low-frequency information (such as lighting, color) are processed and fused separately, ensuring that the image has a stronger sense of hierarchy and richer details and light and shadow performance.

[0128] The embodiment of the present invention provides an image signal processing system. Figure 2 , Figure 2 The following is an architecture diagram of an image signal processing system provided by an embodiment of the present invention. The system includes:

[0129] The data acquisition module is used to acquire the image to be processed.

[0130] The color correction module is used to perform white balance processing on the image to be processed to obtain a first image.

[0131] The brightness enhancement module is used to enhance the brightness of the first image to obtain a second image.

[0132] The detail enhancement module is used to perform gamma correction and sharpening processing on the second image to obtain a third image and a fourth image respectively; and fuse the third image and the fourth image to obtain a final target image.

[0133] Among them, the image to be processed is an RGB image.

[0134] An image signal processing system provided by an embodiment of the present invention can reduce color deviation through white balance processing; improve image contrast through brightness enhancement; repair underexposure or overexposure through gamma correction; and enhance image detail level and clarity through sharpening processing. By fusing the images after gamma correction and sharpening processing, the negative effects caused by separate processing can be avoided, and the brightness and details can be balanced.

[0135] An embodiment of the present invention provides a pan / tilt device, on which a computer program is embedded. When the program is executed, any image signal processing method provided by the embodiment of the present invention can be implemented.

[0136] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A method for processing an image signal, characterized in that: The method comprises: Acquire an image to be processed; the image to be processed is an RGB image; Performing white balance processing on the image to be processed to obtain a first image; Performing brightness enhancement on the first image to obtain a second image; performing gamma correction and sharpening processing on the second image to obtain a third image and a fourth image respectively; fusing the third image and the fourth image to obtain a final high-quality image; The performing white balance processing on the image to be processed to obtain the first image comprises: Dividing the image to be processed into a plurality of sub-blocks; Convert each subblock to a grayscale image: ； Where R, G, and B are the red, green, and blue channel values ​​of the sub-block respectively; Perform normalization within the block to obtain the weight coefficient matrix of each sub-block; ； in, is the weight coefficient matrix; k is the sub-block identifier; m and n are the sizes of the sub-blocks, indicating that the k-th sub-block has m rows and n columns of pixels; is a grayscale image; and is the pixel position identifier; According to the weight coefficient matrix of each sub-block, the weighted mean and standard deviation of each RGB channel of each sub-block are calculated: ； in, , is the channel identifier; is the weighted mean of the k-th sub-block channel C; The standard deviation of the k-th sub-block channel C; is the kth sub-block at position The C channel value of According to the weighted mean and standard deviation of multiple sub-blocks, the weighted intensity of the RGB channels of the image to be processed is calculated: ； in, , is the channel identifier; represents the weighted strength; k and r are sub-block identifiers; p is the number of sub-blocks; u is the weighted mean; is the standard deviation; According to the weighted intensity of each channel, the color correction coefficient is obtained: ； According to the color correction coefficient, the RGB channel values ​​of the image to be processed are adjusted to obtain a first image.

2. The image signal processing method according to claim 1, characterized in that: The step of enhancing the brightness of the first image to obtain a second image comprises: Convert the first image to HSV space to obtain the original hue component, saturation component and lightness component; Perform histogram equalization on the original brightness component to obtain an enhanced brightness component; The enhanced lightness component is combined with the original hue component and saturation component to obtain a new HSV image, and the new HSV image is converted back to an RGB image to obtain a second image.

3. The image signal processing method according to claim 1, characterized in that: The performing gamma correction and sharpening processing on the second image to obtain a third image and a fourth image respectively comprises: performing adaptive gamma correction on the second image to obtain a third image; According to a preset Gaussian filter, blurring the second image to obtain a blurred image; Calculating the difference between the second image and the blurred image, and performing standardization processing to obtain a detail image; The detail image is added back to the second image to obtain a fourth image.

4. The image signal processing method according to claim 1, characterized in that: The fusing the third image and the fourth image to obtain a final high-quality image includes: Performing Laplace weight calculation on a target image to obtain a first weight map corresponding to the target image; the target image is any one of the third image and the fourth image; Calculate the saliency weight of the target image to obtain its corresponding second weight map; According to the first weight map and the second weight map corresponding to the third image and the fourth image, the corresponding fusion weight map is obtained: ； Wherein, K and i are image identifiers. When K is equal to 1, it corresponds to the third image; when K is equal to 2, it corresponds to the fourth image; LW is the first weight map; SW is the second weight map; FW is the normalized fusion weight map; T is the temporary weight map generated by the calculation process; According to the fusion weight map, the third image and the fourth image are weighted fused to obtain a final high-quality image.

5. The image signal processing method according to claim 4, characterized in that: The Laplace weight calculation is performed on the target image to obtain the first weight map corresponding to the target image, comprising: Convert the target image to a grayscale image; Performing a convolution operation on the grayscale image according to a preset Laplacian operator to obtain a Laplacian image; An absolute value of the Laplacian image is taken to obtain a first weight map.

6. The image signal processing method according to claim 4, characterized in that: The step of calculating the saliency weight of the target image to obtain the corresponding second weight map includes: Convert the target image to the LAB color space to obtain the fifth image, and calculate the mean of each channel thereof; According to the difference between each pixel in the fifth image and the mean value of the whole image, the second weight map is calculated: ； Wherein, (x, y) is the location identifier; SW is the second weight map; are the three channels of the fifth image; mean represents the channel mean.

7. The image signal processing method according to claim 4, characterized in that: The step of performing weighted fusion on the third image and the fourth image according to the fusion weight map to obtain a final high-quality image includes: According to the fusion weight map, the third image and the fourth image are subjected to multi-scale weighted fusion using the Gaussian pyramid and the Laplacian pyramid to obtain a final high-quality image.

8. An image signal processing system, characterized in that: Used to implement the image signal processing method according to claim 1; the system comprises: A data acquisition module, used to acquire an image to be processed; the image to be processed is an RGB image; A color correction module, used for performing white balance processing on the image to be processed to obtain a first image; A brightness enhancement module, used to enhance the brightness of the first image to obtain a second image; The detail enhancement module is used to perform gamma correction and sharpening processing on the second image to obtain a third image and a fourth image respectively; and fuse the third image with the fourth image to obtain a final target image.

9. A pan / tilt device having a computer program embedded therein, characterized in that: When the program is executed, an image signal processing method as claimed in any one of claims 1 to 7 is implemented.

Images