Image processing method and device, equipment and storage medium for black edge optimization
By detecting the highlighted area and adjusting the stretch gain value to optimize the black edge pixel points, the local halo and black edge problems caused by excessive image dynamic range of CMOS image sensor under high dynamic range is solved, and high-quality black edge optimization is achieved deal with.
Patent Information
- Application Number
- CN202111525668.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-14
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-12-14
AI Technical Summary
When the CMOS image sensor is exposed at a high dynamic range image, the exposure ratio of three frames in long, medium and short is fixed, resulting in excessive dynamic range of images in scenes such as night and serious local halos and black edges. The existing technology sacrifices the overall dynamic range of the image. Processing, reducing image quality.
By detecting the highlighted area of the image, adjusting the stretch gain value, optimizing the output data of black edge pixel points, avoiding the filtering of black edge pixel points, retaining the overall dynamic range of the image, and achieving optimization processing of black edges.
While ensuring the overall dynamic range of the image, the black edges are effectively optimized, the image processing quality is improved, and the filtering of black edge pixels and the sacrifice of image dynamic range is avoided.
Smart Images

Figure CN114187204B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to a black edge optimized image processing method and a device, equipment and storage medium thereof. Background Art
[0002] When the CMOS image sensor is performing high-dynamic range (HDR) image exposure, the exposure ratio is fixed due to the limitation of the sensor. For example, the fixed exposure ratio of long, medium and short frames is 256:16:1. In some scene environments such as night, the dynamic range of the long, medium and short frames will differ too much. When tone mapping is performed on the fused HDR image, the local stretching ratio will be too high, resulting in local halo phenomenon when the pixel data is converted from the brightness domain to the RAW domain, that is, the black edge phenomenon around the bright light.
[0003] Currently, conventional Tone Mapping algorithms do not specifically address the above-mentioned local halo and black edge problems. Algorithms in the prior art can avoid the local halo and black edge problems by sacrificing the overall dynamic range of the image, but sacrificing the overall dynamic range of the image will reduce the image quality.
[0004] A Chinese patent with publication number CN 105450908 A discloses a method and device for eliminating black edges from video images using a black-edge optimized image processing method. The method uses a double preset threshold value and a method for judging continuous changes to detect the black edges of the image in the memory and then obtain the starting position of the data valid image signal, ultimately achieving the purpose of eliminating the black edges. The device includes: a data acquisition module, a black edge detection and statistics module, a black edge elimination module, a setting receiving module, and an output module. The invention uses a double preset threshold value and a method for judging continuous changes to detect the black edges of the image in the memory and then obtain the starting position of the data valid image signal, ultimately achieving the purpose of eliminating the black edges, but at the expense of the overall dynamic range of the image.
[0005] Therefore, it is necessary to provide an image processing method for optimizing black edges and a device, equipment and storage medium thereof to solve the above-mentioned problems in the prior art. Summary of the invention
[0006] The object of the present invention is to provide an image processing method for optimizing black edges and a device, equipment and storage medium thereof, so as to optimize the black edges while retaining the overall dynamic range of the image.
[0007] To achieve the above object, the black edge optimized image processing method of the present invention comprises the steps of:
[0008] Providing an original image, and obtaining original data of all pixels based on the original image;
[0009] Processing the original image to obtain a first image, and acquiring first brightness data of all pixels according to the first image;
[0010] Preset a brightness range, perform tone mapping on the first image to obtain a second image, and obtain second brightness data of all pixels according to the second image, so that the second brightness data of all pixels are within the brightness range;
[0011] Acquire a stretching gain value according to the first brightness data and the second brightness data, and acquire output data according to the original data and the stretching gain value;
[0012] Determine a highlight area of the image according to the second brightness data of all the pixels, and obtain black-border pixels according to the second brightness data of the pixels in the highlight area and the output data;
[0013] The stretching gain value is adjusted to adjust the output data of the black-border pixel point to obtain black-border optimized data, and the black-border optimized data is output as the endpoint data of the image.
[0014] The beneficial effects of the black edge optimized image processing method of the present invention are:
[0015] The highlight area of the image is determined by the first brightness data, and the black-border pixel points are obtained by the second brightness data and output data of the pixel points in the highlight area; after obtaining the highlight area and the black-border pixel points in the highlight area, the stretching gain value is adjusted to adjust the output data of the black-border pixel points to obtain black-border optimization data, and the black-border optimization data is output as the endpoint data of the image; the filtering of black-border pixel points and the sacrifice of the overall dynamic range of the image are avoided, so that the black border is optimized while ensuring the overall dynamic range of the image, thereby ensuring the processing quality of the black border of the image.
[0016] Optionally, the step of obtaining output data according to the original data and the stretch gain value includes:
[0017] A multiplication operation is performed on the original data and the stretching gain value to obtain the output data, and a format of the output data is consistent with a data format of the original data.
[0018] Optionally, the step of determining a highlight area of the image according to the second brightness data of all pixels includes:
[0019] Preset brightness threshold and number threshold;
[0020] Divide the second image into a plurality of windows, each of which has a size of M×N, where both M and N are positive integers;
[0021] Calculate the brightness of the pixel in the window, compare the brightness of the pixel with the brightness threshold, and mark the pixel as a highlighted pixel when the brightness of the pixel is greater than the brightness threshold;
[0022] The number of the highlighted pixels in the window is counted, and the highlighted area is obtained according to the comparison result of the number of the highlighted pixels and the number threshold. The beneficial effect is that since the black edge pixels are generally near the highlighted area, the highlighted area is detected first, which provides regional positioning for the search and determination of the black edge pixels, thereby improving the detection efficiency of the black edge pixels.
[0023] Optionally, the step of acquiring the highlight area according to the comparison result of the number of the highlight pixels and the number threshold comprises:
[0024] When the number of highlighted pixels in the window is greater than or equal to the number threshold, the window is determined to be a highlighted area. The beneficial effect is that the highlighted area is determined by counting the number of highlighted pixels in the window, thereby providing a basis for determining the black-bordered pixels in the highlighted area. Since the edge of the highlighted area may have black-bordered pixels, the highlighted area provides an area and range selection for detecting and acquiring black-bordered pixels, thereby improving the efficiency of determining the black-bordered pixels.
[0025] Optionally, the step of acquiring the highlight area according to the comparison result of the number of the highlight pixels and the number threshold comprises:
[0026] When the number of highlighted pixels in the window is less than the number threshold, the window is determined to be a non-highlight area, and the output data of all pixels in the non-highlight area are output as the end point data of the image. The beneficial effect is that after the window is detected to be a non-highlight area, the pixels in the non-highlight area do not need to be processed, and the output data of all pixels in the non-highlight area are directly output, thereby improving the data processing speed.
[0027] Optionally, the step of acquiring black-edge pixel points according to the second brightness data of pixel points in the highlight area and the output data includes:
[0028] A first edge intensity and a second edge intensity are respectively obtained according to the second brightness data of the pixel points in the highlight area and the local edge information of the output data, and a black edge pixel point is obtained according to the first edge intensity and the second edge intensity.
[0029] Optionally, the first edge strength includes a first horizontal edge strength and a first vertical edge strength, the second edge strength includes a second horizontal edge strength and a second vertical edge strength, and the steps of respectively acquiring the first edge strength and the second edge strength according to the first brightness data of the pixel points in the highlight area and the local edge information of the output data include:
[0030] Calculating the edge strength in the horizontal direction of the second brightness data to obtain the first horizontal edge strength, and calculating the edge strength in the vertical direction of the first brightness data to obtain the first vertical edge strength;
[0031] The edge strength in the horizontal direction of the output data is calculated to obtain the second horizontal edge strength, and the edge strength in the vertical direction of the output data is calculated to obtain the second vertical edge strength.
[0032] Optionally, the step of acquiring black edge pixel points according to the first edge strength and the second edge strength includes:
[0033] A multiplication operation is performed on the first horizontal edge strength and the second horizontal edge strength to obtain horizontal edge information, a multiplication operation is performed on the first vertical edge strength and the second vertical edge strength to obtain vertical edge information, and a black edge pixel point is determined according to the horizontal edge information and the vertical edge information.
[0034] Optionally, the step of determining black edge pixel points according to the horizontal edge information and the vertical edge information includes:
[0035] When any one of the horizontal edge information and the vertical edge information is less than 0, it is determined that the pixel point corresponding to the horizontal edge information and the vertical edge information is a black edge pixel point.
[0036] Optionally, when the horizontal edge information and the vertical edge information are both greater than or equal to 0, the pixel point corresponding to the horizontal edge information is regarded as a normal pixel point, and the output data of the normal pixel point is output as the endpoint data of the image.
[0037] Optionally, the black edge optimization data includes first black edge optimization data, and the step of adjusting the stretch gain value to adjust the output data of the black edge pixel point to obtain the black edge optimization data includes:
[0038] filtering the stretching gain value to obtain a first stretching gain value;
[0039] The first stretch gain value is multiplied by the original data of the black-border pixel point to obtain the first black-border optimized data. The beneficial effect is that the black-border pixel value is processed by filtering the stretch gain value to obtain the first stretch gain value, and the first stretch gain value is multiplied by the original data of the black-border pixel point to obtain the first black-border optimized data, and the overall dynamic range of the image is retained.
[0040] Optionally, the black edge optimization data includes second black edge optimization data, and the step of adjusting the stretch gain value to adjust the output data of the black edge pixel point to obtain the black edge optimization data includes:
[0041] Setting a correction control coefficient, performing a multiplication operation on the correction control coefficient and the stretching gain value to obtain a second stretching gain value;
[0042] The second stretching gain value is multiplied by the original data of the black-border pixel point to obtain the second black-border optimized data. The beneficial effect is that the stretching gain value is adjusted by the correction control coefficient to obtain the second stretching gain value, and the second stretching gain value is multiplied by the original data of the black-border pixel point to obtain the second black-border optimized data, thereby realizing the processing of the black-border pixel value.
[0043] The present invention also provides an image processing device for optimizing black edges, comprising:
[0044] A data processing module, used to provide an original image, obtain original data of all pixels respectively according to the original image, process the original image to obtain a first image, obtain first brightness data of all pixels according to the first image, preset a brightness range, perform tone mapping on the first image to obtain a second image, obtain second brightness data of all pixels according to the second image, so that the second brightness data of all pixels are within the brightness range, obtain a stretching gain value according to the first brightness data and the second brightness data of each pixel, and obtain output data according to the original data and the stretching gain value;
[0045] A highlight detection module, used for determining a highlight area of an image according to the second brightness data of all pixels;
[0046] A black edge pixel acquisition module, used for acquiring black edge pixel points according to the second brightness data of the pixel points in the highlight area and the output data;
[0047] A black edge processing module, used for adjusting the stretching gain value to adjust the output data of the black edge pixel points to obtain black edge optimization data;
[0048] The output module is used to output the black edge optimization data as the endpoint data of the image.
[0049] The beneficial effects of the black edge optimized image processing device of the present invention are:
[0050] The data processing module is used to obtain original data, first data, second data and output data to support the acquisition of the highlight area and the black-edge pixel points. The highlight area of the image is determined by the highlight detection module to define an area for determining the black-edge pixel points, thereby improving the efficiency of acquiring the black-edge pixel points; the black-edge pixel points are acquired from the highlight area by the black-edge pixel acquisition module; the stretching gain value is adjusted by the black-edge processing module to adjust the output data of the black-edge pixel points to obtain black-edge optimized data, and the black-edge optimized data is output as the endpoint data of the image by the output module; the optimization of the black-edge pixel points is achieved, the filtering of the black-edge pixel points and the sacrifice of the overall dynamic range of the image are avoided, so that the black edge is optimized while ensuring the overall dynamic range of the image, thereby ensuring the processing quality of the black edge of the image.
[0051] The present invention also provides a device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the image processing method for black edge optimization is implemented when the processor executes the program.
[0052] The present invention also provides a storage medium on which a program is stored, and when the program is executed by a processor, the image processing method for optimizing black edges is implemented.
[0053] The beneficial effects of the device and the storage medium of the present invention are:
[0054] The processing of the black edge pixels is achieved, the filtering of the black edge pixels is avoided, and the sacrifice of the overall dynamic range of the image is avoided, so that the black edges are optimized while ensuring the overall dynamic range of the image, and the processing quality of the black edges of the image is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 A flowchart of an image processing method for black edge optimization according to an embodiment of the present invention;
[0056] Figure 2 A flowchart of determining a highlight area of an image according to first brightness data according to an embodiment of the present invention;
[0057] Figure 3 It is a structural block diagram of an image processing device for black edge optimization according to an embodiment of the present invention. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.
[0059] In view of the problems existing in the prior art, the embodiments of the present invention provide an image processing method for optimizing black edges and its apparatus, device and storage medium, so as to optimize the black edges while retaining the overall dynamic range of the image.
[0060] Figure 1 The present invention is a flowchart of an image processing method for optimizing black edges according to an embodiment of the present invention.
[0061] To achieve the above purpose, refer to Figure 1 , the image processing method for black edge optimization comprises the steps of:
[0062] S0: providing an original image, and obtaining original data of all pixels according to the original image;
[0063] S1: Processing the original image to obtain a first image, and acquiring first brightness data of all pixels according to the first image;
[0064] S2: Preset a brightness range, perform tone mapping on the first image to obtain a second image, and obtain second brightness data of all pixels according to the second image, so that the second brightness data of all pixels are within the brightness range;
[0065] S3: acquiring a stretching gain value according to the first brightness data and the second brightness data, and acquiring output data according to the original data and the stretching gain value;
[0066] S4: determining a highlight area of the image according to the second brightness data of all the pixels, and acquiring black-edge pixels according to the second brightness data of the pixels in the highlight area and the output data;
[0067] S5: Adjust the stretching gain value to adjust the output data of the black-border pixel points to obtain black-border optimized data, and output the black-border optimized data as the endpoint data of the image.
[0068] The advantages of the black-border optimized image processing method of the present invention are: determining the highlight area of the image through the first brightness data, and obtaining the black-border pixel points through the second brightness data and output data of the pixel points of the highlight area; after obtaining the highlight area and the black-border pixel points of the highlight area, adjusting the stretching gain value to adjust the output data of the black-border pixel points to obtain black-border optimized data, and outputting the black-border optimized data as the endpoint data of the image; avoiding the filtering of black-border pixel points and the sacrifice of the overall dynamic range of the image, thereby optimizing the black border while ensuring the overall dynamic range of the image, thereby ensuring the processing quality of the black border of the image.
[0069] The algorithms in the prior art can avoid the problems of local halo and black edge by sacrificing the overall dynamic range of the image, such as directly filtering out the black edge pixels, but the overall dynamic range of the image is sacrificed, which will reduce the image quality. The present invention first detects the highlight area of the image, then searches for black edge pixels in the highlight area of the image, and finally optimizes the black edge pixels instead of filtering out the black edge pixels, thereby ensuring the overall dynamic range of the image while optimizing the black edge pixels and ensuring the optimized black edge quality of the image.
[0070] In some embodiments, the image in step S5 is an image in a high dynamic range image fusion process.
[0071] In some embodiments, the brightness range in step S2 is comprehensively determined based on image processing requirements, shooting equipment, and scenes.
[0072] In some optional embodiments, the brightness range is in the range of 10bit-12bit, wherein the brightness range of 10bit is 0-1023 and the brightness range of 12bit is 0-4095.
[0073] As an optional implementation manner of the present invention, the step of obtaining output data according to the original data and the stretch gain value includes:
[0074] A multiplication operation is performed on the original data and the stretching gain value to obtain the output data, and a format of the output data is consistent with a data format of the original data.
[0075] In some specific embodiments, a stretch gain value is obtained according to the first brightness data and the second brightness data, and a calculation formula for obtaining output data according to the original data and the stretch gain value is:
[0076]
[0077] imageBayer′=gain×imageBayer
[0078] Wherein, gain is the stretching gain value, imageY' is the second brightness data, imageY is the first brightness data, imageBayer' is the output data, and imageBayer is the original data.
[0079] In some embodiments, both the original data imageBayer and the output data imageBayer′ are data in RAW format.
[0080] Figure 2 This is a flow chart of determining a highlight area of an image according to the first brightness data according to an embodiment of the present invention.
[0081] As an optional embodiment of the present invention, refer to Figure 2 In step S4, the step of determining the highlight area of the image according to the second brightness data of all the pixels includes the following steps:
[0082] S401: Preset brightness threshold and number threshold;
[0083] S402: Divide the second image into a plurality of windows, each of which has a size of M×N, where M and N are both positive integers;
[0084] S403: Calculate the brightness of the pixel in the window, compare the brightness of the pixel with the brightness threshold, and mark the pixel as a highlight pixel when the brightness of the pixel is greater than the brightness threshold;
[0085] S404: Count the number of the highlighted pixels in the window, and obtain the highlighted area according to the comparison result of the number of the highlighted pixels and the number threshold. The advantage is that since the black-edge pixels are generally near the highlighted area, the highlighted area is detected first, which provides regional positioning for the search and determination of the black-edge pixels, thereby improving the detection efficiency of the black-edge pixels.
[0086] As an optional implementation of the present invention, in step S404, the step of obtaining the highlight area according to the comparison result of the number of highlight pixels and the number threshold comprises:
[0087] When the number of highlighted pixels in the window is greater than or equal to the number threshold, the window is determined to be a highlighted area. The advantage is that the highlighted area is determined by counting the number of highlighted pixels in the window, thereby providing a basis for determining the black-bordered pixels in the highlighted area. Since the edge of the highlighted area may have black-bordered pixels, the highlighted area provides area and range selection for detecting and acquiring black-bordered pixels, thereby improving the efficiency of determining the black-bordered pixels.
[0088] As an optional implementation of the present invention, in step S204, the step of obtaining the highlight area according to the comparison result of the number of highlight pixels and the number threshold further includes:
[0089] When the number of highlighted pixels in the window is less than the number threshold, the window is determined to be a non-highlight area, and the output data of all pixels in the non-highlight area are output as the end point data of the image. The beneficial effect is that after the window is detected to be a non-highlight area, the pixels in the non-highlight area do not need to be processed, and the output data of all pixels in the non-highlight area are directly output, thereby improving the data processing speed.
[0090] In some specific embodiments of the present invention, the specific step of determining the highlight area of the image according to the second brightness data includes:
[0091] Preset a brightness threshold T and a number threshold K, where 0≤T≤the maximum brightness value of the image corresponding to the second brightness data, and 0≤K≤M×N-1;
[0092] In some optional embodiments, the brightness threshold T is determined according to the bit width of the original image and the shooting scene, and the value range of the brightness threshold T is 0-800.
[0093] In some other optional embodiments, the brightness threshold T ranges from 0 to 3000.
[0094] Divide the second image into a plurality of windows, each of which has a size of M×N, where M and N are both positive integers;
[0095] In some embodiments, the size of the window is M×N, which means that there are M×N pixels in each window; according to the image processing requirements, the value of M×N is selected from 5×5, 7×7 and 9×9 to select the most reasonable or appropriate one, so as to avoid the negative impact caused by the fading of the black border due to the window being too large, and avoid destroying the normal black border boundary, so as to improve the accuracy of determining the black border pixels in the highlight area.
[0096] Calculate the brightness of the pixel in the window, compare the brightness of the pixel with the brightness threshold, and mark the pixel as a highlighted pixel when the brightness Lum of the pixel is greater than the brightness threshold T, that is, Lum>T;
[0097] Counting the number n of the highlighted pixels in the window;
[0098] When the number n of highlighted pixels in the window is greater than or equal to the number threshold K, that is, n≥K, the window is determined to be a highlighted area;
[0099] When the number n of highlighted pixel points within the window is less than the number threshold K, that is, n < K, determine that the window is a non-highlighted area, and output the output data imageBayer' of the pixel points in the non-highlighted area as the end data of the image.
[0100] As an optional implementation manner of the present invention, in step S4, the step of obtaining black-edge pixel points according to the second luminance data and the output data of the pixel points in the highlighted area includes:
[0101] Obtain a first edge intensity and a second edge intensity respectively according to the second luminance data and the local edge information of the output data of the pixel points in the highlighted area, and obtain black-edge pixel points according to the first edge intensity and the second edge intensity.
[0102] In some embodiments, calculate the local edge information of the second luminance data and the output data respectively by any one of the Sobel operator, Prewitt operator, and Lapladan operator to obtain the first edge intensity and the second edge intensity. Since the Sobel operator, Prewitt operator, and Lapladan operator are common techniques in the art, they will not be elaborated here.
[0103] As an optional implementation manner of the present invention, the first edge intensity includes a first horizontal edge intensity and a first vertical edge intensity, and the second edge intensity includes a second horizontal edge intensity and a second vertical edge intensity. The step of obtaining the first edge intensity and the second edge intensity respectively according to the first luminance data and the local edge information of the output data of the pixel points in the highlighted area includes:
[0104] Calculate the edge intensity in the horizontal direction of the second luminance data to obtain the first horizontal edge intensity, and calculate the edge intensity in the vertical direction of the first luminance data to obtain the first vertical edge intensity;
[0105] Calculate the edge intensity in the horizontal direction of the output data to obtain the second horizontal edge intensity, and calculate the edge intensity in the vertical direction of the output data to obtain the second vertical edge intensity.
[0106] As an optional implementation manner of the present invention, in step S302, the step of obtaining black-edge pixel points according to the first edge intensity and the second edge intensity includes:
[0107] A multiplication operation is performed on the first horizontal edge strength and the second horizontal edge strength to obtain horizontal edge information, a multiplication operation is performed on the first vertical edge strength and the second vertical edge strength to obtain vertical edge information, and a black edge pixel point is determined according to the horizontal edge information and the vertical edge information.
[0108] As an optional implementation manner of the present invention, the step of determining black edge pixel points according to the horizontal edge information and the vertical edge information includes:
[0109] When any one of the horizontal edge information and the vertical edge information is less than 0, it is determined that the pixel point corresponding to the horizontal edge information and the vertical edge information is a black edge pixel point.
[0110] As an optional implementation manner of the present invention, the step of determining black edge pixel points according to the horizontal edge information and the vertical edge information includes:
[0111] When the horizontal edge information and the vertical edge information are both greater than or equal to 0, the pixel corresponding to the horizontal edge information is regarded as a normal pixel, and the output data of the normal pixel is output as the end point data of the image.
[0112] In some specific embodiments of the present invention, the specific steps of obtaining black edge pixel points according to the first brightness data and output data of the pixel points in the highlight area include:
[0113] Calculate the edge strength in the horizontal direction of the second brightness data imageY' to obtain a first horizontal edge strength YX, and calculate the edge strength in the vertical direction of the first brightness data to obtain a first vertical edge strength YY;
[0114] The edge strength in the horizontal direction of the output data imageBayer' is calculated to obtain a second horizontal edge strength BayerX, and the edge strength in the vertical direction of the output data is calculated to obtain a second vertical edge strength BayerY.
[0115] In some embodiments, the edge strength in the horizontal and vertical directions of the second brightness data imageY' can be calculated by the Sobel operator to obtain the first horizontal edge strength YX and the first vertical edge strength YY, and the edge strength in the horizontal and vertical directions of the output data imageBayer' can be calculated by the Sobel operator to obtain the second horizontal edge strength BayerX and the second vertical edge strength BayerY.
[0116] In other embodiments, the edge strength in the horizontal and vertical directions of the first brightness data imageY' and the edge strength in the horizontal and vertical directions of the output data imageBayer' can also be calculated by the Lapladan operator to obtain the first horizontal edge strength YX, the first vertical edge strength YY, the second horizontal edge strength BayerX and the second vertical edge strength BayerY respectively.
[0117] Performing a multiplication operation on the first horizontal edge intensity YX and the second horizontal edge intensity BayerX to obtain horizontal edge information YX×BayerX, and performing a multiplication operation on the first vertical edge intensity YY and the second vertical edge intensity BayerY to obtain vertical edge information YY×BayerY;
[0118] When any one of the horizontal edge information YX×BayerX and the vertical edge information YY×BayerY is less than 0, it is determined that the pixel point corresponding to the horizontal edge information is a black edge pixel point;
[0119] When the horizontal edge information YX×BayerX and the vertical edge information YY×BayerY are both greater than or equal to 0, it is determined that the pixel corresponding to the horizontal edge information is a normal pixel, and the output data imageBayer' of the normal pixel is output as the end point data of the image.
[0120] In some embodiments, the black edge pixel points determined according to the horizontal edge information and the vertical edge information can be expressed by the following formula:
[0121]
[0122] Wherein, YX is the first horizontal edge strength, BayerX is the second horizontal edge strength, YY is the first vertical edge strength, and BayerY is the second vertical edge strength;
[0123] When flagBlack=1, it means that this pixel is a black-border pixel;
[0124] When flagBlack=0, it means that this pixel is a normal pixel.
[0125] In some embodiments of the present invention, in step S5, the black edge optimization data includes first black edge optimization data, and the step of adjusting the stretch gain value to adjust the output data of the black edge pixel point to obtain the black edge optimization data includes:
[0126] filtering the stretching gain value to obtain a first stretching gain value;
[0127] Performing a multiplication operation on the first stretching gain value and the original data of the black-border pixel point to obtain the first black-border optimization data;
[0128] After obtaining the first black edge optimization data, the first black edge optimization data is output as the end point data of the image. The first stretch gain value is obtained by filtering the stretch gain value, and the first stretch gain value is multiplied by the original data of the black edge pixel point to obtain the first black edge optimization data, thereby realizing the processing of the black edge pixel value and retaining the overall dynamic range of the image.
[0129] In some specific embodiments of the present invention, the specific steps of adjusting the stretch gain value to adjust the output data of the black edge pixel point to obtain the black edge optimized data include:
[0130] Performing filtering on the stretching gain value gain in a local range, wherein the window size of the local range is M×N, where M and N are both positive integers, to obtain a first stretching gain value gain';
[0131] A multiplication operation is performed on the first stretching gain value gain' and the original data imageBayer of the black edge pixel to obtain first black edge optimization data, and the calculation formula is as follows:
[0132]
[0133] flagBlack'=gain'×imageBayer
[0134] Among them, weight is the filter weight in the local range, It represents the weighted average of the stretch gain values of all pixels in the M×N window, and flagBlack' is the first black edge optimization data.
[0135] In some embodiments, the filtering weights weights in the local area may be adjusted according to the filter.
[0136] In some embodiments, the filtering weight weight within the local range may adopt an average value of 1, and the sum of the weights corresponding to all points in the window is M×N.
[0137] In some other embodiments of the present invention, in step S5, the black edge optimization data includes second black edge optimization data, and the step of adjusting the stretch gain value to adjust the output data of the black edge pixel point to obtain the black edge optimization data includes:
[0138] Setting a correction control coefficient, performing a multiplication operation on the correction control coefficient and the stretching gain value to obtain a second stretching gain value;
[0139] Performing a multiplication operation on the second stretching gain value and the original data of the black-border pixel point to obtain second black-border optimized data;
[0140] After the second black edge optimization data is obtained, the second black edge optimization data is output as the end point data of the image. The advantage is that the stretch gain value is adjusted by the correction control coefficient to obtain the second stretch gain value, and the second stretch gain value is multiplied by the original data of the black edge pixel point to obtain the second black edge optimization data, thereby realizing the processing of the black edge pixel value.
[0141] In some other specific embodiments of the present invention, the specific steps of processing the black edge pixels include:
[0142] Setting a correction control coefficient actor, performing a multiplication operation on the correction control coefficient actor and the stretching gain value gain to obtain a second stretching gain value gain";
[0143] In some embodiments, the value of the correction control coefficient actor is determined according to the shooting equipment, image processing requirements and shooting scene.
[0144] In some optional embodiments, the correction control coefficient actor has a value range greater than 1 and less than or equal to 5, that is, 1 <actor≤5。
[0145] The second stretching gain value "gain" is multiplied by the original data imageBayer of the black edge pixel to obtain the second black edge optimization data, and the calculation formula is as follows:
[0146] gain” = actor × gain;
[0147] flagBlack”=gain”×imageBayer
[0148] Among them, "flagBlack" is the second black border optimization data.
[0149] Figure 3 It is a structural block diagram of an image processing device for black edge optimization according to an embodiment of the present invention.
[0150] The present invention also provides an image processing device for optimizing black edges, referring to Figure 3 , the black edge optimized image processing device comprises:
[0151] The data processing module 1 is used to provide an original image, obtain the original data of all pixels respectively according to the original image, process the original image to obtain a first image, obtain the first brightness data of all pixels according to the first image, preset a brightness interval, perform tone mapping on the first image to obtain a second image, obtain the second brightness data of all pixels according to the second image, so that the second brightness data of all pixels are within the brightness interval, obtain a stretching gain value according to the first brightness data and the second brightness data of each pixel, and obtain output data according to the original data and the stretching gain value.
[0152] A highlight detection module 2, used for determining a highlight region of an image according to the second brightness data of all pixels;
[0153] A black edge pixel acquisition module 3, used for acquiring black edge pixel points according to the second brightness data of the pixel points in the highlight area and the output data;
[0154] The black edge processing module 4 is used to adjust the stretching gain value to adjust the output data of the black edge pixel points to obtain black edge optimization data;
[0155] The output module 5 is used to output the black edge optimization data as the end point data of the image.
[0156] The beneficial effects of the black edge optimized image processing device of the present invention are:
[0157] The data processing module 1 obtains original data, first data, second data and output data to support the acquisition of the highlight area and the black-edge pixel points. The highlight area of the image is determined by the highlight detection module 2 to define an area for determining the black-edge pixel points, thereby improving the efficiency of acquiring the black-edge pixel points; the black-edge pixel acquisition module 3 obtains black-edge pixel points from the highlight area; the black-edge processing module 4 adjusts the stretching gain value to adjust the output data of the black-edge pixel points to obtain black-edge optimized data, and the output module 5 outputs the black-edge optimized data as the endpoint data of the image; the optimization of the black-edge pixel points is achieved, the filtering of the black-edge pixel points and the sacrifice of the overall dynamic range of the image are avoided, so that the black edge is optimized while ensuring the overall dynamic range of the image, thereby ensuring the processing quality of the black edge of the image.
[0158] In some embodiments of the present invention, reference Figure 3 The steps of processing the black edges of the image by the image processing device with black edge optimization include:
[0159] (1) providing an original image through the data processing module 1, obtaining original data of all pixels according to the original image, processing the original image to obtain a first image, obtaining first brightness data of all pixels according to the first image, presetting a brightness range, performing tone mapping on the first image to obtain a second image, obtaining second brightness data of all pixels according to the second image, so that the second brightness data of all pixels are within the brightness range, obtaining a stretching gain value according to the first brightness data and the second brightness data of each pixel, and obtaining output data according to the original data and the stretching gain value;
[0160] (2) determining the highlight area of the image according to the second brightness data of all the pixels by the highlight detection module 2, the specific steps include:
[0161] Preset a brightness threshold and a number threshold, obtain a first image corresponding to the first brightness data, divide the first image into a plurality of windows, each of the windows has a size of M×N, and both M and N are positive integers;
[0162] Preset brightness threshold and number threshold;
[0163] Divide the second image into a plurality of windows, each of which has a size of M×N, where both M and N are positive integers;
[0164] Calculate the brightness of the pixel in the window, compare the brightness of the pixel with the brightness threshold, and mark the pixel as a highlighted pixel when the brightness of the pixel is greater than the brightness threshold;
[0165] Counting the number of the highlighted pixels in the window, and obtaining the highlighted area according to a comparison result between the number of the highlighted pixels and the number threshold;
[0166] The output module 5 is connected to the data processing module 1, and the output data of all the pixels in the non-highlight area are output as the end point data of the image through the output module 5;
[0167] (3) The black-border pixel acquisition module 3 acquires black-border pixel points according to the second brightness data and the output data of the pixel points in the highlight area, and the specific steps include:
[0168] Calculating the edge strength in the horizontal direction of the first brightness data to obtain a first horizontal edge strength, and calculating the edge strength in the vertical direction of the first brightness data to obtain a first vertical edge strength;
[0169] Calculating the edge strength in the horizontal direction of the output data to obtain a second horizontal edge strength, and calculating the edge strength in the vertical direction of the output data to obtain a second vertical edge strength;
[0170] Performing a multiplication operation on the first horizontal edge strength and the second horizontal edge strength to obtain horizontal edge information, and performing a multiplication operation on the first vertical edge strength and the second vertical edge strength to obtain vertical edge information;
[0171] When any one of the horizontal edge information and the vertical edge information is less than 0, it is determined that the pixel point corresponding to the horizontal edge information is a black edge pixel point.
[0172] When the horizontal edge information and the vertical edge information are both greater than or equal to 0, the pixel corresponding to the horizontal edge information is determined to be a normal pixel, and the output data of the normal pixel is output as the end point data of the image through the output module 5.
[0173] (4) The stretching gain value is adjusted by the black edge processing module 4 to adjust the output data of the black edge pixel point to obtain black edge optimization data, and the specific steps include:
[0174] filtering the stretching gain value to obtain a first stretching gain value;
[0175] Performing a multiplication operation on the first stretching gain value and the original data of the black-border pixel point to obtain first black-border optimized data;
[0176] The output module 5 is connected to the black edge processing module 4 , and the second black edge optimization data is output as the end point data of the image through the output module 5 .
[0177] In some other embodiments of the present invention, the specific steps of adjusting the stretching gain value by the black edge processing module 4 to adjust the output data of the black edge pixel point to obtain the black edge optimization data include:
[0178] Setting a correction control coefficient, wherein the correction control coefficient is greater than 1 and less than or equal to 3, and performing a multiplication operation on the correction control coefficient and the stretching gain value to obtain a second stretching gain value;
[0179] Performing a multiplication operation on the second stretching gain value and the original data of the black-border pixel point to obtain second black-border optimized data;
[0180] The second black edge optimization data is output as the end point data of the image through the output module 5.
[0181] The present invention also provides a device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the image processing method for black edge optimization is implemented when the processor executes the program.
[0182] The present invention also provides a storage medium on which a program is stored, and when the program is executed by a processor, the image processing method for optimizing black edges is implemented.
[0183] The advantages of the device and the storage medium of the present invention are: realizing the processing of the black-edge pixel points, avoiding the filtering of the black-edge pixel points, and avoiding the sacrifice of the overall dynamic range of the image, thereby optimizing the black edge processing while ensuring the overall dynamic range of the image, thereby ensuring the processing quality of the black edge of the image.
[0184] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.
Claims
1. A black edge optimized image processing method, characterized in that: Includes steps: Providing an original image, and obtaining original data of all pixels based on the original image; Processing the original image to obtain a first image, and acquiring first brightness data of all pixels according to the first image; Preset a brightness range, perform tone mapping on the first image to obtain a second image, and obtain second brightness data of all pixels according to the second image, so that the second brightness data of all pixels are within the brightness range; A stretch gain value is obtained according to the first brightness data and the second brightness data, and output data is obtained according to the original data and the stretch gain value, wherein the calculation formula of the stretch gain value is: gain = imageY' / (imageY+1) Wherein, gain is the stretching gain value, imageY' is the second brightness data, and imageY is the first brightness data; Determine a highlight area of the image according to the second brightness data of all the pixels, obtain a first edge intensity and a second edge intensity respectively according to the second brightness data of the pixels in the highlight area and the local edge information of the output data, and obtain a black edge pixel according to the first edge intensity and the second edge intensity; The stretching gain value is adjusted to adjust the output data of the black-border pixel point to obtain black-border optimized data, and the black-border optimized data is output as the end point data of the image.
2. The image processing method for black edge optimization according to claim 1, characterized in that: The step of obtaining output data according to the original data and the stretch gain value comprises: A multiplication operation is performed on the original data and the stretching gain value to obtain the output data, and a format of the output data is consistent with a data format of the original data.
3. The image processing method for black edge optimization according to claim 1, characterized in that: The step of determining the highlight area of the image according to the second brightness data of all the pixels comprises: Preset brightness threshold and number threshold; Divide the second image into a plurality of windows, each of which has a size of M×N, where both M and N are positive integers; Calculate the brightness of the pixel in the window, compare the brightness of the pixel with the brightness threshold, and mark the pixel as a highlighted pixel when the brightness of the pixel is greater than the brightness threshold; The number of the highlighted pixels in the window is counted, and the highlighted area is obtained according to a comparison result between the number of the highlighted pixels and the number threshold.
4. The black edge optimized image processing method according to claim 3, characterized in that: The step of obtaining the highlight area according to the comparison result of the number of the highlight pixels and the number threshold comprises: When the number of highlighted pixels in the window is greater than or equal to the number threshold, the window is determined to be a highlighted area.
5. The black edge optimized image processing method according to claim 3, characterized in that: The step of obtaining the highlight area according to the comparison result of the number of the highlight pixels and the number threshold comprises: When the number of highlighted pixels in the window is less than the number threshold, the window is determined to be a non-highlighted area, and the output data of all pixels in the non-highlighted area are output as the end point data of the image.
6. The image processing method for black edge optimization according to claim 5, characterized in that: The steps of respectively acquiring a first edge strength and a second edge strength according to the first brightness data of the pixel points in the highlight area and the local edge information of the output data include: Calculating the edge strength in the horizontal direction of the second brightness data to obtain a first horizontal edge strength, and calculating the edge strength in the vertical direction of the first brightness data to obtain a first vertical edge strength; The edge strength in the horizontal direction of the output data is calculated to obtain a second horizontal edge strength, and the edge strength in the vertical direction of the output data is calculated to obtain a second vertical edge strength.
7. The image processing method for black edge optimization according to claim 6, characterized in that: The step of obtaining black edge pixel points according to the first edge strength and the second edge strength includes: A multiplication operation is performed on the first horizontal edge strength and the second horizontal edge strength to obtain horizontal edge information, a multiplication operation is performed on the first vertical edge strength and the second vertical edge strength to obtain vertical edge information, and a black edge pixel point is determined according to the horizontal edge information and the vertical edge information.
8. The black edge optimized image processing method according to claim 7, characterized in that: The step of determining black edge pixel points according to the horizontal edge information and the vertical edge information comprises: When any one of the horizontal edge information and the vertical edge information is less than 0, it is determined that the pixel point corresponding to the horizontal edge information and the vertical edge information is a black edge pixel point.
9. The image processing method for black edge optimization according to claim 7, characterized in that: The step of determining black edge pixel points according to the horizontal edge information and the vertical edge information comprises: When the horizontal edge information and the vertical edge information are both greater than or equal to 0, the pixel points corresponding to the horizontal edge information are regarded as normal pixel points, and the output data of all the pixel points in the non-highlight area are output as the end point data of the image.
10. The image processing method for black edge optimization according to claim 1, characterized in that: The black edge optimization data includes first black edge optimization data, and the step of adjusting the stretch gain value to adjust the output data of the black edge pixel point to obtain the black edge optimization data includes: filtering the stretching gain value to obtain a first stretching gain value; A multiplication operation is performed on the first stretching gain value and the original data of the black-border pixel point to obtain first black-border optimized data.
11. The black edge optimized image processing method according to claim 1, characterized in that: The black edge optimization data includes second black edge optimization data, and the step of adjusting the stretch gain value to adjust the output data of the black edge pixel point to obtain the black edge optimization data includes: Setting a correction control coefficient, and performing a multiplication operation on the correction control coefficient and the stretching gain value to obtain a second stretching gain value; A multiplication operation is performed on the second stretching gain value and the original data of the black-border pixel point to obtain second black-border optimized data.
12. A black edge optimized image processing device, used to implement the black edge optimized image processing method according to any one of claims 1 to 11, characterized in that: include: A data processing module, used to provide an original image, obtain original data of all pixels respectively according to the original image, process the original image to obtain a first image, obtain first brightness data of all pixels according to the first image, preset a brightness interval, perform tone mapping on the first image to obtain a second image, obtain second brightness data of all pixels according to the second image, so that the second brightness data of all pixels are within the brightness interval, obtain a stretching gain value according to the first brightness data and the second brightness data of each pixel, and obtain output data according to the original data and the stretching gain value; A highlight detection module, used for determining a highlight area of an image according to the second brightness data of all pixels; A black edge pixel acquisition module, used for acquiring black edge pixel points according to the second brightness data of the pixel points in the highlight area and the output data; A black edge processing module, used for adjusting the stretching gain value to adjust the output data of the black edge pixel points to obtain black edge optimization data; The output module is used to output the black edge optimization data as the endpoint data of the image.
13. A device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the image processing method for black edge optimization as described in any one of claims 1 to 11 is implemented.
14. A storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the image processing method for black edge optimization as described in any one of claims 1 to 11 is implemented.
