Image brightness adjusting method and device

Abstract

The invention discloses an image brightness adjusting method and device. The method comprises the following steps: determining image overlapping areas among a plurality of to-be-adjusted images and overlapping pixel sets in the image overlapping areas; at least two to-be-adjusted images and overlapping pixel sets corresponding to the two to-be-adjusted images in the image overlapping area are included in one image overlapping area; determining a brightness mean value corresponding to each overlapped pixel set according to the brightness value of each pixel in the overlapped pixel set; according to the brightness mean value corresponding to the same image overlapping area, calculating a brightness balance relation between reference brightness coefficients corresponding to each overlapping pixel set in the same image overlapping area through a brightness balance equation; calculating a global brightness residual error according to a brightness balance relation in all image overlapping areas, and determining a brightness adjustment coefficient of each overlapping pixel set according to the global brightness residual error; and adjusting the brightness value of each pixel in the corresponding to-be-adjusted image according to the brightness adjustment coefficient.

Description

Technical Field

[0001] This application relates to the field of data processing, and particularly to an image brightness adjustment method and device. Background Art

[0002] With the development of intelligent assisted driving technology, the surround view system has become one of the common driving assistance systems for intelligent vehicles. The images displayed by the surround view system are usually composed of images captured by four cameras with different installation positions and installation heights on the vehicle body. Due to problems such as camera parameters, positions, and lighting differences, the brightness of the images captured by each camera is often inconsistent. When splicing images, it is first necessary to adjust the brightness of the images captured by the four cameras.

[0003] When splicing images, the overlapping areas of the image field of view usually also have overlapping of the spliced pictures. In the related art, taking the brightness of one picture in the overlapping pictures as the standard, the brightness of the other picture is adjusted. However, this method does not consider the overall brightness balance of the four pictures, and is prone to cause overall brightness imbalance. Summary of the Invention

[0004] The embodiments of this application provide an image brightness adjustment method and device, which can improve the brightness balance of the spliced images.

[0005] In view of this, the first aspect of this application provides an image brightness adjustment method, and the method includes:

[0006] Determine the image overlapping areas between multiple images to be adjusted and the overlapping pixel sets within each of the image overlapping areas; wherein, within one of the image overlapping areas, there are at least two of the images to be adjusted and the overlapping pixel sets respectively corresponding to the two images to be adjusted within the image overlapping area;

[0007] According to the brightness values of the pixels in the overlapping pixel sets, determine the brightness mean values respectively corresponding to each of the overlapping pixel sets;

[0008] According to the brightness mean values corresponding to the same image overlapping area, calculate the brightness balance relationship between the reference brightness coefficients corresponding to each of the overlapping pixel sets within the same image overlapping area through a brightness balance equation;

[0009] According to the brightness balance relationship in all the image overlapping areas, calculate the global brightness residual, and determine the brightness adjustment coefficient for each of the overlapping pixel sets according to the global brightness residual;

[0010] According to the brightness adjustment coefficient, adjust the brightness values of the pixels in the image to be adjusted corresponding to the brightness adjustment coefficient.

[0011] Optionally, determining the brightness adjustment coefficients of each overlapping pixel set according to the global brightness residual includes:

[0012] Determining the respective reference brightness coefficients corresponding to the minimum of the global brightness residual as the brightness adjustment coefficients of each overlapping pixel set.

[0013] Optionally, the method further includes:

[0014] Calculating a stereo object detection coefficient corresponding to the overlapping pixel set according to the brightness values of the pixels in the overlapping pixel set;

[0015] If the stereo object detection coefficient is less than a preset detection threshold, determining the brightness adjustment coefficient corresponding to the overlapping pixel set as a default brightness value.

[0016] Optionally, the method further includes:

[0017] Obtaining the first m frames of images corresponding to the image to be adjusted in the time series, and using the first m frames of images as the reference adjustment images corresponding to the image to be adjusted; wherein, the image field of view range corresponding to the reference adjustment image is the same as the image field of view range corresponding to the image to be adjusted, the first m frames of images are continuous in the time series, and the last frame of the first m frames of images is adjacent to the image to be adjusted in the time series; m is an integer greater than 1;

[0018] Adjusting the brightness adjustment coefficients corresponding to each overlapping pixel set in the image to be adjusted according to the brightness adjustment coefficients corresponding to each overlapping pixel set in the m reference adjustment images, and adjusting the brightness values of the pixels in the overlapping pixel set using the adjusted brightness adjustment coefficients.

[0019] Optionally, adjusting the brightness values of the pixels in the image to be adjusted corresponding to the brightness adjustment coefficient includes:

[0020] Determining the image to be adjusted corresponding to the brightness adjustment coefficient according to the overlapping pixel set corresponding to the brightness adjustment coefficient and the image to be adjusted corresponding to the overlapping pixel set;

[0021] When the image to be adjusted corresponds to one brightness adjustment coefficient, adjusting the brightness values of the pixels in the image to be adjusted using the brightness adjustment coefficient;

[0022] When the image to be adjusted corresponds to multiple brightness adjustment coefficients, determining a brightness smoothing adjustment coefficient of the image to be adjusted according to the multiple brightness adjustment coefficients, and adjusting the brightness values of the pixels in the image to be adjusted according to the brightness adjustment coefficient and the brightness smoothing adjustment coefficient.

[0023] Optionally, the method further includes:

[0024] Detect the image format of the image to be adjusted. If the image format is different from the standard image format, convert the image format of the image to be adjusted to the standard image format.

[0025] Optionally, the standard image format is the YUV image format.

[0026] A second aspect of the present application provides an image brightness adjustment device, the device includes:

[0027] A first determination unit, configured to: determine the image overlapping region between multiple images to be adjusted and the overlapping pixel sets within each of the image overlapping regions; wherein, within one of the image overlapping regions, at least two of the images to be adjusted and the overlapping pixel sets respectively corresponding to the two images to be adjusted within the image overlapping region are included;

[0028] A second determination unit, configured to: determine the brightness mean value corresponding to each of the overlapping pixel sets according to the brightness values of the pixels in the overlapping pixel sets;

[0029] A first calculation unit, configured to: calculate the brightness balance relationship between the reference brightness coefficients corresponding to the overlapping pixel sets within the same image overlapping region through a brightness balance equation according to the brightness mean values corresponding to the same image overlapping region;

[0030] A second calculation unit, configured to: calculate the global brightness residual according to the brightness balance relationships in all the image overlapping regions, and determine the brightness adjustment coefficient for each overlapping pixel set according to the global brightness residual;

[0031] A brightness adjustment unit, configured to: adjust the brightness values of the pixels in the image to be adjusted corresponding to the brightness adjustment coefficient according to the brightness adjustment coefficient.

[0032] Optionally, the second calculation unit is specifically configured to:

[0033] Determine the reference brightness coefficients corresponding to the minimum global brightness residual as the brightness adjustment coefficients for each overlapping pixel set.

[0034] Optionally, the device further includes a third calculation unit, configured to:

[0035] Calculate the three-dimensional object detection coefficient corresponding to the overlapping pixel set according to the brightness values of the pixels in the overlapping pixel set;

[0036] If the three-dimensional object detection coefficient is less than a preset detection threshold, determine the brightness adjustment coefficient corresponding to the overlapping pixel set as the default brightness value.

[0037] As can be seen from the above technical solutions, the present application has the following advantages:

[0038] The present application provides an image brightness adjustment method. By determining the image overlapping regions and the corresponding overlapping pixel sets among multiple images to be adjusted, the overlapping pixel set is a set of pixels in the images to be adjusted that overlap with other images in the image overlapping regions. Each image overlapping region includes at least two images to be adjusted and their respective corresponding overlapping pixel sets in the image overlapping region. According to the brightness values of the individual pixels recorded in the overlapping pixel sets, the respective brightness means corresponding to the overlapping pixel sets are determined. By using the brightness balance equation, the brightness balance relationship among the reference brightness coefficients corresponding to the overlapping pixel sets in the same image overlapping region is calculated. Through this brightness balance relationship, the brightness values of all pixels corresponding to the same image overlapping region can be adjusted to be consistent. According to the brightness balance relationship in all image overlapping regions, the global brightness residual is calculated. Through the global brightness residual, the brightness differences in all image overlapping regions can be determined. According to the global brightness residual, the brightness adjustment coefficients of the overlapping pixel sets are determined, and then the brightness values of the individual pixels in the corresponding images to be adjusted are adjusted according to the brightness adjustment coefficients corresponding to the overlapping pixel sets. In the image brightness adjustment method provided by the present application, not only the brightness balance within a single image overlapping region is considered, but also the global brightness balance corresponding to all image overlapping regions is considered, improving the balance and consistency of the brightness adjustment for multiple images to be adjusted and enhancing the effect of image brightness adjustment. Description of the Drawings

[0039] Figure 1 It is a flowchart of an image brightness adjustment method provided by an embodiment of the present application;

[0040] Figure 2 It is a schematic diagram of image overlap provided by an embodiment of the present application;

[0041] Figure 3 It is a distribution schematic diagram of the product of the brightness mean and the reference brightness coefficient corresponding to the image overlapping region provided by an embodiment of the present application;

[0042] Figure 4 It is a flowchart of an image brightness adjustment method provided by an embodiment of the scenario of the present application;

[0043] Figure 5 It is a schematic structural diagram of an image brightness adjustment device provided by an embodiment of the present application;

[0044] Figure 6 It is a schematic structural diagram of an image brightness adjustment device provided by an embodiment of the scenario of the present application. Detailed Embodiments

[0045] Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not used to limit the protection scope of the present application.

[0046] It should be noted that the modifiers "one" and "multiple" mentioned in the present application are illustrative rather than restrictive. Those skilled in the art should understand that unless clearly specified otherwise in the context, it should be understood as "one or more".

[0047] See Figure 1 , Figure 1 which is a flowchart of a method for adjusting image brightness provided by an embodiment of the present application. The method specifically includes the following steps:

[0048] Step 101: Determine the image overlapping regions between multiple images to be adjusted and the overlapping pixel sets within each of the image overlapping regions.

[0049] In a single image to be adjusted, the image overlapping region is the region where the image overlaps with other images. Each pixel in this image overlapping region is stored in the overlapping pixel set. Among them, within a single image overlapping region, there are at least two images to be adjusted and the overlapping pixel sets corresponding to each of the two images to be adjusted within the image overlapping region. When an image to be adjusted overlaps with other images, there may be one image overlapping region or multiple image overlapping regions in this image to be adjusted. For this image to be adjusted, the pixels in different image overlapping regions are stored in different overlapping pixel sets.

[0050] See Figure 2 , Figure 2 which is a schematic diagram of image overlapping provided by an embodiment of the present application. The transparent chassis captures images through cameras installed around the vehicle body, which serve as the image input for the vehicle trajectory prediction algorithm. Through algorithm calculation, the image under the vehicle is filled in real time to achieve the effect of the transparent chassis. Therefore, the brightness balance of the four cameras will affect the display effect of the transparent chassis. In a vehicle with a surround view function, the cameras installed in the front, rear, left, and right of the vehicle body are respectively used to collect images in the front, rear, left, and right directions. The images in these four directions are all images to be adjusted. In order to obtain a complete surround view image, the collected images need to be stitched. In Figure 2When splicing the images in four directions, a total of four overlapping regions are generated, namely overlapping region 1, overlapping region 2, overlapping region 3, and overlapping region 4. Among them, overlapping region 1 is the image overlapping region generated by the forward image and the left image, overlapping region 2 is the image overlapping region generated by the forward image and the right image, overlapping region 3 is the image overlapping region generated by the backward image and the left image, and overlapping region 4 is the image overlapping region generated by the right image and the backward image. In each image overlapping region, the viewing ranges corresponding to different images are the same. That is to say, the viewing ranges of the left image and the forward image in overlapping region 1 are the same.

[0051] Further, the image brightness adjustment method provided by the embodiments of the present application may further include the following steps:

[0052] Detect the image format of the image to be adjusted. If the image format is different from the standard image format, perform image format conversion on the image to be adjusted to convert the image format of the image to be adjusted into the standard image format.

[0053] When obtaining the image to be adjusted, detect the image format of the image to be adjusted. If the image format of the image to be adjusted is the standard image format, then the subsequent brightness adjustment steps can be directly performed on the image to be adjusted; if the image format of the image to be adjusted is different from the standard image format, then the image to be adjusted needs to be subjected to image format conversion to convert the image format of the image to be adjusted into the standard image format.

[0054] Specifically, the standard image format is the YUV image format.

[0055] If the image to be adjusted is an RGB image, then the RGB image can be converted into a YUV image, and the conversion formula is as follows:

[0056] Y = 0.299 * R + 0.587 * G + 0.114 * B

[0057] U = -0.169 * R - 0.331 * G + 0.500 * B + 128

[0058] V = 0.500 * B - 0.419 * G - 0.081 * B + 128

[0059] In the RGB image, R is the pixel data of each pixel in the red color channel, G is the pixel data of each pixel in the green color channel, and B is the pixel data of each pixel in the blue color channel; in the YUV image, Y is the brightness data, and U and V represent chrominance, which are used to describe the image color and saturation and specify the color of the pixel.

[0060] An image format adjustment step is added to uniformly adjust the images to be adjusted in different image formats to a standard image format, thereby improving the compatibility of image formats and unifying the data formats corresponding to the images to be adjusted. When processing multiple images to be adjusted, data in the same data format are used, thereby improving the efficiency of image processing.

[0061] Step 102: Determine the brightness mean values ​​corresponding to each of the overlapping pixel sets according to the brightness values ​​of each pixel in the overlapping pixel sets.

[0062] In the embodiment of the present application, an overlapping pixel set corresponds to an image to be adjusted and an image overlapping region in the image to be adjusted. In the overlapping pixel set, each pixel of the image to be adjusted in the image overlapping region is stored. Figure 2 The overlapping area 1 shown includes overlapping pixel sets corresponding to the forward image and the left image respectively. Taking the forward image as an example, in the overlapping pixel set corresponding to the forward image in the overlapping area 1, all pixels of the forward image in the overlapping area 1 are stored.

[0063] The brightness value of each pixel in the overlapping pixel set is obtained, and the brightness mean value between all pixels in the overlapping pixel set can be calculated according to the brightness value of each pixel, and the brightness mean value is used as the brightness mean value corresponding to the overlapping pixel set.

[0064] exist Figure 2 In the panoramic image shown, each image to be adjusted corresponds to two image overlapping areas, forming a total of eight overlapping pixel sets y0[m]~y7[m]. Wherein, m is the number of pixels in an image overlapping area, y0[m] is the overlapping pixel set corresponding to the forward image in the overlapping area 1, y1[m] is the overlapping pixel set corresponding to the left image in the overlapping area 1, y2[m] is the overlapping pixel set corresponding to the forward image in the overlapping area 2, y3[m] is the overlapping pixel set corresponding to the right image in the overlapping area 2, y4[m] is the overlapping pixel set corresponding to the backward image in the overlapping area 3, y5[m] is the overlapping pixel set corresponding to the left image in the overlapping area 3, y6[m] is the overlapping pixel set corresponding to the backward image in the overlapping area 4, and y7[m] is the overlapping pixel set corresponding to the right image in the overlapping area 4; correspondingly, according to the brightness values ​​of the pixels stored in y0[m]~y7[m], the brightness mean values ​​l0~l7 corresponding to each overlapping pixel set are calculated respectively.

[0065] Step 103: Calculate the brightness balance relationship between the reference brightness coefficients corresponding to each overlapping pixel set in the overlapping area of ​​the same image through a brightness balance equation according to the brightness mean value of the overlapping area of ​​the same image.

[0066] After calculating the average brightness corresponding to each image overlapping region, first process the set of overlapping pixels in each image overlapping region. Based on the correspondence between the set of overlapping pixels and the average brightness, and the correspondence between the set of overlapping pixels and the image overlapping region, determine the correspondence between the average brightness and the image overlapping region. For example, the correspondence between the average brightness l0, l1 and the overlapping region 1 in the above text.

[0067] Input the average brightness corresponding to the same image overlapping region into the brightness balance equation, and calculate the brightness balance relationship between each reference brightness coefficient in this image overlapping region. That is to say, the brightness balance relationship is the relative numerical relationship between the reference brightness coefficients corresponding to the same image overlapping region. In the brightness balance equation, for each set of overlapping pixels corresponding to the same image overlapping region, set the product of its corresponding average brightness and its corresponding reference brightness coefficient to the same value. Specifically, taking Figure 2 the four images to be adjusted and their corresponding image overlapping regions shown as an example, the brightness balance equation is shown as follows:

[0068] l i k i -l i+1 k i+1 =0(i=0,2,4,6)

[0069] Wherein, k i is the reference brightness coefficient corresponding to each set of overlapping pixels. Refer to Figure 3 , Figure 3 which is the distribution schematic diagram of the product of the average brightness and the reference brightness coefficient corresponding to the image overlapping region provided by the embodiment of the present application. Since y0 and y1, y2 and y3, y4 and y5, y6 and y7 respectively correspond to the same image overlapping region, in the brightness balance equation, balance the products of the average brightness and the reference brightness coefficient between y0 and y1, y2 and y3, y4 and y5, y6 and y7 respectively, to obtain the relative numerical relationship between the reference brightness coefficients corresponding to the same image overlapping region, that is, the brightness balance relationship. In the brightness balance equation, when the average brightness corresponding to the set of overlapping pixels is larger, the corresponding reference brightness coefficient is smaller; conversely, when the average brightness corresponding to the set of overlapping pixels is smaller, the corresponding reference brightness coefficient is larger; thus, the brightness values of the set of overlapping pixels belonging to the same image overlapping region can be balanced through the reference brightness coefficient.

[0070] In a possible implementation manner, the image brightness adjustment method provided by the embodiment of the present application may further include the following steps:

[0071] Step 11: Calculate the stereo object detection coefficient corresponding to the set of overlapping pixels according to the brightness values of each pixel in the set of overlapping pixels.

[0072] Before calculating the brightness balance relationship between the reference brightness coefficients corresponding to each overlapping pixel set, the stereo object detection coefficient corresponding to the overlapping pixel set can be calculated one by one according to the brightness values of the pixels in the overlapping pixel set. The calculation formula corresponding to the stereo object detection coefficient is shown as follows:

[0073]

[0074] where Y cof is the stereo object detection coefficient, m is the number of pixels in the overlapping pixel set, and y ai , y bi are overlapping pixel sets corresponding to the same image overlapping region. For example Figure 2 the overlapping pixel sets y0 and y1 that belong to overlapping region 1 in

[0075] Step 12: If the stereo object detection coefficient is less than the preset detection threshold, determine the brightness adjustment coefficient corresponding to the overlapping pixel set as the default brightness value.

[0076] Calculate the stereo object detection coefficients corresponding to each image overlapping region one by one. If the stereo object detection coefficient is less than the preset detection threshold, it indicates that there is an obstacle in the visual field corresponding to the image overlapping region, and the obstacle significantly blocks the visual field of the image to be adjusted that contains the image overlapping region. At this time, the image to be adjusted is distorted, and the brightness value corresponding to the image overlapping region cannot truly describe the true brightness value in this visual field. Therefore, it is meaningless to adjust the brightness of the pixels in each overlapping pixel set in this image overlapping region, and the brightness adjustment coefficients of each overlapping pixel set corresponding to this image overlapping region can be directly set as the default brightness value. For example, the brightness adjustment coefficient is determined to be 1, which saves the calculation process and calculation resources for subsequent calculation of each overlapping pixel set in this image overlapping region and improves the efficiency of image brightness adjustment.

[0077] If the stereo object detection coefficient is greater than or equal to the preset detection threshold, it indicates that there is no obstacle in the image overlapping region, or the obstacle does not significantly block the visual field of the image to be adjusted, and the brightness balance in this image overlapping region can be improved through brightness adjustment.

[0078] Step 104: Calculate the global brightness residual according to the brightness balance relationship in all image overlapping regions, and determine the brightness adjustment coefficient of each overlapping pixel set according to the global brightness residual.

[0079] After obtaining the respective corresponding brightness balance relationships in all image overlapping regions, use the brightness balance relationships in all image overlapping regions to calculate the global brightness residuals corresponding to all image overlapping regions. The brightness differences between multiple image overlapping regions can be determined through the global brightness residuals, so as to Figure 2 Taking the four images to be adjusted and their corresponding image overlapping regions shown in Figure 2 as an example, the calculation formula for the global brightness residual is shown as follows:

[0080]

[0081] where F is the global brightness residual, the value of n is 7, both N and G are coefficients of the residual change rate equation. When the ratio of N to G is greater than 400, a better brightness balance effect can be obtained.

[0082] Substitute the reference brightness coefficients and brightness means corresponding to each overlapping pixel set into the above calculation formula for the global brightness residual to obtain the corresponding relationship between the global brightness residual and the reference brightness coefficient; when the obtained global brightness residuals are different, the corresponding reference brightness coefficients of each overlapping pixel set are also different. The value of the reference brightness coefficient corresponding to each overlapping pixel set can be determined according to the value of the global brightness residual, and the value of the reference brightness coefficient at this time is determined as the brightness adjustment coefficient corresponding to each overlapping pixel set.

[0083] Specifically, when determining the brightness adjustment coefficients of each overlapping pixel set according to the global brightness residual, the reference brightness coefficients corresponding to the minimum global brightness residual can be determined as the brightness adjustment coefficients of each overlapping pixel set. In this way, the brightness differences in each image overlapping region can be adjusted to a smaller value, improving the brightness balance of each image overlapping region.

[0084] Step 105: Adjust the brightness values of each pixel in the image to be adjusted corresponding to the brightness adjustment coefficient.

[0085] After determining the brightness adjustment coefficients corresponding to each overlapping pixel set, according to the corresponding relationship between the overlapping pixel set and the image to be adjusted, determine the brightness adjustment coefficients corresponding to each image to be adjusted, and adjust the brightness values of each pixel in the image to be adjusted according to the corresponding brightness adjustment coefficient.

[0086] Since in the image to be adjusted, there may be multiple overlapping pixel sets, then for an image to be adjusted, there may be multiple corresponding brightness adjustment coefficients. When adjusting the image to be adjusted according to multiple brightness adjustment coefficients, the average value of multiple brightness adjustment coefficients can be taken, or other pixels in the image to be adjusted that do not belong to the image overlapping region can be adjusted for brightness smoothing with reference to multiple brightness adjustment coefficients.

[0087] In a possible implementation, step 105 can be implemented in the following manner:

[0088] Step 21: Determine the image to be adjusted corresponding to the brightness adjustment coefficient according to the set of overlapping pixels corresponding to the brightness adjustment coefficient and the image to be adjusted corresponding to the set of overlapping pixels.

[0089] According to the correspondence between the brightness adjustment coefficient and the set of overlapping pixels, and the correspondence between the set of overlapping pixels and the image to be adjusted, determine the correspondence between the brightness adjustment coefficient and the image to be adjusted, so as to determine the image to be adjusted corresponding to each brightness adjustment coefficient.

[0090] Step 22: When the image to be adjusted corresponds to one brightness adjustment coefficient, use the brightness adjustment coefficient to adjust the brightness values of each pixel in the image to be adjusted.

[0091] When the image to be adjusted corresponds to only one brightness adjustment coefficient and only includes one image overlapping area in the image to be adjusted, the brightness values of each pixel in the image to be adjusted can be directly adjusted using this brightness adjustment coefficient.

[0092] Step 23: When the image to be adjusted corresponds to multiple brightness adjustment coefficients, determine the brightness smoothing adjustment coefficient of the image to be adjusted according to the multiple brightness adjustment coefficients, and adjust the brightness values of each pixel in the image to be adjusted according to the brightness adjustment coefficient and the brightness smoothing adjustment coefficient.

[0093] When the image to be adjusted corresponds to multiple brightness adjustment coefficients, there are multiple image overlapping areas in the image to be adjusted and multiple sets of overlapping pixels are correspondingly included. Then, in order to take into account the brightness adjustment coefficients corresponding to each of the multiple sets of overlapping pixels and improve the effect of brightness adjustment, it is necessary to perform brightness smoothing on the pixels in the image to be adjusted that do not belong to the image overlapping area. Determine the brightness smoothing adjustment coefficient corresponding to the image to be adjusted according to the multiple brightness adjustment coefficients corresponding to the image to be adjusted, and adjust the brightness values of each pixel in the image to be adjusted according to the brightness smoothing adjustment coefficient and the brightness adjustment coefficient.

[0094] Taking Figure 2 the forward image shown as an example, there are two image overlapping areas in the forward image, namely overlapping area 1 and overlapping area 2. Then this forward image will correspond to two brightness adjustment coefficients. Since the two image overlapping areas in the forward image are located at both ends of the forward image, it is necessary to perform brightness smoothing on the pixels in the middle of the forward image that do not belong to the image overlapping area. Taking the brightness adjustment coefficients corresponding to the image overlapping areas at both ends as the starting value and the ending value respectively, so that the pixels in the middle of the image to be adjusted can smoothly change from the starting value to the ending value at a certain rate of change, and this rate of change is the brightness smoothing adjustment coefficient corresponding to this forward image.

[0095] In a possible implementation, the image brightness adjustment method provided by the embodiments of the present application further includes the following steps:

[0096] Step 31: Obtain the first m frames of images corresponding to the image to be adjusted in the time series, and use the first m frames of images as the reference adjustment images corresponding to the image to be adjusted.

[0097] In a vehicle configured with a surround view function, the continuous real-time images around the vehicle body are displayed on the vehicle center control display screen. Therefore, when adjusting the brightness of each frame of image in the continuous real-time images, the brightness balance between the continuous images also needs to be considered. When adjusting the brightness of multiple images to be adjusted at the current moment, obtain the continuous m frames of images before the current moment corresponding to the images to be adjusted, and use these images as the reference adjustment images for adjusting the brightness of the images to be adjusted at the current moment. Among them, the image field of view range corresponding to the reference adjustment images is the same as the image field of view range corresponding to the images to be adjusted, the first m frames of images are continuous in the time series, and the last frame of the first m frames of images is adjacent to the image to be adjusted in the time series. That is to say, the first m frames of images include the image of the previous frame before the current moment; m is an integer greater than 1.

[0098] For example, when adjusting the brightness of four images to be adjusted collected by the vehicle at the p-th moment, it is necessary to obtain the four-way images corresponding to each moment from the (p - m)-th moment to the (p - 1)-th moment, that is, 4m reference adjustment images.

[0099] Step 32: According to the brightness adjustment coefficients corresponding to each overlapping pixel set in the m reference adjustment images, adjust the brightness adjustment coefficients corresponding to each overlapping pixel set in the image to be adjusted, and use the adjusted brightness adjustment coefficients to adjust the brightness values of each pixel in the overlapping pixel set.

[0100] For an image to be adjusted, after determining the m reference adjustment images corresponding to its first m frames, according to the brightness adjustment coefficients corresponding to each overlapping pixel set in the m reference adjustment images, adjust the brightness adjustment coefficients corresponding to the overlapping pixel set in the image to be adjusted, so as to avoid a sudden change in brightness between the image to be adjusted and the m reference adjustment images corresponding to its first m frames.

[0101] Taking the forward image in Figure 2 as an example, where m = 3 and the brightness of the overlapping area 1 in the forward image is adjusted. When adjusting the brightness of the forward image at the current moment, obtain the forward images of the previous 3 frames before the current moment as the reference adjustment images, and at the same time obtain the brightness adjustment coefficients corresponding to the overlapping area 1 in the forward images of the previous 3 frames, which are k last0 and k last1, k last2 , and then adjust the brightness adjustment coefficient k corresponding to the overlapping area 1 in the forward image at the current moment in a compromise ratio i . The specific calculation formula is shown as follows:

[0102]

[0103] cur_k i = scale * last_k i +(1 - scale)*k i

[0104] where, last_k i is the average value of the brightness adjustment coefficients corresponding to the overlapping area 1 in the first 3 reference adjustment images, scale is the brightness smoothing ratio coefficient, and cur_k i is the adjusted brightness adjustment coefficient. Adjusting the image to be adjusted at the current moment according to the first m frames of images improves the brightness balance between frames when the image to be adjusted is continuously displayed.

[0105] See Figure 4 , Figure 4 which is the flowchart of the image brightness adjustment method provided by the scene embodiment of this application. After obtaining the image to be adjusted, first detect whether the image to be adjusted is a YUV image. If it is, directly execute the subsequent brightness adjustment steps; if not, convert the image to be adjusted into a YUV image.

[0106] After determining the image format, determine the image overlapping areas in the multiple image overlapping areas and the overlapping pixel sets within each image overlapping area, calculate the stereo object detection coefficient according to the pixel data recorded in the overlapping pixel sets, and determine whether it is necessary to calculate the brightness adjustment coefficient for the overlapping pixel sets according to the stereo object detection coefficient; when the stereo object detection coefficient is less than the preset detection threshold, directly set the brightness adjustment coefficient corresponding to the overlapping pixel set to the default brightness value 1; when the stereo object detection coefficient is greater than or equal to the preset detection threshold, determine the brightness adjustment coefficient corresponding to each overlapping pixel set through the brightness balance equation and the global brightness residual.

[0107] After determining the brightness adjustment coefficient corresponding to each overlapping pixel set, perform brightness smoothing processing on the images of adjacent time frames, calculate the average value last_k of the brightness adjustment coefficients of the first 3 consecutive reference adjustment images before the image to be adjusted corresponding to the overlapping pixel set, and use this average value last_k and the brightness smoothing ratio coefficient to perform proportional adjustment on the brightness adjustment coefficient corresponding to the overlapping pixel set to obtain the adjusted proportional adjustment coefficient cur_k.

[0108] When stitching and fusing multiple images to be adjusted, the multiple images to be adjusted are stitched and fused according to the brightness adjustment coefficients of each overlapping pixel set in the images to be adjusted, and a stitched image is obtained and displayed on the in-vehicle center control display screen.

[0109] Through an image brightness adjustment method provided by this application, an image overlapping region and corresponding overlapping pixel sets between multiple images to be adjusted are determined. The overlapping pixel set is a set of pixels in the images to be adjusted that overlap with other images in the image overlapping region. In one image overlapping region, at least two images to be adjusted and their respective corresponding overlapping pixel sets in the image overlapping region are included. According to the brightness values of each pixel recorded in the overlapping pixel set, the respective corresponding brightness means of each overlapping pixel set are determined. The brightness balance relationship between the reference brightness coefficients corresponding to each overlapping pixel set in the same image overlapping region is calculated through a brightness balance equation. Through this brightness balance relationship, the brightness values of all pixels corresponding to the same image overlapping region can be adjusted to be consistent. The global brightness residual is calculated according to this brightness balance relationship in all image overlapping regions. Through the global brightness residual, the brightness difference in all image overlapping regions can be determined. The brightness adjustment coefficient of each overlapping pixel set is determined according to the global brightness residual, and then the brightness values of each pixel in the corresponding image to be adjusted are adjusted according to the brightness adjustment coefficient corresponding to each overlapping pixel set. In the image brightness adjustment method provided by this application, not only the brightness balance within a single image overlapping region is considered, but also the global brightness balance corresponding to all image overlapping regions is considered, improving the balance and consistency of brightness adjustment for multiple images to be adjusted and enhancing the effect of image brightness adjustment.

[0110] See Figure 5 , Figure 5 which is a schematic structural diagram of an image brightness adjustment device provided by an embodiment of this application. The device includes: a first determination unit 501, a second determination unit 502, a first calculation unit 503, a second calculation unit 504, and a brightness adjustment unit 505.

[0111] The first determination unit 501 is configured to: determine the image overlapping region between multiple images to be adjusted and the overlapping pixel sets within each of the image overlapping regions; wherein, in one of the image overlapping regions, at least two of the images to be adjusted and their respective corresponding overlapping pixel sets in the image overlapping region are included;

[0112] The second determination unit 502 is configured to: determine the respective corresponding brightness means of each of the overlapping pixel sets according to the brightness values of each pixel in the overlapping pixel set;

[0113] The first calculation unit 503 is configured to: calculate the luminance balance relationship between the reference luminance coefficients corresponding to each overlapping pixel set within the overlapping area of the same image through a luminance balance equation based on the average luminance of the overlapping area of the corresponding same image;

[0114] The second calculation unit 504 is configured to: calculate the global luminance residual according to the luminance balance relationship in all image overlapping areas, and determine the luminance adjustment coefficient for each overlapping pixel set based on the global luminance residual;

[0115] The luminance adjustment unit 505 is configured to: adjust the luminance value of each pixel in the image to be adjusted corresponding to the luminance adjustment coefficient according to the luminance adjustment coefficient.

[0116] Optionally, the second calculation unit 504 is specifically configured to:

[0117] Determine the reference luminance coefficients corresponding to the minimum global luminance residual as the luminance adjustment coefficients for each overlapping pixel set.

[0118] Optionally, the device further includes a third calculation unit, configured to:

[0119] Calculate the three-dimensional object detection coefficient corresponding to the overlapping pixel set according to the luminance values of each pixel in the overlapping pixel set;

[0120] If the three-dimensional object detection coefficient is less than a preset detection threshold, determine the luminance adjustment coefficient corresponding to the overlapping pixel set as the default luminance value.

[0121] Optionally, the device further includes a luminance smoothing unit, configured to:

[0122] Obtain the first m frames of images corresponding to the image to be adjusted in the time series, and use the first m frames of images as the reference adjustment images corresponding to the image to be adjusted; wherein, the image field of view range corresponding to the reference adjustment image is the same as the image field of view range corresponding to the image to be adjusted, the first m frames of images are continuous in the time series, and the last frame of the first m frames of images is adjacent to the image to be adjusted in the time series; m is an integer greater than 1;

[0123] Adjust the luminance adjustment coefficients corresponding to each overlapping pixel set in the image to be adjusted according to the luminance adjustment coefficients corresponding to each overlapping pixel set in the m reference adjustment images, and adjust the luminance value of each pixel within the overlapping pixel set using the adjusted luminance adjustment coefficients.

[0124] Optionally, the luminance adjustment unit 505 is specifically configured to:

[0125] Determine the image to be adjusted corresponding to the brightness adjustment coefficient according to the set of overlapping pixels corresponding to the brightness adjustment coefficient and the image to be adjusted corresponding to the set of overlapping pixels;

[0126] When the image to be adjusted corresponds to one brightness adjustment coefficient, use the brightness adjustment coefficient to adjust the brightness values of each pixel in the image to be adjusted;

[0127] When the image to be adjusted corresponds to multiple brightness adjustment coefficients, determine the brightness smoothing adjustment coefficient of the image to be adjusted according to the multiple brightness adjustment coefficients, and adjust the brightness values of each pixel in the image to be adjusted according to the brightness adjustment coefficient and the brightness smoothing adjustment coefficient.

[0128] Optionally, the device further includes an image format conversion unit for:

[0129] Detect the image format of the image to be adjusted. If the image format is different from the standard image format, perform image format conversion on the image to be adjusted to convert the image format of the image to be adjusted to the standard image format.

[0130] Optionally, the standard image format is the YUV image format.

[0131] See Figure 6 , Figure 6 which is a schematic structural diagram of an image brightness adjustment device provided by an embodiment of the application scenario. The device includes: an image acquisition module, a brightness balance algorithm module, a brightness smoothing module, and a panoramic stitching and fusion module. Among them, the image acquisition module is used to acquire multiple images to be adjusted, the brightness balance algorithm module is used to calculate the brightness adjustment coefficients of the set of overlapping pixels corresponding to each image overlapping area in the image to be adjusted, the brightness smoothing module is used to adjust the brightness balance of the images to be adjusted in the same field of view in a continuous time series to avoid brightness jumps in the images to be adjusted in the continuous time series, and the panoramic stitching and fusion module is used to stitch the images to be adjusted after brightness balance and output the stitched image to the display screen of the vehicle central control for display.

[0132] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0133] Although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous.

[0134] It should be understood that the various steps described in the method embodiments of the present application may be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this regard.

[0135] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices, and units described above may refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0136] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image brightness adjustment method, characterized in that The method includes: Determining an image overlapping region between multiple images to be adjusted and an overlapping pixel set within each of the image overlapping regions; wherein, within one of the image overlapping regions, there are at least two of the images to be adjusted and the overlapping pixel sets respectively corresponding to the two images to be adjusted within the image overlapping region; Determining, according to the brightness values of the pixels in each of the overlapping pixel sets, the brightness average value respectively corresponding to each of the overlapping pixel sets; Calculating, according to the brightness average values corresponding to the same image overlapping region, the brightness balance relationship between the reference brightness coefficients corresponding to each of the overlapping pixel sets within the same image overlapping region through a brightness balance equation; Calculating a global brightness residual according to the brightness balance relationships in all the image overlapping regions, and determining the brightness adjustment coefficient for each of the overlapping pixel sets according to the global brightness residual; Adjusting the brightness value of each pixel in the image to be adjusted corresponding to the brightness adjustment coefficient according to the brightness adjustment coefficient.

2. The method according to claim 1, wherein The determining the brightness adjustment coefficient for each of the overlapping pixel sets according to the global brightness residual includes: Determining the reference brightness coefficients corresponding to the minimum global brightness residual as the brightness adjustment coefficients for each of the overlapping pixel sets.

3. The method according to claim 1, wherein The method further includes: Calculating a stereo object detection coefficient corresponding to the overlapping pixel set according to the brightness values of the pixels in the overlapping pixel set; If the stereo object detection coefficient is less than a preset detection threshold, determining the brightness adjustment coefficient corresponding to the overlapping pixel set as a default brightness value.

4. The method according to claim 1, wherein The method further includes: Obtaining the first m frames of images corresponding to the image to be adjusted in a time series, and using the first m frames of images as the reference adjustment images corresponding to the image to be adjusted; wherein, the image field of view range corresponding to the reference adjustment images is the same as the image field of view range corresponding to the image to be adjusted, the first m frames of images are continuous in the time series, and the last frame of the first m frames of images is adjacent to the image to be adjusted in the time series; m is an integer greater than 1; Adjusting the brightness adjustment coefficients respectively corresponding to each of the overlapping pixel sets in the image to be adjusted according to the brightness adjustment coefficients respectively corresponding to each of the overlapping pixel sets in the m reference adjustment images, and using the adjusted brightness adjustment coefficients to adjust the brightness value of each pixel within the overlapping pixel set.

5. The method according to claim 1, characterized in that, The adjusting the brightness value of each pixel in the image to be adjusted corresponding to the brightness adjustment coefficient includes: Determining the image to be adjusted corresponding to the brightness adjustment coefficient according to the overlapping pixel set corresponding to the brightness adjustment coefficient and the image to be adjusted corresponding to the overlapping pixel set; When the image to be adjusted corresponds to one brightness adjustment coefficient, using the brightness adjustment coefficient to adjust the brightness value of each pixel in the image to be adjusted; When the image to be adjusted corresponds to multiple brightness adjustment coefficients, determining a brightness smoothing adjustment coefficient for the image to be adjusted according to the multiple brightness adjustment coefficients, and adjusting the brightness value of each pixel in the image to be adjusted according to the brightness adjustment coefficient and the brightness smoothing adjustment coefficient.

6. The method according to claim 1, wherein The method further includes: Detect the image format of the image to be adjusted. If the image format is different from the standard image format, perform image format conversion on the image to be adjusted to convert the image format of the image to be adjusted into the standard image format.

7. The method according to claim 6, characterized in that, The standard image format is the YUV image format.

8. An image brightness adjustment device, characterized in that, The device includes: A first determination unit, configured to: determine the image overlapping regions between multiple images to be adjusted and the set of overlapping pixels within each of the image overlapping regions; wherein, within one of the image overlapping regions, there are at least two of the images to be adjusted and the corresponding sets of overlapping pixels of the two images to be adjusted within the image overlapping region; A second determination unit, configured to: determine the average brightness value corresponding to each of the sets of overlapping pixels according to the brightness values of the pixels in the set of overlapping pixels. A first calculation unit, configured to: calculate the brightness balance relationship between the reference brightness coefficients corresponding to each set of overlapping pixels within the same image overlapping region through a brightness balance equation according to the average brightness values corresponding to the same image overlapping region. A second calculation unit, configured to: calculate the global brightness residual according to the brightness balance relationships in all the image overlapping regions, and determine the brightness adjustment coefficient for each set of overlapping pixels according to the global brightness residual. A brightness adjustment unit, configured to: adjust the brightness values of each pixel in the image to be adjusted corresponding to the brightness adjustment coefficient according to the brightness adjustment coefficient.

9. The device according to claim 8, wherein, The second calculation unit is specifically configured to: Determine the brightness adjustment coefficients for each set of overlapping pixels as the reference brightness coefficients corresponding to the minimum global brightness residual.

10. The device according to claim 8, characterized in that, The device further includes a third calculation unit, configured to: Calculate the stereo object detection coefficient corresponding to the set of overlapping pixels according to the brightness values of the pixels in the set of overlapping pixels. If the stereo object detection coefficient is less than a preset detection threshold, determine the brightness adjustment coefficient corresponding to the set of overlapping pixels as the default brightness value.

Images