A surround image white balance method and device

Abstract

The present application provides a surround image white balance method and device, the method comprising: obtaining overhead images in all directions around the vehicle; calculating the three-channel average grayscale corresponding to the overhead images in all directions and the three-channel overall average grayscale corresponding to the overhead images in all directions; calculating the three-channel relative grayscale corresponding to the overhead images in all directions according to the three-channel average grayscale corresponding to the overhead images in all directions and the three-channel overall average grayscale; performing global white balance processing on the overhead images in all directions according to the three-channel relative grayscale corresponding to the overhead images in all directions, and obtaining white-balanced images corresponding to the overhead images in all directions. The present application takes into account the three-channel overall average grayscale corresponding to the overhead images in all directions when calculating the relative grayscale, so that the global white balance processing can effectively eliminate the influence of light changes on the overhead images in all directions, and the white balance effect is better and can better adapt to the complex environment of the vehicle.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a surround view image white balance method and device. Background Art

[0002] The existing vehicle-mounted surround view image white balance generally takes one of the four images corresponding to the front, back, left, and right directions of the vehicle as the target image, calculates the average grayscale of the R, G, and B channels of the four images, and adjusts the remaining three images according to the average grayscale ratio of the channels corresponding to the target image, to obtain the four images after white balance for splicing into a 360-degree surround view. This surround view image white balance method is simple to calculate, but the adjustment method is not flexible enough and cannot adapt to the complex environment of the vehicle. When encountering an environment with large brightness differences such as light or shadow, the white balance effect is poor, and the simple calculation of the ratio does not consider the overall effect of the surround view spliced ​​by the four images. Summary of the invention

[0003] In view of this, the present application provides a surround view image white balance method and device to solve the above technical problems, and the technical solution is as follows:

[0004] A surround image white balance method, comprising:

[0005] Obtain overhead images in all directions around the vehicle;

[0006] Calculate the three-channel average grayscale corresponding to the overhead images in each direction and the three-channel overall average grayscale corresponding to the overhead images in all directions, wherein the three-channel average grayscale includes the average grayscale of each channel of the corresponding overhead images, and the three-channel overall average grayscale includes the average grayscale of the same channel of the overhead images in all directions;

[0007] According to the three-channel average grayscale and the three-channel overall average grayscale corresponding to the overhead images in each direction, the three-channel relative grayscale corresponding to the overhead images in each direction is calculated, wherein the three-channel relative grayscale includes the relative grayscale of the same channel of the corresponding overhead image and the overhead image in the adjacent direction;

[0008] According to the relative grayscales of the three channels corresponding to the overhead images in various directions, global white balance processing is performed on the overhead images in various directions to obtain white balanced images corresponding to the overhead images in various directions.

[0009] Optionally, calculating the average grayscale of the three channels corresponding to the overhead view images in each direction and the overall average grayscale of the three channels corresponding to the overhead view images in all directions includes:

[0010] Calculate the effective area of ​​the overhead image in each direction. The effective area refers to the area of ​​the effective region contained in the corresponding overhead image. The effective region is located within the shooting area of ​​the camera corresponding to the corresponding overhead image.

[0011] For each direction, according to the effective area of ​​the overhead image in the direction and the three-channel grayscale value of the overhead image in the direction, the three-channel average grayscale corresponding to the overhead image in the direction is calculated to obtain the three-channel average grayscale corresponding to the overhead image in each direction;

[0012] According to the average grayscale of the three channels corresponding to the overhead images in each direction, the overall average grayscale of the three channels is calculated.

[0013] Optionally, calculate the effective area of ​​the top-down image in each direction, including:

[0014] For each direction, the overhead image in that direction is processed into a binary image, and the grayscale values ​​contained in the binary image are added, and the sum is used as the effective area of ​​the overhead image in that direction to obtain the effective area of ​​the overhead image in each direction.

[0015] Optionally, calculating the three-channel average grayscale corresponding to the overhead view image in the direction according to the effective area of ​​the overhead view image in the direction and the three-channel grayscale values ​​of the overhead view image in the direction includes:

[0016] Adding the grayscale values ​​of each r channel contained in the overhead image in the direction to obtain a first sum value, and dividing the first sum value by the effective area of ​​the overhead image in the direction to obtain the average grayscale of the r channel corresponding to the overhead image in the direction;

[0017] Adding the grayscale values ​​of the g channels contained in the overhead image in the direction to obtain a second sum value, and dividing the second sum value by the effective area of ​​the overhead image in the direction to obtain the average grayscale of the g channels corresponding to the overhead image in the direction;

[0018] The grayscale values ​​of each b channel contained in the overhead image in this direction are added together to obtain a third sum value, and the third sum value is divided by the effective area of ​​the overhead image in this direction to obtain the average grayscale of the b channel corresponding to the overhead image in this direction.

[0019] Optionally, according to the average grayscales of the three channels corresponding to the overhead images in each direction, the overall average grayscale of the three channels is calculated, including:

[0020] The average grayscales of the r channels corresponding to the overhead images in each direction are multiplied to obtain a first product value, and the first product value is raised to the fourth power to obtain the overall average grayscale of the r channel;

[0021] Multiply the average grayscales of the g channels corresponding to the overhead images in each direction to obtain a second product value, and raise the second product value to the fourth power to obtain the overall average grayscale of the g channel;

[0022] The average grayscales of the b channels corresponding to the overhead images in each direction are multiplied to obtain a third product value, and the third product value is raised to the fourth power to obtain the overall average grayscale of the b channel.

[0023] Optionally, according to the three-channel average grayscales and the three-channel overall average grayscales corresponding to the overhead view images in each direction, the three-channel relative grayscales corresponding to the overhead view images in each direction are calculated, including:

[0024] For each of the directions:

[0025] According to the average grayscale of the r channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the r channel corresponding to the overhead image in the direction, and the overall average grayscale of the r channel, the initial relative grayscale of the r channel corresponding to the overhead image in the direction is determined, and the initial relative grayscale of the r channel corresponding to the overhead image in the direction is adjusted by a preset activation function to obtain the relative grayscale of the r channel corresponding to the overhead image in the direction;

[0026] According to the average grayscale of the g channel corresponding to the top-view image in the adjacent direction of the direction, the average grayscale of the g channel corresponding to the top-view image in the direction and the overall average grayscale of the g channel, the initial relative grayscale of the g channel corresponding to the top-view image in the direction is determined, and the initial relative grayscale of the g channel corresponding to the top-view image in the direction is adjusted by an activation function to obtain the relative grayscale of the g channel corresponding to the top-view image in the direction;

[0027] According to the average grayscale of the b channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the b channel corresponding to the overhead image in the direction and the overall average grayscale of the b channel, the initial relative grayscale of the b channel corresponding to the overhead image in the direction is determined, and the initial relative grayscale of the b channel corresponding to the overhead image in the direction is adjusted by an activation function to obtain the relative grayscale of the b channel corresponding to the overhead image in the direction;

[0028] In order to obtain the relative grayscale of the r channel, the relative grayscale of the g channel and the relative grayscale of the b channel corresponding to the overhead image in each direction.

[0029] Optionally, determining the initial r channel relative grayscale corresponding to the overhead image in the direction according to the average grayscale of the r channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the r channel corresponding to the overhead image in the direction, and the overall average grayscale of the r channel includes:

[0030] Calculate the ratio of the average grayscale of the r channel corresponding to the overhead image in the adjacent direction to the average grayscale of the r channel corresponding to the overhead image in the direction as the first ratio;

[0031] Taking the square root of the first ratio, a square rooted first ratio is obtained;

[0032] Divide the overall average grayscale of the r channel by the first ratio after the square root to obtain the initial relative grayscale of the r channel corresponding to the overhead image in that direction;

[0033] Determining the initial relative grayscale of the g channel corresponding to the overhead image in the direction according to the average grayscale of the g channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the g channel corresponding to the overhead image in the direction, and the overall average grayscale of the g channel includes:

[0034] Calculate the ratio of the average grayscale of the g channel corresponding to the overhead image in the direction adjacent to the direction to the average grayscale of the g channel corresponding to the overhead image in the direction as the second ratio;

[0035] Taking the square root of the second ratio, a square rooted second ratio is obtained;

[0036] Divide the overall average grayscale of the g channel by the second ratio after the square root to obtain the initial relative grayscale of the g channel corresponding to the overhead image in that direction;

[0037] According to the average grayscale of the b channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the b channel corresponding to the overhead image in the direction, and the overall average grayscale of the b channel, the initial b channel relative grayscale corresponding to the overhead image in the direction is determined, including:

[0038] Calculate the ratio of the average grayscale of the b channel corresponding to the overhead image in the adjacent direction to the average grayscale of the b channel corresponding to the overhead image in the direction as the third ratio;

[0039] Taking the square root of the third ratio, a square rooted third ratio is obtained;

[0040] The overall average grayscale of the b channel is divided by the third ratio after the square root, and the initial b channel relative grayscale corresponding to the overhead image in that direction is obtained.

[0041] Optionally, performing global white balance processing on the overhead images in each direction according to the three-channel relative grayscales respectively corresponding to the overhead images in each direction to obtain white-balanced images respectively corresponding to the overhead images in each direction includes:

[0042] For each of the directions:

[0043] The relative grayscale of the r channel corresponding to the overhead image in the direction is multiplied by the grayscale values ​​of each r channel contained in the overhead image in the direction to obtain the adjusted grayscale values ​​of each r channel contained in the overhead image in the direction;

[0044] The relative grayscale of the g channel corresponding to the overhead image in the direction is multiplied by the grayscale values ​​of each g channel contained in the overhead image in the direction to obtain the adjusted grayscale values ​​of each g channel contained in the overhead image in the direction;

[0045] The relative grayscale of the b channel corresponding to the overhead image in the direction is multiplied by the grayscale values ​​of each b channel contained in the overhead image in the direction to obtain the adjusted grayscale values ​​of each b channel contained in the overhead image in the direction;

[0046] The white-balanced image corresponding to the overhead view image in the direction is composed of each adjusted r channel grayscale value, each adjusted g channel grayscale value, and each adjusted b channel grayscale value contained in the overhead view image in the direction;

[0047] In order to obtain the white balanced images corresponding to the overhead images in each direction.

[0048] Optionally, the overhead images in various directions include a first overhead image, a second overhead image, a third overhead image and a fourth overhead image, wherein the first overhead image refers to an overhead image in front of the vehicle, the second overhead image refers to an overhead image behind the vehicle, the third overhead image refers to an overhead image of the vehicle in a clockwise direction, and the fourth overhead image refers to an overhead image of the vehicle in a counterclockwise direction;

[0049] The effective area included in the first overhead image refers to the left front corner point and the right front corner point of the vehicle area included in the first overhead image, and the area formed by the left front corner point and the right front corner point of the first overhead image;

[0050] The effective area included in the second top-view image refers to the area formed by the left rear corner point and the right rear corner point of the vehicle area included in the second top-view image, and the left rear corner point and the right rear corner point of the second top-view image;

[0051] The effective area included in the third overhead image refers to the area consisting of the left front corner point and the left rear corner point of the vehicle area included in the third overhead image, and the left front corner point and the left rear corner point of the third overhead image;

[0052] The effective area included in the fourth bird's-eye view image refers to the right front corner point and the right rear corner point of the vehicle area included in the fourth bird's-eye view image, and the area formed by the right front corner point and the right rear corner point of the fourth bird's-eye view image.

[0053] A surround image white balance device, comprising:

[0054] A bird's-eye view image acquisition module is used to acquire bird's-eye view images in all directions around the vehicle;

[0055] The average grayscale calculation module is used to calculate the three-channel average grayscale corresponding to the overhead images in each direction and the three-channel overall average grayscale corresponding to the overhead images in all directions, wherein the three-channel average grayscale includes the average grayscale of each channel of the corresponding overhead images, and the three-channel overall average grayscale includes the average grayscale of the same channel of the overhead images in all directions;

[0056] A relative grayscale calculation module is used to calculate the three-channel relative grayscales corresponding to the overhead images in each direction according to the three-channel average grayscales and the three-channel overall average grayscales corresponding to the overhead images in each direction, wherein the three-channel relative grayscales include the relative grayscales of the same channel of the corresponding overhead image and the overhead image in the adjacent direction;

[0057] The image white balance module is used to perform global white balance processing on the overhead images in each direction according to the three-channel relative grayscales corresponding to the overhead images in each direction, so as to obtain white balanced images corresponding to the overhead images in each direction.

[0058] Through the above technical solutions, it can be known that the surround image white balance method provided by the present application first obtains the overhead images in all directions around the vehicle, then calculates the three-channel average grayscale corresponding to the overhead images in all directions and the three-channel overall average grayscale corresponding to the overhead images in all directions, then calculates the three-channel relative grayscale corresponding to the overhead images in all directions according to the three-channel average grayscale corresponding to the overhead images in all directions and the three-channel overall average grayscale, and finally performs global white balance processing on the overhead images in all directions according to the three-channel relative grayscale corresponding to the overhead images in all directions, and obtains the white-balanced images corresponding to the overhead images in all directions. When calculating the relative grayscale, the present application takes into account the three-channel overall average grayscale corresponding to the overhead images in all directions, so that the global white balance processing can effectively eliminate the influence of light changes on the overhead images in all directions, so that the white balance effect of the overhead images in all directions is better and can better adapt to the complex environment of the vehicle. In addition, the influence of light changes is eliminated, so that the overhead images in all directions can be spliced ​​into a vehicle surround image with better effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0060] Figure 1A schematic diagram of a flow chart of a surround view image white balance method provided in an embodiment of the present application;

[0061] Figure 2 Schematic diagram of top-view images in various directions provided by the embodiments of the present application;

[0062] Figure 3 A schematic diagram of the structure of a surround image white balance device provided in an embodiment of the present application;

[0063] Figure 4 This is a hardware structure block diagram of the surround view image white balance device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0065] In view of the problems existing in the above-mentioned ideas, the inventors of this case conducted in-depth research and finally proposed a surround-view image white balance method. The surround-view image white balance method provided by this application will be introduced in detail through the following embodiments.

[0066] See also Figure 1 , shows a schematic flow chart of a surround image white balance method provided in an embodiment of the present application, and the surround image white balance method may include:

[0067] Step S101: Acquire overhead images in all directions around the vehicle.

[0068] In this step, the top view images in various directions are obtained by dedistorting and inverse perspective transforming images of the same size collected by multiple cameras installed at different positions on the vehicle body. For the processing of dedistortion and inverse perspective transformation, please refer to the introduction in the prior art and will not be repeated here.

[0069] Optionally, the above-mentioned multiple cameras can be multiple fisheye cameras; in this embodiment, the resolutions of the multiple fisheye cameras are the same, and the sizes of the overhead view images obtained in all directions are the same, all of which are w*h, where w represents the length of the overhead view image, and h represents the width of the overhead view image. The specific values ​​of w and h are determined according to the resolution of the fisheye camera.

[0070] In this step, the overhead image in any direction contains valid areas and invalid areas. The valid area is located in the shooting area of ​​the camera corresponding to the overhead image. The pixel value in the valid area is usually not 0, and the pixel grayscale value in the invalid area is 0.

[0071] For example, in an optional embodiment, the overhead view images in various directions include a first overhead view image, a second overhead view image, a third overhead view image and a fourth overhead view image, where the first overhead view image refers to the overhead view image in front of the vehicle, the second overhead view image refers to the overhead view image behind the vehicle, the third overhead view image refers to the overhead view image of the vehicle clockwise, and the fourth overhead view image refers to the overhead view image of the vehicle counterclockwise.

[0072] Among them, the valid area included in the first overhead image refers to the area composed of the left front corner point and the right front corner point of the vehicle area included in the first overhead image, and the left front corner point and the right front corner point of the first overhead image; the valid area included in the second overhead image refers to the area composed of the left rear corner point and the right rear corner point of the vehicle area included in the second overhead image, and the left rear corner point and the right rear corner point of the second overhead image; the valid area included in the third overhead image refers to the area composed of the left front corner point and the left rear corner point of the vehicle area included in the third overhead image, and the left front corner point and the left rear corner point of the third overhead image; the valid area included in the fourth overhead image refers to the area composed of the right front corner point and the right rear corner point of the vehicle area included in the fourth overhead image, and the right front corner point and the right rear corner point of the fourth overhead image.

[0073] See also Figure 2 As shown, it is a schematic diagram of the overhead view images in various directions provided by the present embodiment, wherein (a) is a schematic diagram of the first overhead view image, the effective area included in the first overhead view image refers to the F area shown in the shade, the M area is the vehicle area, and the pixel grayscale values ​​in the M, L, R, and B areas are all 0; (b) is a schematic diagram of the second overhead view image, the effective area included in the second overhead view image refers to the B area shown in the shade, and the pixel grayscale values ​​in the M, L, R, and F areas are all 0; (c) is a schematic diagram of the third overhead view image, the effective area included in the third overhead view image refers to the L area shown in the shade, and the pixel grayscale values ​​in the M, R, F, and B areas are all 0; (d) is a schematic diagram of the fourth overhead view image, the effective area included in the fourth overhead view image refers to the R area shown in the shade, and the pixel grayscale values ​​in the M, L, F, and B areas are all 0.

[0074] Step S102: Calculate the average grayscale of the three channels corresponding to the overhead view images in each direction and the overall average grayscale of the three channels corresponding to the overhead view images in all directions.

[0075] The three-channel average grayscale includes the average grayscale of each channel of the corresponding overhead image, and the three-channel overall average grayscale includes the average grayscale of the same channel of the overhead image in all directions.

[0076] Optionally, the three-channel average grayscale includes the average grayscale of the r channel of the corresponding overhead image, the average grayscale of the g channel, and the average grayscale of the b channel, and the three-channel overall average grayscale includes the average grayscale of the r channel of the overhead image of all room types, the average grayscale of the g channel, and the average grayscale of the b channel.

[0077] Step S103 : Calculate the three-channel relative grayscales corresponding to the overhead view images in each direction according to the three-channel average grayscales corresponding to the overhead view images in each direction and the three-channel overall average grayscale.

[0078] The three-channel relative grayscale includes the relative grayscale of the same channel of the corresponding overhead image and the overhead image in the adjacent direction, which can reflect the difference in pixel grayscale values ​​between the corresponding overhead image and the overhead image in the adjacent direction.

[0079] Optionally, the three-channel relative grayscale includes the relative grayscale of the r channel, the relative grayscale of the g channel, and the relative grayscale of the b channel corresponding to the overhead view image and the overhead view image in an adjacent direction.

[0080] It can be understood that the top-view image in any direction has adjacent top-view images in the clockwise direction and adjacent top-view images in the counterclockwise direction. In this step, the "adjacent direction" can be either clockwise or counterclockwise. However, if it is in the clockwise direction, when calculating the relative grayscale of the three channels corresponding to the top-view image in any direction, the calculation is based on the adjacent top-view image in the clockwise direction. Similarly, if it is in the counterclockwise direction, when calculating the relative grayscale of the three channels corresponding to the top-view image in any direction, the calculation is based on the adjacent top-view image in the counterclockwise direction.

[0081] Step S104 , performing global white balance processing on the overhead view images in each direction according to the three-channel relative grayscales respectively corresponding to the overhead view images in each direction, to obtain white balanced images respectively corresponding to the overhead view images in each direction.

[0082] In this step, the three-channel relative grayscale corresponding to the overhead view image in any direction includes the three-channel relative grayscale of each pixel contained in the overhead view image. The pixel grayscale value of the overhead view image can be adjusted based on the three-channel relative grayscale of each pixel to obtain an adjusted image corresponding to the overhead view image. The adjusted image is the white-balanced image corresponding to the overhead view image.

[0083] As described in the previous steps, the overhead view image in any direction contains valid areas and invalid areas. In this step, the valid areas contained in the overhead view images in each direction are spliced ​​together to obtain a 360-degree surround view image of the vehicle.

[0084] The surround image white balance method provided by the present application first obtains the overhead images in all directions around the vehicle, then calculates the three-channel average grayscale corresponding to the overhead images in all directions and the three-channel overall average grayscale corresponding to the overhead images in all directions, then calculates the three-channel relative grayscale corresponding to the overhead images in all directions according to the three-channel average grayscale corresponding to the overhead images in all directions and the three-channel overall average grayscale, and finally performs global white balance processing on the overhead images in all directions according to the three-channel relative grayscale corresponding to the overhead images in all directions, and obtains the white-balanced images corresponding to the overhead images in all directions. The present application takes into account the three-channel overall average grayscale corresponding to the overhead images in all directions when calculating the relative grayscale, so that the global white balance processing can effectively eliminate the influence of light changes on the overhead images in all directions, so that the white balance effect of the overhead images in all directions is better and can better adapt to the complex environment of the vehicle. In addition, the influence of light changes is eliminated, so that the overhead images in all directions can be spliced ​​into a vehicle surround image with better effect.

[0085] An embodiment of the present application illustrates the process of the aforementioned "step S102, calculating the average grayscale of the three channels corresponding to the overhead view images in each direction and the overall average grayscale of the three channels corresponding to the overhead view images in all directions".

[0086] Specifically, the process of "step S102, calculating the average grayscale of the three channels corresponding to the overhead images in each direction and the overall average grayscale of the three channels corresponding to the overhead images in all directions" includes:

[0087] Step a1: Calculate the effective area of ​​the overhead image in each direction.

[0088] Here, the effective area refers to the area of ​​the effective region contained in the corresponding top view image, and the effective region is located in the shooting region of the camera corresponding to the corresponding top view image. For the specific introduction of the effective region, please refer to the introduction in the above embodiment, which will not be repeated here.

[0089] Optionally, the process of this step "calculating the effective area of ​​the overhead view image in each direction" may include: for each of the directions, processing the overhead view image in that direction into a binary image, and adding the grayscale values ​​contained in the binary image, and taking the sum as the effective area of ​​the overhead view image in that direction, so as to obtain the effective area of ​​the overhead view image in each direction.

[0090] Specifically, considering that the grayscale value of pixels in the invalid area is 0, this step can set a smaller threshold, such as 0.5, and process the overhead image into a binary image based on the preset threshold. Then, the grayscale values ​​contained in the binary image are added to obtain the effective area of ​​the corresponding overhead image.

[0091] by Figure 2 The first top view image shown is used as an example for explanation. The effective area of ​​the first top view image is:

[0092]

[0093] In the formula, bin F represents the binary image corresponding to the first overhead image, S F Indicates the effective area of ​​the first overhead image.

[0094] Similarly, the effective area S of the second top view image can be obtained B , the effective area S of the third top view image L and the effective area S of the fourth top view image R .

[0095] Step a2: for each direction, according to the effective area of ​​the overhead view image in that direction and the three-channel grayscale value of the overhead view image in that direction, calculate the three-channel average grayscale corresponding to the overhead view image in that direction, so as to obtain the three-channel average grayscale corresponding to the overhead view image in each direction.

[0096] Optionally, the specific implementation process of this step includes the following steps a21 to a23:

[0097] Step a21, adding the grayscale values ​​of each r channel contained in the overhead image in this direction to obtain a first sum value, and dividing the first sum value by the effective area of ​​the overhead image in this direction to obtain the average grayscale of the r channel corresponding to the overhead image in this direction.

[0098] Step a22, adding the grayscale values ​​of the g channels contained in the overhead image in this direction to obtain a second sum, and dividing the second sum by the effective area of ​​the overhead image in this direction to obtain the average grayscale of the g channels corresponding to the overhead image in this direction.

[0099] Step a23, adding the grayscale values ​​of each b channel contained in the overhead image in this direction to obtain a third sum value, and dividing the third sum value by the effective area of ​​the overhead image in this direction to obtain the average grayscale of the b channel corresponding to the overhead image in this direction.

[0100] by Figure 2 The first top-view image shown is used as an example for explanation. The following formula (2) is used to calculate the average grayscale of the r channel corresponding to the first top-view image.

[0101]

[0102] Among them, G Fr represents the average grayscale of the r channel corresponding to the first top view image, F r (i, j) represents the grayscale value of the r channel of the pixel with coordinates (i, j) in the first overhead image.

[0103] Similarly, the average grayscale G of the g channel corresponding to the first top view image can be obtained Fg and the average gray level G of the b channel Fb , and the average grayscale G of the r channel corresponding to the second top view image Br , g channel average gray G Bg and the average gray level G of the b channel Bb , and the average grayscale G of the r channel corresponding to the third top view image Lr , g channel average gray G Lg and the average gray level G of the b channel Lb , and the average grayscale G of the r channel corresponding to the fourth top view image Rr , g channel average gray G Rg and the average gray level G of the b channel Rb .

[0104] It should be noted that this embodiment does not limit the sequence of the aforementioned steps a21 to a23.

[0105] Step a3: Calculate the overall average grayscale of the three channels according to the average grayscale of the three channels corresponding to the overhead images in each direction.

[0106] Optionally, the specific implementation process of this step includes the following steps a31 to a33:

[0107] Step a31, multiply the average grayscales of the r channels corresponding to the overhead images in each direction to obtain a first product value, and raise the first product value to the fourth power to obtain the overall average grayscale of the r channel.

[0108] Step a32, multiply the average grayscales of the g channels corresponding to the overhead images in each direction to obtain a second product value, and raise the second product value to the fourth power to obtain the overall average grayscale of the g channel.

[0109] Step a33, multiply the average grayscales of the b channels corresponding to the overhead images in each direction to obtain a third product value, and raise the third product value to the fourth power to obtain the overall average grayscale of the b channel.

[0110] Specifically, the following formula (3) is used to calculate the overall average grayscale of the r channel, the overall average grayscale of the g channel, and the overall average grayscale of the b channel.

[0111]

[0112] Among them, G r Represents the overall average grayscale of the r channel, G g Indicates the overall average grayscale of the g channel, G b Indicates the overall average grayscale of the b channel.

[0113] It should be noted that this embodiment does not limit the sequence of the aforementioned steps a31 to a33.

[0114] Through the above steps a1 to a3, the three-channel average grayscale and the three-channel overall average grayscale corresponding to the overhead view images in each direction can be obtained. It should be noted that the above implementation processes are only the preferred implementation processes given in this application and are not intended to limit this application. For example, when calculating the overall average grayscale of the three channels, the average grayscale of the same channel of the overhead view images in all directions can also be weighted and summed, and so on.

[0115] This embodiment has a small amount of calculation and good optimization effect. Compared with the traditional white balance method, the overall average grayscale is added; when calculating the relative grayscale values ​​of adjacent images, the overall average grayscale is considered, so that the images after white balance are closer, the white balance effect is better, and the effect of splicing into a 360-degree surround view image of the vehicle is better.

[0116] The following embodiment introduces the process of the aforementioned "step S103, calculating the relative grayscale of the three channels corresponding to the overhead view images in each direction according to the average grayscale of the three channels and the overall average grayscale of the three channels corresponding to the overhead view images in each direction".

[0117] The following takes any one of the directions as an example to illustrate the process of calculating the three-channel relative grayscale corresponding to the overhead image in the direction in step S103. Specifically, the process may include:

[0118] Step b1, determine the initial r channel relative grayscale corresponding to the overhead image in this direction according to the average grayscale of the r channel corresponding to the overhead image in the adjacent direction of this direction, the average grayscale of the r channel corresponding to the overhead image in this direction and the overall average grayscale of the r channel, and adjust the initial r channel relative grayscale corresponding to the overhead image in this direction through a preset activation function to obtain the r channel relative grayscale corresponding to the overhead image in this direction.

[0119] Optionally, the process of "determining the initial r channel relative grayscale corresponding to the overhead view image in this direction according to the average grayscale of the r channel corresponding to the overhead view image in the adjacent direction of the direction, the average grayscale of the r channel corresponding to the overhead view image in this direction and the overall average grayscale of the r channel" may include: calculating the ratio of the average grayscale of the r channel corresponding to the overhead view image in the adjacent direction of the direction to the average grayscale of the r channel corresponding to the overhead view image in this direction as the first ratio, taking the square root of the first ratio to obtain the first ratio after the square root, and dividing the overall average grayscale of the r channel by the first ratio after the square root to obtain the initial r channel relative grayscale corresponding to the overhead view image in this direction.

[0120] Taking the clockwise direction as an example, for Figure 2 The first top-view image shown, the top-view image adjacent to the first top-view image in the clockwise direction is the fourth top-view image, then the initial r channel relative grayscale corresponding to the first top-view image is:

[0121]

[0122] In the formula, λ Fr It represents the initial r channel relative grayscale corresponding to the first overhead image, which can reflect the average grayscale ratio of the fourth overhead image to the first overhead image in the r channel.

[0123] Considering the calculated λ Fr When it is large (greater than 1), directly multiplying it with the grayscale value of the r channel of the first overhead view image will cause the grayscale value of the r channel of the first overhead view image to change greatly and become distorted. Therefore, the activation function can be used to adjust it to reduce the difference in the grayscale value of the r channel of the first overhead view image during white balance.

[0124] Here, the activation function is defined as the following formula (5).

[0125]

[0126] For the first overhead image, the process of the step of "adjusting the initial r channel relative grayscale corresponding to the overhead image in this direction by using a preset activation function" includes: adjusting the initial r channel relative grayscale λ corresponding to the first overhead image Fr Substitute into the activation function, that is, λ' Fr =f(λ Fr ), and obtain the relative grayscale λ' of the r channel corresponding to the first top view image Fr .

[0127] Similarly, the relative grayscale λ' of the r channel corresponding to the second top view image (the third top view image in the clockwise direction) can be obtained. Br , the relative grayscale λ′ of the r channel corresponding to the third top view image (the top view image adjacent to the first top view image in the clockwise direction)Lr , and the relative grayscale λ' of the r channel corresponding to the fourth top view image (the top view image adjacent to the clockwise direction is the second top view image) Rr .

[0128] Step b2, determine the initial g channel relative grayscale corresponding to the overhead image in this direction according to the average grayscale of the g channel corresponding to the overhead image in the adjacent direction of this direction, the average grayscale of the g channel corresponding to the overhead image in this direction and the overall average grayscale of the g channel, and adjust the initial g channel relative grayscale corresponding to the overhead image in this direction through an activation function to obtain the g channel relative grayscale corresponding to the overhead image in this direction.

[0129] Optionally, the process of "determining the initial g channel relative grayscale corresponding to the overhead view image in this direction according to the average grayscale of the g channel corresponding to the overhead view image in the adjacent direction of the direction, the average grayscale of the g channel corresponding to the overhead view image in this direction and the overall average grayscale of the g channel" may include: calculating the ratio of the average grayscale of the g channel corresponding to the overhead view image in the adjacent direction of the direction to the average grayscale of the g channel corresponding to the overhead view image in this direction, as the second ratio, and taking the square root of the second ratio to obtain the second ratio after the square root, and dividing the overall average grayscale of the g channel by the second ratio after the square root to obtain the initial g channel relative grayscale corresponding to the overhead view image in this direction.

[0130] Taking the clockwise direction as an example, for Figure 2 The first top-view image shown, the top-view image adjacent to the first top-view image in the clockwise direction is the fourth top-view image, then the initial g channel relative grayscale corresponding to the first top-view image is:

[0131]

[0132] In the formula, λ Fg It represents the initial relative grayscale of the g channel corresponding to the first overhead image, which can reflect the average grayscale ratio of the fourth overhead image to the first overhead image in the g channel.

[0133] Considering the calculated λ Fg When it is large (greater than 1), directly multiplying it with the grayscale value of the g channel of the first overhead view image will cause the grayscale value of the g channel of the first overhead view image to change greatly and become distorted. Therefore, the activation function can be used to adjust it to reduce the difference in the grayscale value of the g channel of the first overhead view image during white balance.

[0134] For the first overhead image, the process of the step of "adjusting the initial g channel relative grayscale corresponding to the overhead image in this direction by using the activation function" includes: adjusting the initial g channel relative grayscale λ corresponding to the first overhead image Fg Substitute into the activation function, that is, λ' Fg =f(λFg ), and obtain the relative grayscale λ' of the g channel corresponding to the first top view image Fg .

[0135] Similarly, the relative grayscale λ' of the g channel corresponding to the second top view image can be obtained Bg , the relative grayscale λ' of the g channel corresponding to the third top view image Lg , and the relative grayscale λ' of the g channel corresponding to the fourth top view image Rg .

[0136] Step b3, determine the initial b-channel relative grayscale corresponding to the overhead view image in this direction according to the average b-channel grayscale corresponding to the overhead view image in the adjacent direction of this direction, the average b-channel grayscale corresponding to the overhead view image in this direction and the overall average grayscale of the b-channel, and adjust the initial b-channel relative grayscale corresponding to the overhead view image in this direction through a preset activation function to obtain the b-channel relative grayscale corresponding to the overhead view image in this direction.

[0137] Optionally, the process of "determining the initial b-channel relative grayscale corresponding to the overhead view image in the direction according to the b-channel average grayscale corresponding to the overhead view image in the adjacent direction of the direction, the b-channel average grayscale corresponding to the overhead view image in the direction and the overall average grayscale of the b-channel" may include: calculating the ratio of the b-channel average grayscale corresponding to the overhead view image in the adjacent direction of the direction to the b-channel average grayscale corresponding to the overhead view image in the direction as a third ratio, taking the square root of the third ratio to obtain the third ratio after the square root, and dividing the overall average grayscale of the b channel by the third ratio after the square root to obtain the initial b-channel relative grayscale corresponding to the overhead view image in the direction.

[0138] Taking the clockwise direction as an example, for Figure 2 The first top-view image shown, the top-view image adjacent to the first top-view image in the clockwise direction is the fourth top-view image, then the initial b channel relative grayscale corresponding to the first top-view image is:

[0139]

[0140] In the formula, λ Fb It represents the initial b-channel relative grayscale corresponding to the first overhead image, which can reflect the average grayscale ratio of the fourth overhead image to the first overhead image in the b-channel.

[0141] Considering the calculated λ Fb When it is large (greater than 1), directly multiplying it with the grayscale value of the b channel of the first overhead image will cause the grayscale value of the b channel of the first overhead image to change greatly and become distorted. Therefore, it can be adjusted through the activation function to reduce the difference in the grayscale value of the b channel of the first overhead image during white balance.

[0142] For the first overhead image, the process of the step of "adjusting the initial b channel relative grayscale corresponding to the overhead image in this direction by using the activation function" includes: adjusting the initial b channel relative grayscale λ corresponding to the first overhead image Fb Substitute into the activation function, that is, λ' Fb =f(λ Fb ), and obtain the relative grayscale λ' of the b channel corresponding to the first top view image Fb .

[0143] Similarly, the relative grayscale λ' of the b channel corresponding to the second top view image can be obtained Bb , the relative grayscale λ′ of the b channel corresponding to the third top view image Lb , and the relative grayscale λ' of the b channel corresponding to the fourth top view image Rb .

[0144] The calculations are performed for each direction according to the above steps b1 to b3, so that the relative grayscale of the R channel, the relative grayscale of the G channel and the relative grayscale of the B channel corresponding to the overhead image in each direction can be obtained.

[0145] This embodiment provides a surround view image white balance method with small calculation amount and good optimization effect. Compared with the traditional white balance scheme, an activation function is added to adjust the initial three-channel relative grayscale values ​​corresponding to the overhead view image, so that the grayscale value of the overhead view image can be maintained in a reasonable range while being adjusted, and the overhead view image will not be distorted or the overhead view image information will not be lost due to a large adjustment range.

[0146] In an optional embodiment, this embodiment describes the process of the aforementioned "step S104, performing global white balance processing on the overhead view images in each direction according to the relative grayscales of the three channels corresponding to the overhead view images in each direction, to obtain white balanced images corresponding to the overhead view images in each direction."

[0147] The following takes any one of the directions as an example to illustrate the process of obtaining the white-balanced image corresponding to the overhead image in the direction in step S104. Specifically, the process may include:

[0148] Step c1, multiplying the relative grayscale of the r channel corresponding to the overhead image in the direction by the grayscale values ​​of the r channels contained in the overhead image in the direction to obtain the adjusted grayscale values ​​of the r channels contained in the overhead image in the direction.

[0149] by Figure 2 Taking the first top-view image shown as an example, the following formula (8) can be used to obtain the adjusted r channel grayscale values ​​contained in the first top-view image.

[0150] F′ r (i,j)=F r (i,j)*λ'Fr Formula (8)

[0151] In the formula, F′ r (i, j) represents the adjusted r channel grayscale value corresponding to the pixel with coordinates (i, j) in the first overhead image.

[0152] For the second top view image, the third top view image and the fourth top view image, the calculation formula corresponds to formula (8), and this application will not repeat them again.

[0153] Step c2: multiply the relative grayscale of the g channel corresponding to the overhead image in the direction by the grayscale values ​​of the g channels contained in the overhead image in the direction to obtain the adjusted grayscale values ​​of the g channels contained in the overhead image in the direction.

[0154] by Figure 2 Taking the first top view image shown as an example, the following formula (9) can be used to obtain the adjusted g channel grayscale values ​​contained in the first top view image.

[0155] F′ g (i,j)=F g (i,j)*λ' Fg Formula (9)

[0156] In the formula, F′ g (i, j) represents the adjusted g channel grayscale value corresponding to the pixel with coordinates (i, j) in the first overhead image.

[0157] For the second top view image, the third top view image and the fourth top view image, the calculation formula corresponds to formula (9), and this application will not repeat them again.

[0158] Step c3: multiply the b channel relative grayscale corresponding to the overhead image in the direction by the b channel grayscale values ​​contained in the overhead image in the direction to obtain the adjusted b channel grayscale values ​​contained in the overhead image in the direction.

[0159] by Figure 2 Taking the first top view image shown as an example, the following formula (10) can be used to obtain the adjusted b channel grayscale values ​​contained in the first top view image.

[0160] F′ b (i,j)=F b (i,j)*λ' Fb Formula (10)

[0161] In the formula, F′ b (i, j) represents the adjusted b channel grayscale value corresponding to the pixel with coordinates (i, j) in the first overhead image.

[0162] For the second top-view image, the third top-view image and the fourth top-view image, the calculation formula corresponds to formula (10), and this application will not repeat them again.

[0163] Step c4, forming a white-balanced image corresponding to the overhead view image in the direction using the adjusted r channel grayscale values, the adjusted g channel grayscale values ​​and the adjusted b channel grayscale values ​​contained in the overhead view image in the direction.

[0164] The calculations are performed for each direction according to the above steps c1 to c4, so that the white-balanced images corresponding to the overhead images in each direction can be obtained.

[0165] Combining this embodiment with the above embodiments, the surround image white balance method provided by the present application has a small amount of calculation and a better optimization effect. Compared with the traditional white balance scheme, the overall average grayscale is added, and the overall average grayscale is considered when calculating the relative grayscale values ​​of adjacent images, so that the images after white balance are closer and the stitching effect is better; an activation function is added to adjust the original relative grayscale value, so that the grayscale can be maintained in a reasonable range while being adjusted, and the image will not be distorted or lost due to a large adjustment range. Information.

[0166] As described in the aforementioned step S104, the present application can stitch together the effective areas included in the overhead view images in various directions, that is, a 360-degree surround view image of the vehicle can be obtained.

[0167] Specifically, Figure 2 For example, the formulas for the three-channel grayscale values ​​of the (i, j) pixel contained in the vehicle's 360-degree surround image are as follows:

[0168]

[0169] In the formula, S r (i, j) represents the grayscale value of the pixel r channel with coordinates (i, j) in the spliced ​​vehicle 360-degree surround image S, F′ r (i, j) represents the adjusted r channel grayscale value corresponding to the pixel with coordinates (i, j) in the first overhead image, F represents the F area in the first overhead image, and B′ r (i, j) represents the adjusted r channel grayscale value corresponding to the pixel with coordinates (i, j) in the second top view image, B represents the B area in the second top view image, L' r (i, j) represents the adjusted r channel grayscale value corresponding to the pixel with coordinates (i, j) in the third top view image, L represents the L area in the first top view image, and R′ r(i, j) represents the adjusted r channel grayscale value corresponding to the pixel with coordinates (i, j) in the fourth overhead image, R represents the R area in the fourth overhead image, and M represents the M area (vehicle area) in any overhead image;

[0170]

[0171] In the formula, S g (i, j) represents the grayscale value of the pixel g channel with coordinates (i, j) in the spliced ​​vehicle 360-degree surround image S, F′ g (i, j) represents the adjusted g channel grayscale value corresponding to the pixel with coordinates (i, j) in the first overhead image, F represents the F area in the first overhead image, and B' g (i, j) represents the adjusted g channel grayscale value corresponding to the pixel with coordinates (i, j) in the second top view image, B represents the B area in the second top view image, L' g (i, j) represents the adjusted g channel grayscale value corresponding to the pixel with coordinates (i, j) in the third top view image, L represents the L area in the first top view image, and R' g (i, j) represents the adjusted g channel grayscale value corresponding to the pixel with coordinates (i, j) in the fourth overhead image, R represents the R area in the fourth overhead image, and M represents the M area (vehicle area) in any overhead image;

[0172]

[0173] In the formula, S b (i, j) represents the grayscale value of the pixel b channel with coordinates (i, j) in the spliced ​​vehicle 360-degree surround image S, F′ b (i, j) represents the adjusted b channel grayscale value corresponding to the pixel with coordinates (i, j) in the first overhead image, F represents the F area in the first overhead image, and B′ b (i, j) represents the adjusted b channel grayscale value corresponding to the pixel with coordinates (i, j) in the second top view image, B represents the B area in the second top view image, L' b (i, j) represents the adjusted b channel grayscale value corresponding to the pixel with coordinates (i, j) in the third top view image, L represents the L area in the first top view image, and R′ b (i, j) represents the adjusted b channel grayscale value corresponding to the pixel with coordinates (i, j) in the fourth overhead image, R represents the R area in the fourth overhead image, and M represents the M area (vehicle area) in any overhead image.

[0174] The surround view image white balance method provided in the present application enables a better white balance effect of the overhead view images in all directions, so that the 360-degree surround view image of the vehicle obtained by splicing in this embodiment has a better effect.

[0175] The embodiment of the present application further provides a surround view image white balancing device. The surround view image white balancing device provided by the embodiment of the present application is described below. The surround view image white balancing device described below and the surround view image white balancing method described above can be referenced to each other.

[0176] See also Figure 3 , which shows a schematic diagram of the structure of the surround image white balance device provided in an embodiment of the present application, such as Figure 3 As shown, the surround view image white balance device may include: a top view image acquisition module 301 , an average grayscale calculation module 302 , a relative grayscale calculation module 303 and an image white balance module 304 .

[0177] The overhead image acquisition module 301 is used to acquire overhead images in all directions around the vehicle.

[0178] The average grayscale calculation module 302 is used to calculate the three-channel average grayscale corresponding to the overhead view images in each direction and the three-channel overall average grayscale corresponding to the overhead view images in all directions, wherein the three-channel average grayscale includes the average grayscale of each channel of the corresponding overhead view image, and the three-channel overall average grayscale includes the average grayscale of the same channel of the overhead view images in all directions.

[0179] The relative grayscale calculation module 303 is used to calculate the three-channel relative grayscale corresponding to the overhead view images in each direction according to the three-channel average grayscale and the three-channel overall average grayscale corresponding to the overhead view images in each direction, wherein the three-channel relative grayscale includes the relative grayscale of the same channel of the corresponding overhead view image and the overhead image in the adjacent direction.

[0180] The image white balance module 304 is used to perform global white balance processing on the overhead images in each direction according to the three-channel relative grayscales respectively corresponding to the overhead images in each direction, so as to obtain white balanced images respectively corresponding to the overhead images in each direction.

[0181] The surround image white balance device provided by the present application first obtains the overhead images in all directions around the vehicle, then calculates the three-channel average grayscale corresponding to the overhead images in all directions and the three-channel overall average grayscale corresponding to the overhead images in all directions, then calculates the three-channel relative grayscale corresponding to the overhead images in all directions according to the three-channel average grayscale corresponding to the overhead images in all directions and the three-channel overall average grayscale, and finally performs global white balance processing on the overhead images in all directions according to the three-channel relative grayscale corresponding to the overhead images in all directions, and obtains the white-balanced images corresponding to the overhead images in all directions. The present application takes into account the three-channel overall average grayscale corresponding to the overhead images in all directions when calculating the relative grayscale, so that the global white balance processing can effectively eliminate the influence of light changes on the overhead images in all directions, so that the white balance effect of the overhead images in all directions is better and can better adapt to the complex environment of the vehicle. In addition, the influence of light changes is eliminated, so that the overhead images in all directions can be spliced ​​into a vehicle surround image with better effect.

[0182] In a possible implementation, the average grayscale calculation module 302 may include: an effective area calculation module, a three-channel average grayscale calculation module, and a three-channel overall average grayscale calculation module.

[0183] The effective area calculation module is used to calculate the effective area of ​​the overhead image in each direction. The effective area refers to the area of ​​the effective area contained in the corresponding overhead image. The effective area is located within the shooting area of ​​the camera corresponding to the corresponding overhead image.

[0184] The three-channel average grayscale calculation module is used to calculate the three-channel average grayscale corresponding to the overhead image in each direction according to the effective area of ​​the overhead image in that direction and the three-channel grayscale value of the overhead image in that direction, so as to obtain the three-channel average grayscale corresponding to the overhead image in each direction.

[0185] The three-channel overall average grayscale calculation module is used to calculate the three-channel overall average grayscale according to the three-channel average grayscales corresponding to the overhead images in each direction.

[0186] In a possible implementation, the effective area calculation module is specifically used to process the overhead view image in each direction into a binary image, and add the grayscale values ​​contained in the binary image, and use the sum as the effective area of ​​the overhead view image in that direction to obtain the effective area of ​​the overhead view image in each direction.

[0187] In a possible implementation, the above-mentioned three-channel average grayscale calculation module may include: an r channel average grayscale calculation module, a g channel average grayscale calculation module and a b channel average grayscale calculation module when calculating the three-channel average grayscale corresponding to the overhead image in this direction based on the effective area of ​​the overhead image in this direction and the three-channel grayscale value of the overhead image in this direction.

[0188] The r channel average grayscale calculation module is used to add the r channel grayscale values ​​contained in the overhead image in this direction to obtain a first sum value, and divide the first sum value by the effective area of ​​the overhead image in this direction to obtain the r channel average grayscale corresponding to the overhead image in this direction.

[0189] The g channel average grayscale calculation module is used to add the g channel grayscale values ​​contained in the overhead image in this direction to obtain a second sum value, and divide the second sum value by the effective area of ​​the overhead image in this direction to obtain the g channel average grayscale corresponding to the overhead image in this direction.

[0190] The b channel average grayscale calculation module is used to add the b channel grayscale values ​​contained in the overhead image in this direction to obtain a third sum value, and divide the third sum value by the effective area of ​​the overhead image in this direction to obtain the b channel average grayscale corresponding to the overhead image in this direction.

[0191] In a possible implementation, the above three-channel overall average grayscale calculation module may include: an r channel overall average grayscale calculation module, a g channel overall average grayscale calculation module and a b channel overall average grayscale calculation module.

[0192] The r channel overall average grayscale calculation module is used to multiply the r channel average grayscales corresponding to the overhead images in each direction to obtain a first product value, and to raise the first product value to the fourth power to obtain the r channel overall average grayscale.

[0193] The g channel overall average grayscale calculation module is used to multiply the g channel average grayscales corresponding to the overhead images in each direction to obtain a second product value, and raise the second product value to the fourth power to obtain the g channel overall average grayscale.

[0194] The b channel overall average grayscale calculation module is used to multiply the b channel average grayscales corresponding to the overhead images in each direction to obtain a third product value, and then raise the third product value to the fourth power to obtain the b channel overall average grayscale.

[0195] In a possible implementation, for each of the directions, the relative grayscale calculation module 303 may include: an r channel relative grayscale calculation module, a g channel relative grayscale calculation module and a b channel relative grayscale calculation module when calculating the three-channel relative grayscale corresponding to the overhead image in that direction.

[0196] The r channel relative grayscale calculation module is used to determine the initial r channel relative grayscale corresponding to the overhead image in this direction according to the r channel average grayscale corresponding to the overhead image in the adjacent direction of this direction, the r channel average grayscale corresponding to the overhead image in this direction and the overall average grayscale of the r channel, and adjust the initial r channel relative grayscale corresponding to the overhead image in this direction through a preset activation function to obtain the r channel relative grayscale corresponding to the overhead image in this direction.

[0197] The g channel relative grayscale calculation module is used to determine the initial g channel relative grayscale corresponding to the overhead image in this direction according to the g channel average grayscale corresponding to the overhead image in the adjacent direction of this direction, the g channel average grayscale corresponding to the overhead image in this direction and the overall average grayscale of the g channel, and adjust the initial g channel relative grayscale corresponding to the overhead image in this direction through an activation function to obtain the g channel relative grayscale corresponding to the overhead image in this direction.

[0198] The b channel relative grayscale calculation module is used to determine the initial b channel relative grayscale corresponding to the overhead image in this direction according to the b channel average grayscale corresponding to the overhead image in the adjacent direction of this direction, the b channel average grayscale corresponding to the overhead image in this direction and the overall average grayscale of the b channel, and adjust the initial b channel relative grayscale corresponding to the overhead image in this direction through an activation function to obtain the b channel relative grayscale corresponding to the overhead image in this direction.

[0199] In a possible implementation, the r channel relative grayscale calculation module may include: a first ratio calculation module, a first ratio square root module and an initial r channel relative grayscale calculation module when determining the initial r channel relative grayscale corresponding to the overhead view image in the direction according to the r channel average grayscale corresponding to the overhead view image in the adjacent direction of the direction, the r channel average grayscale corresponding to the overhead view image in the direction and the overall average grayscale of the r channel.

[0200] The first ratio calculation module is used to calculate the ratio of the average grayscale of the r channel corresponding to the overhead image in the adjacent direction to the direction and the average grayscale of the r channel corresponding to the overhead image in the direction as the first ratio.

[0201] The first ratio square root module is used to square the first ratio to obtain the first ratio after the square root.

[0202] The initial r channel relative grayscale calculation module is used to divide the overall average grayscale of the r channel by the first ratio after the square root to obtain the initial r channel relative grayscale corresponding to the overhead image in this direction.

[0203] Correspondingly, the above-mentioned g channel relative grayscale calculation module may include: a second ratio calculation module, a second ratio square root module and an initial g channel relative grayscale calculation module when determining the initial g channel relative grayscale corresponding to the overhead view image in this direction based on the g channel average grayscale corresponding to the overhead view image in the adjacent direction of this direction, the g channel average grayscale corresponding to the overhead view image in this direction and the overall average grayscale of the g channel.

[0204] The second ratio calculation module is used to calculate the ratio of the average grayscale of the g channel corresponding to the overhead image in the adjacent direction to the direction and the average grayscale of the g channel corresponding to the overhead image in the direction as the second ratio.

[0205] The second ratio square root module is used to square the second ratio to obtain the square rooted second ratio.

[0206] The initial g channel relative grayscale calculation module is used to divide the overall average grayscale of the g channel by the second ratio after the square root to obtain the initial g channel relative grayscale corresponding to the overhead image in this direction.

[0207] Correspondingly, the above-mentioned b channel relative grayscale calculation module may include: a third ratio calculation module, a third ratio square root module and an initial b channel relative grayscale calculation module when determining the initial b channel relative grayscale corresponding to the overhead view image in this direction based on the b channel average grayscale corresponding to the overhead view image in the adjacent direction of this direction, the b channel average grayscale corresponding to the overhead view image in this direction and the overall average grayscale of the b channel.

[0208] The third ratio calculation module is used to calculate the ratio of the average grayscale of the b channel corresponding to the overhead image in the adjacent direction to the direction and the average grayscale of the b channel corresponding to the overhead image in the direction as the third ratio.

[0209] The third ratio square root module is used to square the third ratio to obtain the third ratio after the square root.

[0210] The initial b channel relative grayscale calculation module is used to divide the overall average grayscale of the b channel by the third ratio after the square root to obtain the initial b channel relative grayscale corresponding to the overhead image in this direction.

[0211] In a possible implementation, for each of the directions, the image white balance module 304 may include: an r channel grayscale value adjustment module, a g channel grayscale value adjustment module, a b channel grayscale value adjustment module and a white balanced image determination module when obtaining the white balanced image corresponding to the overhead image in that direction.

[0212] The r channel grayscale value adjustment module is used to multiply the r channel relative grayscale corresponding to the overhead image in this direction by the r channel grayscale values ​​contained in the overhead image in this direction to obtain the adjusted r channel grayscale values ​​contained in the overhead image in this direction.

[0213] The g channel grayscale value adjustment module is used to multiply the g channel relative grayscale corresponding to the overhead image in this direction by the g channel grayscale values ​​contained in the overhead image in this direction to obtain the adjusted g channel grayscale values ​​contained in the overhead image in this direction.

[0214] The b channel grayscale value adjustment module is used to multiply the b channel relative grayscale corresponding to the overhead image in this direction by the b channel grayscale values ​​contained in the overhead image in this direction to obtain the adjusted b channel grayscale values ​​contained in the overhead image in this direction.

[0215] The white-balanced image determination module is used to form a white-balanced image corresponding to the overhead image in this direction from each adjusted r channel grayscale value, each adjusted g channel grayscale value and each adjusted b channel grayscale value contained in the overhead image in this direction.

[0216] In a possible implementation, the overhead view images in the above-mentioned directions include a first overhead view image, a second overhead view image, a third overhead view image and a fourth overhead view image. The first overhead view image refers to the overhead view image in front of the vehicle, the second overhead view image refers to the overhead view image behind the vehicle, the third overhead view image refers to the overhead view image of the vehicle clockwise, and the fourth overhead view image refers to the overhead view image of the vehicle counterclockwise.

[0217] Among them, the valid area included in the first overhead image refers to the area composed of the left front corner point and the right front corner point of the vehicle area included in the first overhead image, and the left front corner point and the right front corner point of the first overhead image; the valid area included in the second overhead image refers to the area composed of the left rear corner point and the right rear corner point of the vehicle area included in the second overhead image, and the left rear corner point and the right rear corner point of the second overhead image; the valid area included in the third overhead image refers to the area composed of the left front corner point and the left rear corner point of the vehicle area included in the third overhead image, and the left front corner point and the left rear corner point of the third overhead image; the valid area included in the fourth overhead image refers to the area composed of the right front corner point and the right rear corner point of the vehicle area included in the fourth overhead image, and the right front corner point and the right rear corner point of the fourth overhead image.

[0218] The present application also provides a surround image white balance device. Optionally, Figure 4 The hardware structure diagram of the surround image white balance device is shown in FIG. Figure 4 , the hardware structure of the surround image white balance device may include: at least one processor 401, at least one communication interface 402, at least one memory 403 and at least one communication bus 404;

[0219] In the embodiment of the present application, the number of the processor 401, the communication interface 402, the memory 403, and the communication bus 404 is at least one, and the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404;

[0220] The processor 401 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0221] The memory 403 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0222] The memory 403 stores a program, and the processor 401 can call the program stored in the memory 403, and the program is used to:

[0223] Obtain overhead images in all directions around the vehicle;

[0224] Calculate the three-channel average grayscale corresponding to the overhead images in each direction and the three-channel overall average grayscale corresponding to the overhead images in all directions, wherein the three-channel average grayscale includes the average grayscale of each channel of the corresponding overhead images, and the three-channel overall average grayscale includes the average grayscale of the same channel of the overhead images in all directions;

[0225] According to the three-channel average grayscale and the three-channel overall average grayscale of the overhead images in each direction, the three-channel relative grayscale corresponding to the overhead images in each direction is calculated, wherein the three-channel relative grayscale includes the relative grayscale of the same channel of the corresponding overhead image and the overhead image in the adjacent direction;

[0226] According to the relative grayscales of the three channels corresponding to the overhead images in various directions, global white balance processing is performed on the overhead images in various directions to obtain white balanced images corresponding to the overhead images in various directions.

[0227] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0228] The embodiment of the present application further provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the above-mentioned surround view image white balance method is implemented.

[0229] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0230] Finally, it should be noted that, in this article, relational terms such as and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprises a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0231] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0232] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A surround image white balance method, characterized in that: include: Obtain overhead images in all directions around the vehicle; Calculate the three-channel average grayscales corresponding to the overhead images in each direction and the three-channel overall average grayscales corresponding to the overhead images in all directions, wherein the three-channel average grayscales include the average grayscales of each channel of the corresponding overhead images, and the three-channel overall average grayscales include the average grayscales of the same channel of the overhead images in all directions; According to the three-channel average grayscales respectively corresponding to the overhead images in each direction and the overall average grayscale of the three channels, the three-channel relative grayscales respectively corresponding to the overhead images in each direction are calculated, wherein the three-channel relative grayscales include the relative grayscales of the same channel of the corresponding overhead image and the overhead image in the adjacent direction; According to the three-channel relative grayscales respectively corresponding to the overhead images in the respective directions, global white balance processing is performed on the overhead images in the respective directions to obtain white balanced images respectively corresponding to the overhead images in the respective directions; The step of calculating the relative grayscales of the three channels corresponding to the overhead images in each direction according to the average grayscales of the three channels corresponding to the overhead images in each direction and the overall average grayscale of the three channels comprises: For each of the directions: Determine the initial r channel relative grayscale corresponding to the overhead image in the direction according to the average grayscale of the r channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the r channel corresponding to the overhead image in the direction, and the overall average grayscale of the r channel, and adjust the initial r channel relative grayscale corresponding to the overhead image in the direction through a preset activation function to obtain the r channel relative grayscale corresponding to the overhead image in the direction; Determine the initial relative grayscale of the g channel corresponding to the overhead image in the direction according to the average grayscale of the g channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the g channel corresponding to the overhead image in the direction, and the overall average grayscale of the g channel, and adjust the initial relative grayscale of the g channel corresponding to the overhead image in the direction by the activation function to obtain the relative grayscale of the g channel corresponding to the overhead image in the direction; Determine the initial b-channel relative grayscale corresponding to the overhead image in the direction according to the average grayscale of the b-channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the b-channel corresponding to the overhead image in the direction, and the overall average grayscale of the b-channel, and adjust the initial b-channel relative grayscale corresponding to the overhead image in the direction by the activation function to obtain the b-channel relative grayscale corresponding to the overhead image in the direction; To obtain the relative grayscale of the r channel, the relative grayscale of the g channel and the relative grayscale of the b channel corresponding to the overhead images in each direction; The step of performing global white balance processing on the overhead images in each direction according to the three-channel relative grayscales respectively corresponding to the overhead images in each direction to obtain white-balanced images respectively corresponding to the overhead images in each direction includes: For each of the directions: The relative grayscale of the r channel corresponding to the overhead image in the direction is multiplied by the grayscale values ​​of each r channel contained in the overhead image in the direction to obtain the adjusted grayscale values ​​of each r channel contained in the overhead image in the direction; The relative grayscale of the g channel corresponding to the overhead image in the direction is multiplied by the grayscale values ​​of each g channel contained in the overhead image in the direction to obtain the adjusted grayscale values ​​of each g channel contained in the overhead image in the direction; The relative grayscale of the b channel corresponding to the overhead image in the direction is multiplied by the grayscale values ​​of each b channel contained in the overhead image in the direction to obtain the adjusted grayscale values ​​of each b channel contained in the overhead image in the direction; The white-balanced image corresponding to the overhead view image in the direction is composed of each adjusted r channel grayscale value, each adjusted g channel grayscale value, and each adjusted b channel grayscale value contained in the overhead view image in the direction; In order to obtain white-balanced images corresponding to the overhead images in each direction.

2. The surround image white balance method according to claim 1, characterized in that: The calculating the three-channel average grayscales corresponding to the overhead images in each direction and the three-channel overall average grayscales corresponding to the overhead images in all directions includes: Calculating the effective area of ​​the overhead image in each direction, where the effective area refers to the area of ​​the effective region included in the corresponding overhead image, and the effective region is located within the shooting region of the camera corresponding to the corresponding overhead image; For each of the directions, according to the effective area of ​​the overhead image in the direction and the three-channel grayscale values ​​of the overhead image in the direction, the three-channel average grayscale corresponding to the overhead image in the direction is calculated to obtain the three-channel average grayscales corresponding to the overhead images in the directions; The overall average grayscale of the three channels is calculated according to the average grayscales of the three channels respectively corresponding to the overhead images in each direction.

3. The surround image white balance method according to claim 2, characterized in that: The calculating the effective area of ​​the overhead image in each direction includes: For each of the directions, the overhead image in that direction is processed into a binary image, and the grayscale values ​​contained in the binary image are added, and the sum is used as the effective area of ​​the overhead image in that direction to obtain the effective area of ​​the overhead image in each direction.

4. The surround image white balance method according to claim 2, characterized in that: The calculating, according to the effective area of ​​the overhead image in the direction and the three-channel grayscale values ​​of the overhead image in the direction, the three-channel average grayscale corresponding to the overhead image in the direction comprises: Adding the grayscale values ​​of each r channel contained in the overhead image in the direction to obtain a first sum value, and dividing the first sum value by the effective area of ​​the overhead image in the direction to obtain the average grayscale of the r channel corresponding to the overhead image in the direction; Adding the grayscale values ​​of the g channels contained in the overhead image in the direction to obtain a second sum value, and dividing the second sum value by the effective area of ​​the overhead image in the direction to obtain the average grayscale of the g channels corresponding to the overhead image in the direction; The grayscale values ​​of each b channel contained in the overhead image in this direction are added to obtain a third sum value, and the third sum value is divided by the effective area of ​​the overhead image in this direction to obtain the average grayscale of the b channel corresponding to the overhead image in this direction.

5. The surround image white balance method according to claim 4, characterized in that: The calculating the overall average grayscale of the three channels according to the average grayscales of the three channels respectively corresponding to the overhead images in each direction comprises: Multiply the average grayscales of the r channels corresponding to the overhead images in each direction to obtain a first product value, and raise the first product value to the fourth power to obtain the overall average grayscale of the r channel; Multiply the average grayscales of the g channels corresponding to the overhead images in each direction to obtain a second product value, and raise the second product value to the fourth power to obtain the overall average grayscale of the g channel; The average grayscales of the b channels corresponding to the overhead images in each direction are multiplied to obtain a third product value, and the third product value is raised to the fourth power to obtain the overall average grayscale of the b channel.

6. The surround image white balance method according to claim 5, characterized in that: The step of determining the initial relative grayscale of the r channel corresponding to the overhead image in the direction according to the average grayscale of the r channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the r channel corresponding to the overhead image in the direction, and the overall average grayscale of the r channel comprises: Calculate the ratio of the average grayscale of the r channel corresponding to the overhead image in the adjacent direction to the average grayscale of the r channel corresponding to the overhead image in the direction as the first ratio; Taking the square root of the first ratio to obtain a first square rooted ratio; Divide the overall average grayscale of the r channel by the first ratio after the square root to obtain an initial relative grayscale of the r channel corresponding to the overhead image in that direction; The determining, according to the average grayscale of the g channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the g channel corresponding to the overhead image in the direction, and the overall average grayscale of the g channel, an initial relative grayscale of the g channel corresponding to the overhead image in the direction comprises: Calculate the ratio of the average grayscale of the g channel corresponding to the overhead image in the direction adjacent to the direction to the average grayscale of the g channel corresponding to the overhead image in the direction as the second ratio; Taking the square root of the second ratio to obtain a square rooted second ratio; Divide the overall average grayscale of the g channel by the second ratio after the square root to obtain an initial relative grayscale of the g channel corresponding to the overhead image in that direction; The step of determining the initial b-channel relative grayscale corresponding to the overhead image in the direction according to the b-channel average grayscale corresponding to the overhead image in the adjacent direction of the direction, the b-channel average grayscale corresponding to the overhead image in the direction, and the overall average grayscale of the b-channel comprises: Calculate the ratio of the average grayscale of the b channel corresponding to the overhead image in the adjacent direction to the average grayscale of the b channel corresponding to the overhead image in the direction as the third ratio; Taking the square root of the third ratio to obtain the square root of the third ratio; The overall average grayscale of the b channel is divided by the third ratio after the square root, so as to obtain an initial b channel relative grayscale corresponding to the overhead image in this direction.

7. The surround image white balance method according to claim 2, characterized in that: The overhead images in various directions include a first overhead image, a second overhead image, a third overhead image and a fourth overhead image, wherein the first overhead image refers to an overhead image in front of the vehicle, the second overhead image refers to an overhead image behind the vehicle, the third overhead image refers to an overhead image of the vehicle in a clockwise direction, and the fourth overhead image refers to an overhead image of the vehicle in a counterclockwise direction; The valid area included in the first overhead image refers to the left front corner point and the right front corner point of the vehicle area included in the first overhead image, and the area formed by the left front corner point and the right front corner point of the first overhead image; The valid area included in the second top-view image refers to the area formed by the left rear corner point and the right rear corner point of the vehicle area included in the second top-view image, and the left rear corner point and the right rear corner point of the second top-view image; The effective area included in the third top view image refers to the left front corner point and the left rear corner point of the vehicle area included in the third top view image, and the area formed by the left front corner point and the left rear corner point of the third top view image; The effective area included in the fourth bird's-eye view image refers to the right front corner point and the right rear corner point of the vehicle area included in the fourth bird's-eye view image, and the area formed by the right front corner point and the right rear corner point of the fourth bird's-eye view image.

8. A surround image white balance device, characterized in that: include: A bird's-eye view image acquisition module is used to acquire bird's-eye view images in all directions around the vehicle; An average grayscale calculation module is used to calculate the average grayscale of the three channels corresponding to the overhead images in each direction and the overall average grayscale of the three channels corresponding to the overhead images in all directions, wherein the average grayscale of the three channels includes the average grayscale of each channel of the corresponding overhead images, and the overall average grayscale of the three channels includes the average grayscale of the same channel of the overhead images in all directions; A relative grayscale calculation module, used to calculate the three-channel relative grayscales corresponding to the overhead images in each direction according to the three-channel average grayscales corresponding to the overhead images in each direction and the overall average grayscale of the three channels, wherein the three-channel relative grayscales include the relative grayscales of the same channel of the corresponding overhead image and the overhead image in the adjacent direction; An image white balance module, configured to perform global white balance processing on the overhead images in each direction according to the three-channel relative grayscales respectively corresponding to the overhead images in each direction, so as to obtain white-balanced images respectively corresponding to the overhead images in each direction; Wherein, the relative grayscale calculation module is specifically used for: For each of the directions: Determine the initial r channel relative grayscale corresponding to the overhead image in the direction according to the average grayscale of the r channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the r channel corresponding to the overhead image in the direction, and the overall average grayscale of the r channel, and adjust the initial r channel relative grayscale corresponding to the overhead image in the direction through a preset activation function to obtain the r channel relative grayscale corresponding to the overhead image in the direction; Determine the initial relative grayscale of the g channel corresponding to the overhead image in the direction according to the average grayscale of the g channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the g channel corresponding to the overhead image in the direction, and the overall average grayscale of the g channel, and adjust the initial relative grayscale of the g channel corresponding to the overhead image in the direction by the activation function to obtain the relative grayscale of the g channel corresponding to the overhead image in the direction; Determine the initial b-channel relative grayscale corresponding to the overhead image in the direction according to the average grayscale of the b-channel corresponding to the overhead image in the adjacent direction of the direction, the average grayscale of the b-channel corresponding to the overhead image in the direction, and the overall average grayscale of the b-channel, and adjust the initial b-channel relative grayscale corresponding to the overhead image in the direction by the activation function to obtain the b-channel relative grayscale corresponding to the overhead image in the direction; To obtain the relative grayscale of the r channel, the relative grayscale of the g channel and the relative grayscale of the b channel corresponding to the overhead images in each direction; Wherein, the image white balance module is specifically used for: For each of the directions: The relative grayscale of the r channel corresponding to the overhead image in the direction is multiplied by the grayscale values ​​of each r channel contained in the overhead image in the direction to obtain the adjusted grayscale values ​​of each r channel contained in the overhead image in the direction; The relative grayscale of the g channel corresponding to the overhead image in the direction is multiplied by the grayscale values ​​of the g channels contained in the overhead image in the direction to obtain the adjusted grayscale values ​​of the g channels contained in the overhead image in the direction; The relative grayscale of the b channel corresponding to the overhead image in the direction is multiplied by the grayscale values ​​of each b channel contained in the overhead image in the direction to obtain the adjusted grayscale values ​​of each b channel contained in the overhead image in the direction; The white-balanced image corresponding to the overhead view image in the direction is composed of the adjusted r channel grayscale values, the adjusted g channel grayscale values, and the adjusted b channel grayscale values ​​contained in the overhead view image in the direction; To obtain white-balanced images corresponding to the overhead images in each direction.

Images