White balance adjusting method and related device

Abstract

The invention discloses a white balance adjusting method and a related device, and relates to the technical field of image processing, and the method comprises the steps: obtaining a plurality of images collected by a plurality of cameras at the periphery of a vehicle, sampling the three-channel pixel values of a plurality of sampling points in each view overlapping region corresponding to each image, forming a three-channel pixel value set corresponding to each image to obtain two three-channel pixel value sets corresponding to the plurality of images, and determining respective correction coefficients of the plurality of images according to the two three-channel pixel value sets corresponding to the plurality of images, and performing white balance adjustment on the plurality of images according to the respective correction coefficients of the plurality of images to obtain a plurality of adjusted target images. According to the method and the device, the imaging comparison of the same object in each view overlapping region in the corresponding two images is used, the respective correction coefficients of the plurality of images are determined, and then white balance adjustment is performed, so that color cast and brightness change influences caused by ambient light can be eliminated, and the brightness and color of the images are ensured to be more consistent.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a white balance adjustment method and related devices. Background Art

[0002] Around View Monitor (AVM) systems are a key technology in modern automotive safety, providing drivers with comprehensive information about their surroundings to improve driving safety. AVM systems use multiple cameras to capture real-time images of the vehicle's surroundings and fuse them into a seamless panoramic image, simulating an outside view of the vehicle's surroundings. However, the ambient lighting conditions experienced by multiple cameras can vary, leading to inconsistent color and brightness when stitching images, a problem that urgently needs to be addressed. Summary of the Invention

[0003] In view of the above problems, this application provides a white balance adjustment method and related devices to solve the problem of inconsistent color and brightness when stitching images in the existing technology. The specific solution is as follows:

[0004] A first aspect of the present application provides a white balance adjustment method, comprising:

[0005] Acquire images captured by multiple cameras around the vehicle as multiple images, wherein adjacent cameras in the multiple cameras have overlapping fields of view, so that each image in the multiple images corresponds to two overlapping fields of view areas;

[0006] Sampling three-channel pixel values of a plurality of sampling points within each overlapping region of the visual field corresponding to each image to form a three-channel pixel value set corresponding to each image, so as to obtain two three-channel pixel value sets corresponding to each of the plurality of images;

[0007] determining a correction coefficient for each of the multiple images according to two three-channel pixel value sets corresponding to each of the multiple images;

[0008] White balance adjustment is performed on the multiple images according to the respective correction coefficients of the multiple images to obtain multiple adjusted target images.

[0009] In a possible implementation, determining the correction coefficients of the multiple images according to the two three-channel pixel value sets corresponding to the multiple images includes:

[0010] Calculating a three-channel mean and a three-channel standard deviation of each of the three-channel pixel value sets corresponding to each of the images as a set of eigenvalues for each of the images, so as to obtain two sets of eigenvalues for each of the multiple images;

[0011] determining a reference image from the plurality of images based on the two sets of feature values of each of the plurality of images;

[0012] Determining a preset correction coefficient as the correction coefficient of the reference image;

[0013] In order of the minimum number of images separated from the reference image, each image in the plurality of images except the reference image is sequentially used as an image to be processed;

[0014] determining an initial correction coefficient of the image to be processed based on target three-channel means of each of the image to be processed and an adjacent image to the image to be processed, wherein the minimum number of images between the adjacent image and the reference image is less than or equal to the minimum number of images between the image to be processed and the reference image, and the target three-channel mean refers to the three-channel mean in the overlapping field of view corresponding to the image to be processed and the adjacent image;

[0015] The correction coefficient of the image to be processed is determined according to the initial correction coefficient of the image to be processed and the correction coefficient of the adjacent image.

[0016] In a possible implementation, determining the reference image from the multiple images according to the two sets of feature values of each of the multiple images includes:

[0017] Calculating a color shift feature mean and a brightness feature mean of each of the multiple images based on the two sets of feature values of each of the multiple images;

[0018] Determining comprehensive feature values of each of the multiple images based on the color shift feature mean values and the brightness feature mean values of each of the multiple images;

[0019] The image with the smallest comprehensive characteristic value among the multiple images is determined as the reference image.

[0020] In a possible implementation, determining the comprehensive feature values of the multiple images according to the respective color shift feature means and brightness feature means of the multiple images includes:

[0021] Calculating an average of the brightness feature means of the plurality of images as a target brightness average;

[0022] Determining brightness feature deviation values of each of the multiple images according to the target brightness average value and the brightness feature mean values of each of the multiple images;

[0023] Comprehensive feature values of each of the multiple images are determined according to the brightness feature deviation values of each of the multiple images and the color shift feature mean values of each of the multiple images.

[0024] In a possible implementation, if the image to be processed corresponds to two adjacent images, the two adjacent images are determined as a first adjacent image and a second adjacent image;

[0025] The determining of the initial correction coefficient of the image to be processed based on the target three-channel means of each of the image to be processed and the adjacent images to the image to be processed, and determining the correction coefficient of the image to be processed based on the initial correction coefficient of the image to be processed and the correction coefficient of the adjacent images, includes:

[0026] determining a first initial correction coefficient of the image to be processed according to target three-channel means of each of the image to be processed and the first adjacent image;

[0027] determining a second initial correction coefficient of the image to be processed according to target three-channel means of each of the image to be processed and the second adjacent image;

[0028] Adding a first initial correction coefficient of the image to be processed and a correction coefficient of the first adjacent image to obtain a first sum value;

[0029] Adding the second initial correction coefficient of the image to be processed and the correction coefficient of the second adjacent image to obtain a second sum value;

[0030] An average of the first sum value and the second sum value is calculated as a correction coefficient of the image to be processed.

[0031] In a possible implementation, after determining the correction coefficient of the image to be processed based on the initial correction coefficient of the image to be processed and the correction coefficient of the adjacent image, the method further includes:

[0032] Acquire at least one historical correction coefficient of the image to be processed, wherein the historical correction coefficient refers to a correction coefficient determined at a historical moment;

[0033] Using the correction coefficient of the image to be processed as the current correction coefficient;

[0034] determining a final correction coefficient based on the at least one historical correction coefficient and the current correction coefficient;

[0035] The final correction coefficient is determined as the correction coefficient of the image to be processed.

[0036] In a possible implementation, the multiple images are images of the ground and the space above the ground around the vehicle;

[0037] The white balance adjustment method further includes:

[0038] Obtaining a pre-constructed bowl-shaped three-dimensional mesh model and camera intrinsic and extrinsic parameters corresponding to each of the multiple target images, wherein the bowl-shaped three-dimensional mesh model displays an image of the ground surrounding the vehicle at the bottom of the bowl and an image of the space above the ground at the edge of the bowl, and the bowl-shaped three-dimensional mesh model includes a plurality of pre-divided non-stitched areas and a plurality of stitched areas, each of the non-stitched areas and each of the stitched areas each containing a plurality of grid points, each of the non-stitched areas corresponding to one of the target images, and each of the stitched areas corresponding to two of the target images;

[0039] Determining pixel coordinates of grid points included in each of the plurality of non-stitching areas and the plurality of stitching areas based on camera intrinsic and extrinsic parameters corresponding to each of the plurality of target images, wherein each grid point included in the non-stitching area has one pixel coordinate, and each grid point included in the stitching area has two pixel coordinates;

[0040] For each non-joined area among the plurality of non-joined areas, extracting overall pixel values from one of the target images corresponding to the non-joined area according to pixel coordinates of grid points included in the non-joined area, and mapping the extracted overall pixel values onto the grid points included in the non-joined area to obtain overall pixel values at the grid points included in the non-joined area;

[0041] For each of the plurality of stitching areas, extracting overall pixel values from the two target images corresponding to the stitching area according to the pixel coordinates of the grid points included in the stitching area, weightedly fusing the overall pixel values extracted from the two target images, and mapping the fused pixel values to the grid points included in the non-stitching area to obtain overall pixel values on the grid points included in the stitching area;

[0042] According to the overall pixel values on the grid points respectively included in the multiple non-stitching areas and the multiple stitching areas, pixel interpolation is performed on the non-grid points in the bowl-shaped three-dimensional grid model to obtain a surround view image stitched from the multiple target images.

[0043] A second aspect of the present application provides a white balance adjustment device, comprising:

[0044] an image acquisition module, configured to acquire images captured by a plurality of cameras around the vehicle as a plurality of images, wherein adjacent cameras among the plurality of cameras have overlapping fields of view, such that each of the plurality of images corresponds to two overlapping fields of view;

[0045] a pixel value sampling module, configured to sample three-channel pixel values of a plurality of sampling points within each overlapping region of the visual field corresponding to each image, to form a three-channel pixel value set corresponding to each image, so as to obtain two three-channel pixel value sets corresponding to each of the plurality of images;

[0046] a correction coefficient determination module, configured to determine a correction coefficient for each of the plurality of images based on two sets of three-channel pixel values corresponding to each of the plurality of images;

[0047] The image white balance module is used to perform white balance adjustment on the multiple images according to the respective correction coefficients of the multiple images to obtain multiple adjusted target images.

[0048] A third aspect of the present application provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the white balance adjustment method of the first aspect or any implementation of the first aspect.

[0049] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0050] The memory is used to store computer programs;

[0051] The processor is configured to execute the computer program so that the electronic device can implement the white balance adjustment method of the first aspect or any implementation manner of the first aspect.

[0052] In a fifth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the white balance adjustment method of the above-mentioned first aspect or any implementation method of the first aspect.

[0053] By means of the above technical solution, the white balance adjustment method provided by the present application obtains images captured by multiple cameras around the vehicle as multiple images, samples the three-channel pixel values of multiple sampling points in each overlapping area of the field of view corresponding to each image, and forms a three-channel pixel value set corresponding to each image to obtain two three-channel pixel value sets corresponding to each of the multiple images. According to the two three-channel pixel value sets corresponding to each of the multiple images, the correction coefficients of each of the multiple images are determined, and the white balance of the multiple images is adjusted according to the correction coefficients of each of the multiple images to obtain multiple adjusted target images. Since the objects in the overlapping field of view only have color shift and brightness changes caused by ambient light in the corresponding two images, and the objects themselves do not differ, and the three-channel pixel values of the sampling points can represent the color shift and brightness of the sampling points, therefore, based on the three-channel pixel values of the sampling points in the overlapping field of view, the correction coefficients of each of the multiple images are determined, so that the correction coefficients can reflect the color shift and brightness changes caused by ambient light. Based on the correction coefficients of the multiple images, the white balance of the multiple images is adjusted separately, which can effectively eliminate the influence of color shift and brightness changes caused by ambient light, avoid the problems of inconsistent color and brightness and unnatural transition in the overlapping area when multiple target images are stitched together, and ensure the consistency of brightness and color of the entire surround view image. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0055] Figure 1 A flowchart of a white balance adjustment method provided in this application;

[0056] Figure 2 A schematic diagram of four cameras forming four overlapping fields of view;

[0057] Figure 3 Schematic diagram of pixel sampling corresponding to the overlapping areas of the four fields of view;

[0058] Figure 4 is a schematic diagram of a top view of a bowl-shaped three-dimensional mesh model;

[0059] Figure 5 is a schematic diagram of a front view of a bowl-shaped three-dimensional mesh model;

[0060] Figure 6 A schematic diagram of a surround view image stitched together before and after white balance adjustment provided by this application;

[0061] Figure 7A schematic diagram of the structure of a white balance adjustment device provided in this application;

[0062] Figure 8 This is a schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0063] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0064] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0065] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0066] The present application provides a white balance adjustment method. The white balance adjustment method of an embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0067] Reference Figure 1 , Figure 1 A schematic flow chart of a white balance adjustment method provided in an embodiment of the present application may include:

[0068] Step S401: Acquire images captured by multiple cameras around the vehicle as multiple images.

[0069] Optionally, the cameras around the vehicle may be fisheye cameras; optionally, the field of view of each camera may cover no less than 180° horizontally and no less than 120° vertically.

[0070] Taking the conventional installation method of the surround-view camera of a passenger car as an example, optionally, a total of four cameras can be installed on the front bumper, left mirror, right mirror and rear bumper (or nearby).

[0071] In this embodiment, adjacent cameras among the multiple cameras around the vehicle have overlapping fields of view, so that each of the multiple images corresponds to two overlapping fields of view areas.

[0072] See also Figure 2 As shown in FIG, it is a schematic diagram of four cameras forming four overlapping fields of view. Figure 2 , the first image captured by the camera at the front bumper of the car Corresponding to the visual field overlap area 1 and the visual field overlap area 2; the second image captured by the camera at the left mirror Corresponding to the visual field overlap area 1 and the visual field overlap area 3; the third image captured by the camera at the right mirror Corresponding to the visual field overlap area 2 and the visual field overlap area 4; the fourth image captured by the camera at the rear bumper of the vehicle Corresponding to the visual field overlapping area 3 and the visual field overlapping area 4.

[0073] Step S402: sampling three-channel pixel values of multiple sampling points in each overlapping area of each field of view corresponding to each image to form a three-channel pixel value set corresponding to each image, so as to obtain two three-channel pixel value sets corresponding to each of the multiple images.

[0074] Specifically, since the fields of view of two adjacent cameras overlap, the white balance difference between the two adjacent cameras can be calculated by comparing the images of the same object in the overlapping area of the field of view in the two cameras. Therefore, pixel sampling corresponding to the overlapping area of the field of view is required.

[0075] In this embodiment, n sampling points can be selected in each overlapping area of the field of view in the real world, and the three-channel pixel values of the corresponding pixel points in the two corresponding images are taken. The three-channel pixel values of the n pixel points taken from each image constitute a three-channel pixel value set.

[0076] See also Figure 3 , is a schematic diagram of pixel sampling corresponding to the overlapping areas of the four visual fields. Figure 3 Since each image corresponds to two overlapping fields of view, this embodiment can sample 8 sets of data, that is, each overlapping field of view can sample 2 three-channel pixel value sets, and 4 images form 4 overlapping fields of view, and a total of 8 three-channel pixel value sets can be sampled.

[0077] by Figure 2 Taking the visual field overlap area 1 in the left front of the vehicle as an example, this embodiment can use the ground 0 to 1 meter beyond the front of the vehicle and 0 to 1 meter beyond the left side of the vehicle in the real world as the visual field overlap area 1. The visual field overlap area 1 is imaged by both the camera at the front bumper and the camera at the left rearview mirror (i.e., the first image and the second image mentioned above).

[0078] Optionally, in this embodiment, sampling points can be taken at intervals of 2 cm in length and width within the selected visual field overlap area 1, so that n=2500 sampling points can be obtained, and the world coordinates of each of the 2500 sampling points can be easily obtained. .

[0079] Next, the world coordinates of the 2500 sampling points can be calculated based on the internal and external parameters of the camera at the front bumper of the car. Convert it into the pixel coordinates in the first image to obtain 2500 pixel coordinates in the first image, and then extract the three-channel pixel values from the first image based on the converted pixel coordinates to obtain 2500 three-channel pixel values, which constitute a three-channel pixel value set corresponding to the first image; similarly, according to the internal and external parameters of the camera at the left reflector, the world coordinates of the 2500 sampling points are converted to The pixel coordinates are converted into pixel coordinates in the second image to obtain 2500 pixel coordinates in the second image, and then three-channel pixel values are extracted from the second image based on the converted pixel coordinates to obtain 2500 three-channel pixel values, which constitute a three-channel pixel value set corresponding to the second image.

[0080] All overlapping areas of the visual field are sampled according to the above process, that is, two three-channel pixel value sets corresponding to multiple images can be obtained.

[0081] Optionally, the above three-channel pixel values may be: Y channel pixel value, U channel pixel value and V channel pixel value; optionally, the above three-channel pixel values may also be: R channel pixel value, G channel pixel value and B channel pixel value.

[0082] The intrinsic and extrinsic parameters of any of the above-mentioned cameras can be obtained by pre-calibration. In an optional implementation, a tool such as a checkerboard calibration cloth or calibration plate can be used to make the tool appear in the camera's field of view and capture an image of the work. Feature points are selected on the tool (for example, checkerboard corner points are used as feature points), and the world coordinates of the feature points and their corresponding pixel coordinates in the image are measured and recorded as a set of cases, and multiple sets of cases are recorded. Optionally, the Zhang Zhengyou calibration method and the PnP (Perspective-n-Point) algorithm can be used to calculate the intrinsic and extrinsic parameters of the camera (including intrinsic parameters, distortion coefficients, and extrinsic parameters).

[0083] It should be noted that the above calibration process is only an example and is not intended to limit the present application.

[0084] Step S403: Determine the correction coefficients of the multiple images according to the two three-channel pixel value sets corresponding to the multiple images.

[0085] As mentioned above, two adjacent images have a common overlapping field of view. The three-channel pixel value sets sampled by the two adjacent images in the common overlapping field of view are based on data sampled from the same object. Therefore, the difference between the three-channel pixel value sets corresponding to the same object in the two adjacent images is most likely because the cameras are in different ambient light. Therefore, the correction coefficients of multiple images can be determined by comparing the two three-channel pixel value sets corresponding to the multiple images.

[0086] Step S404 : performing white balance adjustment on the multiple images according to their respective correction coefficients to obtain multiple adjusted target images.

[0087] As mentioned earlier, the differences between the two three-channel pixel value sets corresponding to multiple images are most likely caused by the cameras being in different ambient lighting conditions. Therefore, adjusting the white balance of multiple images based on the correction coefficients determined above can effectively eliminate the effects of color cast and brightness changes caused by ambient light, resulting in multiple target images with consistent white balance.

[0088] The white balance adjustment method provided in the present application obtains images captured by multiple cameras around a vehicle as multiple images, samples the three-channel pixel values of multiple sampling points in each overlapping area of the field of view corresponding to each image, and forms a three-channel pixel value set corresponding to each image to obtain two three-channel pixel value sets corresponding to each of the multiple images. Based on the two three-channel pixel value sets corresponding to each of the multiple images, correction coefficients for each of the multiple images are determined, and white balance adjustment is performed on the multiple images based on the correction coefficients for each of the multiple images to obtain multiple adjusted target images. Since the objects in the overlapping field of view only have color shift and brightness changes caused by ambient light in the corresponding two images, and the objects themselves do not differ, and the three-channel pixel values of the sampling points can represent the color shift and brightness of the sampling points, therefore, based on the three-channel pixel values of the sampling points in the overlapping field of view, the correction coefficients of each of the multiple images are determined, so that the correction coefficients can reflect the color shift and brightness changes caused by ambient light. Based on the correction coefficients of the multiple images, the white balance of the multiple images is adjusted separately, which can effectively eliminate the influence of color shift and brightness changes caused by ambient light, avoid the problems of inconsistent color and brightness and unnatural transition in the overlapping area when multiple target images are stitched together, and ensure the consistency of brightness and color of the entire surround view image.

[0089] In some embodiments of the present application, the process of “step S403, determining the correction coefficients of the multiple images according to the two three-channel pixel value sets corresponding to the multiple images” is introduced.

[0090] In this embodiment, for each of the multiple images, the three-channel mean and three-channel standard deviation of each three-channel pixel value set corresponding to the image can be calculated as a set of eigenvalues for the image. Since the image corresponds to two three-channel pixel value sets, this embodiment can obtain two sets of eigenvalues for the image. By performing the above calculations on each of the multiple images, two sets of eigenvalues can be obtained for each of the multiple images.

[0091] Taking the three-channel pixel value including the Y channel pixel value, the U channel pixel value and the V channel pixel value as an example, the first image in the previous text For example, the two three-channel pixel value sets corresponding to the first image are respectively recorded as a first three-channel pixel value set and a second three-channel pixel value set.

[0092] Calculate the mean of all Y channel pixel values, all U channel pixel values, and all V channel pixel values contained in the first three-channel pixel value set to obtain the Y channel mean , U channel mean and V channel mean ; Calculate the mean of all Y channel pixel values, all U channel pixel values, and all V channel pixel values contained in the second three-channel pixel value set to obtain the Y channel mean , U channel mean and V channel mean .

[0093] Calculate the standard deviation of all Y channel pixel values, all U channel pixel values, and all V channel pixel values contained in the first three-channel pixel value set, and obtain the Y channel standard deviation , U channel standard deviation and V channel standard deviation ; Calculate the standard deviation of all Y channel pixel values, all U channel pixel values and all V channel pixel values contained in the second three-channel pixel value set respectively, and obtain the Y channel standard deviation , U channel standard deviation and V channel standard deviation .

[0094] So, the first image The first set of eigenvalues includes: Y channel mean , U channel mean , V channel mean , Y channel standard deviation , U channel standard deviation and V channel standard deviation ; First image The second set of eigenvalues includes: Y channel mean , U channel mean , V channel mean , Y channel standard deviation , U channel standard deviation and V channel standard deviation .

[0095] In this embodiment, a reference image may be determined from the multiple images according to two sets of feature values corresponding to the multiple images, so that the non-reference images are aligned with the reference image by adjusting the non-reference images.

[0096] To ensure a more realistic surround view image after white balance adjustment, the reference image should be as color-free and simple as possible. To find a reference image that meets these requirements, this embodiment can optionally first calculate the mean color shift and brightness feature values for each of the multiple images based on their respective two sets of feature values. Then, based on these mean color shift and brightness feature values, a composite feature value for each of the multiple images is determined. Finally, the image with the smallest composite feature value among the multiple images is determined as the reference image.

[0097] In one possible implementation, taking any one of multiple images as an example, the process of calculating the mean color deviation feature of the image based on the two sets of eigenvalues of the image may include: calculating the color deviation features corresponding to the two sets of eigenvalues of the image respectively according to the U channel mean, V channel mean, U channel standard deviation and V channel standard deviation contained in the two sets of eigenvalues of the image, and then calculating the mean of the color deviation features corresponding to the two sets of eigenvalues of the image to obtain the mean color deviation feature of the image.

[0098] Optional, with image Taking the i-th group of eigenvalues (i is 1 or 2) as an example, the following formula (1) can be used to calculate the color shift feature corresponding to the i-th group of eigenvalues of the image.

[0099] Formula (1);

[0100] in, Representing an image The color shift feature corresponding to the i-th group of eigenvalues, Representing an image The i-th group of eigenvalues contains the U channel mean, Representing an image The i-th group of eigenvalues contains the V channel mean, Representing an image The i-th group of eigenvalues contains the U channel standard deviation, Representing an image The i-th group of eigenvalues contains the standard deviation of the V channel, Representing an image The first preset weight coefficient corresponding to the i-th group of eigenvalues is in the range of [0,1].

[0101] In order to make those skilled in the art better understand the above formula (1), the meaning of the formula (1) is explained as follows: the physical meaning of the first two terms of the formula is the deviation between the sampling point color and the gray color (U and V are both 128), which can filter out images with serious color cast problems to a certain extent. The physical meaning of the last term of the formula is the sum of the standard deviations of the U and V channels, which can effectively filter out images with complex colors. In specific implementation, you can choose the appropriate .

[0102] The above image The calculation formula of the color shift feature mean is as follows (2).

[0103] Formula (2);

[0104] in, Representing an image The mean value of the color cast feature.

[0105] In one possible implementation, taking any one of multiple images as an example, the process of calculating the brightness feature mean of the image based on the two sets of eigenvalues of the image may include: calculating the average of the Y channel means contained in the two sets of eigenvalues of the image, to obtain the brightness feature mean of the image.

[0106] Optionally, the process of "determining the comprehensive characteristic values of each of the multiple images based on the color deviation feature mean and the brightness feature mean of each of the multiple images" may include: calculating the average of the brightness feature means of each of the multiple images as the target brightness average; determining the brightness feature deviation value of each of the multiple images based on the target brightness average and the brightness feature mean of each of the multiple images; determining the comprehensive characteristic value of each of the multiple images based on the brightness feature deviation value of each of the multiple images and the color deviation feature mean of each of the multiple images.

[0107] It should be understood that in white balance adjustment, for brightness, it is sufficient to ensure that the final brightness of multiple images is consistent. In order to minimize the brightness adjustment distance, it is preferred that when screening the reference image, an image with an average lighting condition among multiple images can be selected. To this end, the average of the brightness feature means of each of the multiple images can be calculated to obtain the target brightness average.

[0108] Taking the case where the multiple images include four images as an example, in this embodiment, the brightness feature mean values of the four images may be summed, and the sum may be divided by four to obtain the target brightness average value.

[0109] Optionally, use multiple images in the image For example, according to the target brightness average and image The brightness feature mean of the image is determined The brightness characteristic deviation value of the image Brightness feature deviation value and image The mean value of the color shift feature of the image is determined The process of obtaining the comprehensive eigenvalue of can refer to the following formula (3).

[0110] Formula (3);

[0111] in, Representing an image The comprehensive characteristic value of Representing an image The mean brightness feature of represents the average target brightness, Representing an image The corresponding second preset weight coefficient has a value range of [0,1], Representing an image The brightness characteristic deviation value of .

[0112] In the above formula (3), this embodiment combines the evaluation criteria for the color shift dimension and the evaluation criteria for the brightness dimension. Since images with severe color shift cannot generally be selected as the reference image, the color shift coefficient is constant at 1 in formula (3). A second preset weight coefficient is added to formula (3) to indicate that the brightness coefficient is optional.

[0113] Optionally, the above two weight coefficients and Both can be 0.

[0114] As mentioned above, when selecting the reference image based on the two sets of eigenvalues mentioned above, images with severe color cast should be avoided. Large areas of highly saturated objects in the imaged object are prone to color cast (e.g., large areas of red are prone to green cast). At the same time, if the object has complex colors, it may cause unstable eigenvalues and should also be avoided. To this end, this embodiment calculates the coefficients It can effectively reflect the degree to which an object is close to gray and its color is pure. The smaller the value, the closer the object in the image is to gray and the simpler the color of the object is, and the more suitable the image is as a reference image. Based on this, this embodiment can determine the image with the smallest comprehensive feature value among multiple images as the reference image.

[0115] In this embodiment, the preset correction coefficient may be determined as the correction coefficient of the reference image. Optionally, the preset correction coefficient is 0, that is, no white balance adjustment may be performed on the reference image.

[0116] In order to determine the correction coefficient of the non-reference image, optionally, this embodiment can sequentially treat each image (i.e., non-reference image) among the multiple images except the reference image as a to-be-processed image in order of the minimum number of images separated from the reference image, and then sequentially determine the correction coefficient of each to-be-processed image according to the following process.

[0117] The first image captured by the camera at the front bumper of the vehicle mentioned above , the second image captured by the camera at the left mirror , the third image captured by the camera at the right mirror and the fourth image captured by the camera at the rear bumper For example, if the reference image determined above is the first image , then in this embodiment, the second image can be first and the third image As the images to be processed, and then the fourth image as the image to be processed.

[0118] For each image to be processed, this embodiment can first determine the initial correction coefficient of the image to be processed based on the target three-channel means of the image to be processed and its adjacent images. Here, the minimum number of images between the adjacent image and the reference image is less than or equal to the minimum number of images between the image to be processed and the reference image. The target three-channel mean refers to the three-channel mean in the overlapping area of the field of view corresponding to the image to be processed and the adjacent image.

[0119] The image to be processed is the second image For example, the adjacent image refers to the first image (The adjacent images at this time With the benchmark image The minimum number of images in the interval is 0, and the images to be processed With the benchmark image The minimum number of images in the interval is 0, which meets the above requirements for adjacent images; although the fourth image Also with the second image Adjacent, but the fourth image With the benchmark image The minimum number of images in an interval is 1, and the number of images to be processed is With the benchmark image The minimum number of images in the interval is 0, which does not meet the above requirements for adjacent images, so the fourth image Not adjacent images), the target three-channel mean refers to the adjacent images With the image to be processed The common corresponding visual field overlap area (i.e. Figure 2The three-channel mean under the overlapping field of view 1) shown, that is, the image to be processed The target three-channel mean is based on the second image The three-channel mean value calculated from the three-channel pixel values of multiple sampling points in the field of view overlap area 1, adjacent images The target three-channel mean is based on the first image The three-channel mean value calculated from the three-channel pixel values of multiple sampling points in the field of view overlap area 1.

[0120] Taking the target three-channel mean including the target Y channel mean, the target U channel mean and the target V channel mean as an example, optionally, the target Y channel mean of the adjacent image can be subtracted from the target Y channel mean of the image to be processed to obtain a Y channel difference, the target U channel mean of the adjacent image can be subtracted from the target U channel mean of the image to be processed to obtain a U channel difference, and the target V channel mean of the adjacent image can be subtracted from the target V channel mean of the image to be processed to obtain a V channel difference. Then, the Y channel difference, the U channel difference and the V channel difference are the initial correction coefficients of the image to be processed.

[0121] Then, the correction coefficient of the image to be processed is determined based on the initial correction coefficient of the image to be processed and the correction coefficients of adjacent images.

[0122] Optionally, the initial correction coefficients of the image to be processed and the correction coefficients of the adjacent images can be added together, and the sum can be used as the correction coefficient of the image to be processed. That is, the initial correction coefficients of the Y channel of the image to be processed and the correction coefficients of the Y channel of the adjacent images are added together to obtain the Y channel correction coefficient of the image to be processed; the initial correction coefficients of the U channel of the image to be processed and the correction coefficients of the U channel of the adjacent images are added together to obtain the U channel correction coefficient of the image to be processed; and the initial correction coefficients of the V channel of the image to be processed and the correction coefficients of the V channel of the adjacent images are added together to obtain the V channel correction coefficient of the image to be processed.

[0123] It should be understood that among multiple images, there may be a situation where one image to be processed corresponds to two adjacent images. For example, if the reference image is the first image , then when the fourth image When the image is determined to be processed, since the fourth image The adjacent images on the left and right sides are the same as the reference image There are 0 images between each, and the fourth image With the benchmark image The interval is 1 image, so the second image and the third image Can be used as the fourth image adjacent images.

[0124] In this embodiment, when the image to be processed corresponds to two adjacent images, this embodiment may determine the two adjacent images as a first adjacent image and a second adjacent image.

[0125] Then, the above process of "determining the initial correction coefficient of the image to be processed based on the target three-channel means of the image to be processed and the adjacent images to be processed, and determining the correction coefficient of the image to be processed based on the initial correction coefficient of the image to be processed and the correction coefficient of the adjacent images" may include: determining the first initial correction coefficient of the image to be processed based on the target three-channel means of the image to be processed and the first adjacent image, determining the second initial correction coefficient of the image to be processed based on the target three-channel means of the image to be processed and the second adjacent image, adding the first initial correction coefficient of the image to be processed and the correction coefficient of the first adjacent image to obtain a first sum, adding the second initial correction coefficient of the image to be processed and the correction coefficient of the second adjacent image to obtain a second sum, and calculating the average of the first sum and the second sum as the correction coefficient of the image to be processed.

[0126] The above processes of "determining the first initial correction coefficient of the image to be processed based on the target three-channel means of each of the image to be processed and the first adjacent image" and "determining the second initial correction coefficient of the image to be processed based on the target three-channel means of each of the image to be processed and the second adjacent image" are the same as the previous process of "determining the initial correction coefficient of the image to be processed based on the target three-channel means of each of the image to be processed and the adjacent images to be processed". For details, please refer to the previous introduction and will not be repeated here.

[0127] That is, when the number of images on both sides of the image that are spaced from the reference image is equal, this embodiment can calculate the cumulative sum of the correction coefficients on both sides respectively, and then calculate the average value of the two as the correction coefficient.

[0128] It should be noted that the above process of calculating the cumulative sum of the correction coefficients on both sides and then averaging the two is only an example and does not limit the present application. In addition, there may be other implementation methods, for example, randomly selecting one side to calculate the correction coefficient, etc.

[0129] In summary, this embodiment provides a process for determining the correction coefficients of multiple images. During the determination process, the color deviation dimension is taken into consideration, and the user can flexibly choose whether to consider the brightness dimension through the brightness coefficient, thereby improving the accuracy of the correction coefficient. In addition, the correction coefficient determination process is more user-friendly and provides a better user experience.

[0130] In a more preferred implementation, considering that for the two frames of surround view scenes that are continuous in time, the reference image determined according to the previous process may change, resulting in a sudden change in the correction coefficient, causing visual problems of sudden changes in color temperature and brightness, and reducing the user experience.

[0131] In order to solve the problem of sudden changes in color temperature and brightness, the inventors of this case have come up with the idea of adding an average filter to prevent jumps. Based on this, after the previous statement "determining the correction coefficient of the image to be processed based on the initial correction coefficient of the image to be processed and the correction coefficients of adjacent images", the embodiment of this application can also use the correction coefficient of the image to be processed as the current correction coefficient, and then obtain at least one historical correction coefficient of the image to be processed (the historical correction coefficient refers to the correction coefficient determined at a historical moment), and then determine the final correction coefficient based on the at least one historical correction coefficient and the current correction coefficient, and determine the final correction coefficient as the correction coefficient of the image to be processed.

[0132] For example, this embodiment can pre-build a first-in-first-out queue to store the correction coefficients of each image (including the image to be processed and the reference image), with at least one historical correction coefficient including the correction coefficients of the 9 historical moments before the current moment, and the first image For example, for the first image , pre-build a queue to store 10 correction coefficients, and get the first image each time After a correction coefficient is obtained, it is stored in the corresponding queue. Since the queue can only store 10 correction coefficients, the earliest correction coefficient will be removed from the queue, so that the queue always stores the most recent 10 correction coefficients (if there are less than 10 correction coefficients, the gaps can be filled with 0).

[0133] Optionally, an average value of at least one historical correction coefficient and a current correction coefficient may be calculated to obtain a final correction coefficient.

[0134] Of course, the process of "determining a final correction coefficient based on at least one historical correction coefficient and a current correction coefficient" can be implemented in other ways, which are not limited in this application. For example, weights can be set for the at least one historical correction coefficient and the current correction coefficient, and then a weighted sum is performed, with the weighted sum result being used as the final correction coefficient.

[0135] This embodiment adds average filtering, which can effectively prevent sudden changes in color temperature and brightness, thereby improving user experience.

[0136] In one possible implementation, the process of "adjusting the white balance of multiple images separately according to the respective correction coefficients of the multiple images" in the previous step S404 may include: for any one of the multiple images, subtracting the correction coefficient of the corresponding channel from the three-channel pixel value of each pixel of the image (for example, the Y channel pixel value, the U channel pixel value, and the V channel pixel value), and truncating the pixel channel values exceeding [0, 255] to within the range of [0, 255], that is, if the pixel channel value calculated for a pixel is less than 0, the pixel channel value of the pixel is set to 0; if the pixel channel value calculated for a pixel is greater than 255, the pixel channel value of the pixel is set to 255.

[0137] The embodiment of the present application performs additional white balance consistency preprocessing on the cameras of the AVM system, thereby avoiding problems such as inconsistent image color and brightness between adjacent cameras and excessive and unnatural overlapping areas, thereby ensuring the consistency of brightness and color of the entire surround view image.

[0138] In order to verify the effect of the surround-view image after white balance adjustment of the present application, the present application also provides a method for splicing multiple target images into a surround-view image.

[0139] Optionally, the above-mentioned multiple images are all images of the ground around the vehicle and the space above the ground.

[0140] Based on this, after obtaining multiple target images according to the above process, the embodiment of the present application can also obtain the pre-constructed bowl-shaped three-dimensional mesh model and the camera internal and external parameters corresponding to each of the multiple target images.

[0141] The calibration method for the camera's internal and external parameters has been introduced in the previous article and will not be repeated here. The following focuses on the bowl-shaped 3D mesh model.

[0142] See also Figure 4 and Figure 5 , which are schematic diagrams of a top view and a front view of a bowl-shaped 3D mesh model, respectively. In this embodiment, the bowl-shaped 3D mesh model allows the image of the ground around the vehicle to be displayed at the bottom of the bowl, and the image of the space above the ground to be displayed at the edge of the bowl.

[0143] In order to correspond to the overlapping area of the field of view mentioned above, this embodiment can pre-divide the bowl-shaped three-dimensional grid model into multiple non-stitching areas and multiple stitching areas, wherein each non-stitching area corresponds to one target image and each stitching area corresponds to two target images.

[0144] like Figure 4 and Figure 5 As shown, each non-joined area and each joined area respectively includes a plurality of grid points.

[0145] by Figure 2Taking the four cameras shown as an example, the bowl-shaped three-dimensional mesh model can be divided into eight areas: front, back, left, right, left front, left back, right front and right back. The front, back, left and right are non-stitching areas, corresponding to the first target image (i.e., the image obtained after white balance adjustment of the first image), the fourth target image (i.e., the image obtained after white balance adjustment of the fourth image), the second target image (i.e., the image obtained after white balance adjustment of the second image) and the third target image (i.e., the image obtained after white balance adjustment of the third image), respectively. The left front, left back, right front and right back are stitching areas. The left front area corresponds to the first target image and the second target image, the left back area corresponds to the fourth target image and the second target image, the right front area corresponds to the first target image and the third target image, and the right back area corresponds to the fourth target image and the third target image.

[0146] In order to be able to extract colors from each target image, each grid point contained in the bowl-shaped three-dimensional grid model can be converted from world coordinates to pixel coordinates in advance. That is, this embodiment can determine the pixel coordinates of the grid points contained in multiple non-stitched areas and multiple stitched areas according to the internal and external parameters of the camera corresponding to each of the multiple target images.

[0147] Since the stitching area corresponds to two target images, and the two target images each correspond to different camera intrinsic and extrinsic parameters, each grid point in the stitching area has two pixel coordinates; while the non-stitching area corresponds to only one target image, each grid point in the non-stitching area has only one pixel coordinate.

[0148] For example, taking the front area (non-stitching area) as an example, the camera internal and external parameters corresponding to the first target image (i.e. Figure 2 The camera intrinsic and extrinsic parameters of the camera at the front bumper of the vehicle are used to convert each grid point in the front area from world coordinates to pixel coordinates in the first target image. Taking the left front area (stitching area) as an example, each grid point in the left front area can be converted from world coordinates to pixel coordinates in the first target image based on the camera intrinsic and extrinsic parameters corresponding to the first target image, and each grid point in the left front area can be converted from world coordinates to pixel coordinates in the second target image based on the camera intrinsic and extrinsic parameters corresponding to the second target image.

[0149] Next, for each non-stitched area among the multiple non-stitched areas, this embodiment can extract the overall pixel value (the overall pixel value is obtained by the three-channel pixel value) from a target image corresponding to the non-stitched area according to the pixel coordinates of the grid points contained in the non-stitched area, and map the extracted overall pixel value to the grid points contained in the non-stitched area to obtain the overall pixel value on the grid points contained in the non-stitched area.

[0150] For example, taking the front area as an example, the overall pixel value at the corresponding pixel coordinates in the first target image can be extracted from the first target image based on the pixel coordinates of each grid point contained in the front area, and then the extracted overall pixel value is mapped to the grid point, thereby obtaining the overall pixel value of all grid points contained in the front area.

[0151] Similarly, for each stitching area in the multiple stitching areas, the overall pixel value is extracted from the two target images corresponding to the stitching area according to the pixel coordinates of the grid points contained in the stitching area, and after weighted fusion of the overall pixel values extracted from the two target images, the fused pixel value is mapped to the grid points contained in the non-stitching area to obtain the overall pixel value on the grid points contained in the stitching area.

[0152] For example, taking the left front area as an example, the overall pixel value at the corresponding pixel coordinates of each grid point contained in the left front area in the first target image can be extracted from the first target image, and then the overall pixel value at the corresponding pixel coordinates of each grid point in the second target image can be extracted from the second target image. Finally, the overall pixel values extracted from the first target image and the second target image for the same grid point are weightedly fused, and the obtained fused pixel value is mapped to the same grid point, thereby obtaining the overall pixel values of all grid points contained in the left front area.

[0153] Optionally, the weights used in the above-mentioned "weighted fusion" can be gradually transitioned from 0 to 1. For example, taking a grid point in the left front area as an example, the weights of the overall pixel value extracted from the first target image and the overall pixel value extracted from the second target image are determined based on the distance between the grid point and the left boundary of the left front area, and the distance between the grid point and the front boundary of the left front area. For example, the distance between the grid point and the left boundary of the left front area is 3, and the distance between the grid point and the front boundary of the left front area is 7. Then, for this grid point, the weight of the overall pixel value extracted from the first target image is 0.3, and the weight of the overall pixel value extracted from the second target image is 0.7.

[0154] Of course, the above weights are only examples and are not intended to limit this application.

[0155] Finally, the present application can perform pixel interpolation on non-grid points in the bowl-shaped three-dimensional grid model based on the overall pixel values on the grid points contained in multiple non-stitched areas and multiple stitched areas, and obtain a surround view image stitched together from multiple target images.

[0156] See also Figure 6 , which is a schematic diagram of a surround view image spliced before and after white balance adjustment provided in this application. Figure 6The left side is a surround view image obtained by stitching together multiple images obtained in step S401. It can be seen that the surround view image has obvious brightness and color inconsistencies. Figure 6 The right side is the surround view image obtained by stitching together multiple target images obtained in step S404. Figure 6 Compared to the surround view on the left, Figure 6 The surround image on the right has no obvious brightness and color changes, which provides a better viewing experience for users.

[0157] A white balance adjustment method provided by an embodiment of the present application is described above. The following describes a device for executing the white balance adjustment method.

[0158] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of a white balance adjustment device provided in an embodiment of the present application. Figure 7 As shown, the device may include:

[0159] An image acquisition module 501 is configured to acquire images captured by multiple cameras around the vehicle as a plurality of images, wherein adjacent cameras in the plurality of cameras have overlapping fields of view, such that each image in the plurality of images corresponds to two overlapping fields of view areas;

[0160] A pixel value sampling module 502 is configured to sample three-channel pixel values of a plurality of sampling points within each overlapping region of each field of view corresponding to each image, to form a three-channel pixel value set corresponding to each image, thereby obtaining two three-channel pixel value sets corresponding to each of the plurality of images;

[0161] A correction coefficient determination module 503 is configured to determine a correction coefficient for each of the multiple images based on two sets of three-channel pixel values corresponding to each of the multiple images;

[0162] The image white balance module 504 is configured to perform white balance adjustment on the multiple images according to the respective correction coefficients of the multiple images to obtain multiple adjusted target images.

[0163] In one possible implementation, when the correction coefficient determination module determines the correction coefficients of the multiple images based on the two three-channel pixel value sets corresponding to the multiple images, it can be specifically used to:

[0164] Calculate the three-channel mean and the three-channel standard deviation of each three-channel pixel value set corresponding to each image as a set of eigenvalues for each image, so as to obtain two sets of eigenvalues for each of the multiple images;

[0165] determining a reference image from the plurality of images according to the two sets of feature values of each of the plurality of images;

[0166] Determining a preset correction coefficient as a correction coefficient of a reference image;

[0167] In order of the minimum number of images separated from the reference image, each image except the reference image in the plurality of images is sequentially used as an image to be processed;

[0168] Determine the initial correction coefficient of the image to be processed based on the target three-channel mean of each of the image to be processed and its adjacent images, where the minimum number of images between the adjacent image and the reference image is less than or equal to the minimum number of images between the image to be processed and the reference image, and the target three-channel mean refers to the three-channel mean in the overlapping field of view corresponding to the image to be processed and the adjacent image;

[0169] The correction coefficient of the image to be processed is determined according to the initial correction coefficient of the image to be processed and the correction coefficients of adjacent images.

[0170] In a possible implementation, when determining the reference image from the multiple images based on the two sets of eigenvalues of the multiple images, the correction coefficient determination module may be specifically configured to:

[0171] Calculating the color deviation feature mean and the brightness feature mean of each of the multiple images according to the two sets of feature values of each of the multiple images;

[0172] Determining comprehensive feature values of the multiple images according to the color shift feature mean and the brightness feature mean of the multiple images;

[0173] The image with the smallest comprehensive eigenvalue among multiple images is determined as the reference image.

[0174] In one possible implementation, when the correction coefficient determination module determines the comprehensive feature values of the multiple images based on the color shift feature mean values and brightness feature mean values of the multiple images, it can be specifically used to:

[0175] Calculate the average of the brightness feature means of multiple images as the target brightness average;

[0176] Determining brightness feature deviation values of the plurality of images according to the target brightness average value and the brightness feature mean values of the plurality of images;

[0177] Comprehensive feature values of the multiple images are determined according to the brightness feature deviation values of the multiple images and the color deviation feature mean values of the multiple images.

[0178] In a possible implementation, if the image to be processed corresponds to two adjacent images, the two adjacent images are determined as a first adjacent image and a second adjacent image.

[0179] Then, the correction coefficient determination module can be used to determine the initial correction coefficient of the image to be processed based on the target three-channel means of the image to be processed and the adjacent images of the image to be processed, and to determine the correction coefficient of the image to be processed based on the initial correction coefficient of the image to be processed and the correction coefficient of the adjacent images. Specifically, it can be used to:

[0180] Determining a first initial correction coefficient of the image to be processed according to target three-channel means of the image to be processed and the first adjacent image;

[0181] determining a second initial correction coefficient of the image to be processed according to target three-channel means of the image to be processed and the second adjacent image;

[0182] Adding a first initial correction coefficient of the image to be processed and a correction coefficient of the first adjacent image to obtain a first sum value;

[0183] Adding the second initial correction coefficient of the image to be processed and the correction coefficient of the second adjacent image to obtain a second sum value;

[0184] The average of the first sum and the second sum is calculated as the correction coefficient of the image to be processed.

[0185] In a possible implementation, after determining the correction coefficient of the image to be processed based on the initial correction coefficient of the image to be processed and the correction coefficients of adjacent images, the correction coefficient determination module may further be used to:

[0186] Obtaining at least one historical correction coefficient of the image to be processed, wherein the historical correction coefficient refers to a correction coefficient determined at a historical moment;

[0187] Using the correction coefficient of the image to be processed as the current correction coefficient;

[0188] determining a final correction coefficient based on at least one historical correction coefficient and a current correction coefficient;

[0189] The final correction coefficient is determined as the correction coefficient of the image to be processed.

[0190] In a possible implementation, the multiple images are images of the ground around the vehicle and the space above the ground.

[0191] Then, the white balance adjustment device provided by the present application may further include: a model and parameter acquisition module, a pixel coordinate calculation module, a first mapping module, a second mapping module and a pixel interpolation module.

[0192] The model and parameter acquisition module is used to obtain the camera internal and external parameters corresponding to a pre-built bowl-shaped three-dimensional mesh model and multiple target images. The bowl-shaped three-dimensional mesh model displays the image of the ground around the vehicle at the bottom of the bowl, and the image of the space above the ground is displayed at the edge of the bowl. The bowl-shaped three-dimensional mesh model includes multiple pre-divided non-stitched areas and multiple stitched areas. Each non-stitched area and each stitched area each contains multiple grid points. Each non-stitched area corresponds to one target image, and each stitched area corresponds to two target images.

[0193] The pixel coordinate calculation module is used to determine the pixel coordinates of the grid points contained in each of the multiple non-stitching areas and the multiple stitching areas based on the camera internal and external parameters corresponding to each of the multiple target images, wherein each grid point contained in the non-stitching area has one pixel coordinate, and each grid point contained in the stitching area has two pixel coordinates.

[0194] The first mapping module is used to extract, for each non-spliced area among the multiple non-spliced areas, an overall pixel value from a target image corresponding to the non-spliced area according to the pixel coordinates of the grid points contained in the non-spliced area, and map the extracted overall pixel value to the grid points contained in the non-spliced area to obtain the overall pixel value on the grid points contained in the non-spliced area.

[0195] The second mapping module is used to extract the overall pixel value from the two target images corresponding to each stitching area of the multiple stitching areas according to the pixel coordinates of the grid points contained in the stitching area, and after weighted fusion of the overall pixel values extracted from the two target images, map the fused pixel values to the grid points contained in the non-stitching area to obtain the overall pixel values on the grid points contained in the stitching area.

[0196] The pixel interpolation module is used to perform pixel interpolation on non-grid points in the bowl-shaped three-dimensional grid model based on the overall pixel values of the grid points contained in the multiple non-spliced areas and the multiple spliced areas, so as to obtain a surround view image spliced from multiple target images.

[0197] The white balance adjustment device provided in this application corresponds to the white balance adjustment method provided above. For details, please refer to the above introduction and will not be repeated here.

[0198] An electronic device is also provided in an embodiment of the present application. Figure 8 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 8The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0199] like Figure 8 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 602 or programs loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing device 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0200] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 8 The electronic device is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0201] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the white balance adjustment methods provided in the embodiments of the present application.

[0202] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any white balance adjustment method provided in the embodiment of the present application.

[0203] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0204] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0205] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0206] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A white balance adjustment method, characterized in that: include: Acquire images captured by multiple cameras around the vehicle as multiple images, wherein adjacent cameras in the multiple cameras have overlapping fields of view, such that each image in the multiple images corresponds to two overlapping fields of view areas; Sampling three-channel pixel values of a plurality of sampling points within each overlapping region of the visual field corresponding to each image to form a three-channel pixel value set corresponding to each image, so as to obtain two three-channel pixel value sets corresponding to each of the plurality of images; determining a correction coefficient for each of the multiple images according to two three-channel pixel value sets corresponding to each of the multiple images; White balance adjustment is performed on the multiple images according to the respective correction coefficients of the multiple images to obtain multiple adjusted target images.

2. The white balance adjustment method according to claim 1, wherein: Determining the correction coefficients of the multiple images according to the two three-channel pixel value sets corresponding to the multiple images includes: Calculating a three-channel mean and a three-channel standard deviation of each of the three-channel pixel value sets corresponding to each of the images as a set of eigenvalues for each of the images, so as to obtain two sets of eigenvalues for each of the multiple images; determining a reference image from the plurality of images based on the two sets of feature values of each of the plurality of images; Determining a preset correction coefficient as the correction coefficient of the reference image; In order of the minimum number of images separated from the reference image, each image in the plurality of images except the reference image is sequentially used as an image to be processed; determining an initial correction coefficient of the image to be processed based on target three-channel means of each of the image to be processed and an adjacent image to the image to be processed, wherein the minimum number of images between the adjacent image and the reference image is less than or equal to the minimum number of images between the image to be processed and the reference image, and the target three-channel mean refers to the three-channel mean in the overlapping field of view corresponding to the image to be processed and the adjacent image; The correction coefficient of the image to be processed is determined according to the initial correction coefficient of the image to be processed and the correction coefficient of the adjacent image.

3. The white balance adjustment method according to claim 2, wherein: Determining a reference image from the multiple images according to the two sets of feature values of each of the multiple images includes: Calculating a color shift feature mean and a brightness feature mean for each of the multiple images based on the two sets of feature values for each of the multiple images; Determining comprehensive feature values of each of the multiple images based on the color shift feature mean values and the brightness feature mean values of each of the multiple images; The image with the smallest comprehensive characteristic value among the multiple images is determined as the reference image.

4. The white balance adjustment method according to claim 3, wherein: Determining the comprehensive feature values of each of the multiple images according to the color shift feature mean values and the brightness feature mean values of each of the multiple images includes: Calculating an average of the brightness feature means of the plurality of images as a target brightness average; Determining brightness feature deviation values of each of the multiple images according to the target brightness average value and the brightness feature mean values of each of the multiple images; Comprehensive feature values of each of the multiple images are determined according to the brightness feature deviation values of each of the multiple images and the color shift feature mean values of each of the multiple images.

5. The white balance adjustment method according to claim 2, wherein: If the image to be processed corresponds to two adjacent images, determining the two adjacent images as a first adjacent image and a second adjacent image; The determining of the initial correction coefficient of the image to be processed based on the target three-channel means of each of the image to be processed and the adjacent images to the image to be processed, and determining the correction coefficient of the image to be processed based on the initial correction coefficient of the image to be processed and the correction coefficient of the adjacent images, includes: determining a first initial correction coefficient of the image to be processed according to target three-channel means of each of the image to be processed and the first adjacent image; determining a second initial correction coefficient of the image to be processed according to target three-channel means of each of the image to be processed and the second adjacent image; Adding a first initial correction coefficient of the image to be processed and a correction coefficient of the first adjacent image to obtain a first sum value; Adding the second initial correction coefficient of the image to be processed and the correction coefficient of the second adjacent image to obtain a second sum value; An average of the first sum value and the second sum value is calculated as a correction coefficient of the image to be processed.

6. The white balance adjustment method according to claim 2, wherein: After determining the correction coefficient of the image to be processed according to the initial correction coefficient of the image to be processed and the correction coefficient of the adjacent image, the method further includes: Acquire at least one historical correction coefficient of the image to be processed, wherein the historical correction coefficient refers to a correction coefficient determined at a historical moment; Using the correction coefficient of the image to be processed as the current correction coefficient; determining a final correction coefficient based on the at least one historical correction coefficient and the current correction coefficient; The final correction coefficient is determined as the correction coefficient of the image to be processed.

7. The white balance adjustment method according to claim 1, wherein: The plurality of images are images of the ground and the space above the ground surrounding the vehicle; The white balance adjustment method further includes: Obtaining a pre-constructed bowl-shaped three-dimensional mesh model and camera intrinsic and extrinsic parameters corresponding to each of the multiple target images, wherein the bowl-shaped three-dimensional mesh model displays an image of the ground surrounding the vehicle at the bottom of the bowl and an image of the space above the ground at the edge of the bowl, and the bowl-shaped three-dimensional mesh model includes a plurality of pre-divided non-stitched areas and a plurality of stitched areas, each of the non-stitched areas and each of the stitched areas each containing a plurality of grid points, each of the non-stitched areas corresponding to one of the target images, and each of the stitched areas corresponding to two of the target images; Determining pixel coordinates of grid points included in each of the plurality of non-stitching areas and the plurality of stitching areas based on camera intrinsic and extrinsic parameters corresponding to each of the plurality of target images, wherein each grid point included in the non-stitching area has one pixel coordinate, and each grid point included in the stitching area has two pixel coordinates; For each non-joined area among the plurality of non-joined areas, extracting overall pixel values from one of the target images corresponding to the non-joined area according to pixel coordinates of grid points included in the non-joined area, and mapping the extracted overall pixel values onto the grid points included in the non-joined area to obtain overall pixel values at the grid points included in the non-joined area; For each of the plurality of stitching areas, extracting overall pixel values from the two target images corresponding to the stitching area according to the pixel coordinates of the grid points included in the stitching area, weightedly fusing the overall pixel values extracted from the two target images, and mapping the fused pixel values to the grid points included in the non-stitching area to obtain overall pixel values on the grid points included in the stitching area; According to the overall pixel values on the grid points respectively included in the multiple non-stitching areas and the multiple stitching areas, pixel interpolation is performed on the non-grid points in the bowl-shaped three-dimensional grid model to obtain a surround view image stitched from the multiple target images.

8. A white balance adjustment device, characterized in that: include: an image acquisition module, configured to acquire images captured by a plurality of cameras around the vehicle as a plurality of images, wherein adjacent cameras among the plurality of cameras have overlapping fields of view, such that each of the plurality of images corresponds to two overlapping fields of view; a pixel value sampling module, configured to sample three-channel pixel values of a plurality of sampling points within each overlapping region of the visual field corresponding to each image, to form a three-channel pixel value set corresponding to each image, so as to obtain two three-channel pixel value sets corresponding to each of the plurality of images; a correction coefficient determination module, configured to determine a correction coefficient for each of the plurality of images based on two sets of three-channel pixel values corresponding to each of the plurality of images; The image white balance module is used to perform white balance adjustment on the multiple images according to the respective correction coefficients of the multiple images to obtain multiple adjusted target images.

9. An electronic device, characterized in that: comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is configured to execute the computer program so that the electronic device can implement the white balance adjustment method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the white balance adjustment method according to any one of claims 1 to 7.

Images