Method and device for color adjustment of a surround view image, equipment and storage medium

Abstract

The present disclosure relates to a color adjustment method and device of a surround view image, equipment and a storage medium, wherein the method comprises: determining a target image and an adjacent reference image from a plurality of images, wherein there is an overlapping area between the target image and the reference image; processing a color equalization coefficient of the reference image, a color average value of the overlapping area in the reference image and a color average value of the overlapping area in the target image according to a preset relationship to determine a color equalization coefficient of the target image; and performing color adjustment on the target image according to the color equalization coefficient of the target image. According to the technical scheme of the present disclosure, the color of the image can be balanced, and the color difference at the joint of the surround view image can be reduced.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a method, apparatus, device and storage medium for color adjustment of a panoramic image. Background Technology

[0002] Due to differences in camera installation position and orientation, current vehicle surround view systems exhibit color differences at the stitching points of surround view images, resulting in noticeable seams and poor visual quality. Therefore, eliminating stitching marks in surround view images is essential.

[0003] In related technologies, Method 1 uses an image stitching algorithm to determine the optimal stitching seam between two adjacent images and performs fusion processing on the stitching seam to eliminate color differences at the stitching seam. This method has poor visual continuity in dynamic images. Method 2 adjusts the color based on histograms to compensate for color differences between adjacent images. This method adjusts the color of each image based on the color at a certain moment. The sampling data is small, making it difficult to meet the color adjustment needs under different environments during vehicle operation. The color adjustment effect needs to be improved. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this disclosure provides a method, apparatus, device and storage medium for color adjustment of panoramic images.

[0005] In a first aspect, embodiments of this disclosure provide a method for adjusting the color of a panoramic image, wherein the panoramic image is generated by stitching together multiple images, and the method includes:

[0006] A target image and a reference image adjacent to the target image are determined from the plurality of images, wherein there is an overlapping region between the target image and the reference image;

[0007] The color balance coefficient of the reference image, the average color value of the overlapping area in the reference image, and the average color value of the overlapping area in the target image are processed according to a preset relationship to determine the color balance coefficient of the target image.

[0008] The target image is color-adjusted based on its color balance coefficient.

[0009] Secondly, embodiments of this disclosure provide a color adjustment device for a panoramic image, the panoramic image being generated by stitching together multiple images, the device comprising:

[0010] An acquisition module is used to determine a target image and a reference image adjacent to the target image from the plurality of images, wherein there is an overlapping area between the target image and the reference image;

[0011] The determining module is used to process the color balance coefficient of the reference image, the average color value of the overlapping area in the reference image, and the average color value of the overlapping area in the target image according to a preset relationship, and to determine the color balance coefficient of the target image.

[0012] The adjustment module is used to adjust the color of the target image according to the color balance coefficient of the target image.

[0013] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the color adjustment method for the panoramic image described in the first aspect.

[0014] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the color adjustment method for the panoramic image described in the first aspect.

[0015] Fifthly, embodiments of this disclosure provide a vehicle including a color adjustment device for a surround view image as described in the second aspect above, or an electronic device as described in the third aspect above.

[0016] Compared with the prior art, the technical solution provided in this disclosure has the following advantages: By acquiring the target image and adjacent reference images from multiple images, and determining the color balance coefficient of the target image based on the color balance coefficient of the reference image, the average color value of the overlapping area in the reference image, and the average color value of the overlapping area in the target image, the color balance coefficient of each image is adjusted using the determined color balance coefficient to achieve color balance processing of the image. This reduces color differences at the stitching seams while ensuring visual continuity in the dynamic image. Furthermore, the current color balance coefficient is determined based on the average color value of the overlapping area of ​​each image during each stitching process. When the sampled data is small, the color of each image can be adjusted based on the current color balance coefficient to ensure the color adjustment effect of the panoramic image at different times and reduce color differences at the seams in the stitched panoramic image. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart illustrating a method for adjusting the color of a panoramic image provided in an embodiment of this disclosure.

[0020] Figure 2 This is a schematic flowchart illustrating another method for adjusting the color of a panoramic image provided in an embodiment of this disclosure;

[0021] Figure 3 This is a top view schematic diagram of a vehicle surround view system provided in an embodiment of the present disclosure;

[0022] Figure 4 A schematic diagram of the structure of a color adjustment device for a panoramic image provided in an embodiment of this disclosure;

[0023] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0024] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0025] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0026] Figure 1 This is a flowchart illustrating a color adjustment method for a surround view image provided in an embodiment of the present disclosure. The method provided in this embodiment can be executed by a color adjustment device for the surround view image. This device can be implemented using software and / or hardware and can be integrated into any electronic device with computing capabilities, such as in-vehicle devices, smartphones, tablets, and other terminals.

[0027] like Figure 1 As shown, the color adjustment method for a panoramic image provided in this embodiment may include:

[0028] Step 101: Determine the target image and the reference image adjacent to the target image from multiple images, wherein there is an overlapping area between the target image and the reference image.

[0029] In this embodiment of the disclosure, the panoramic image is generated by stitching together multiple images, wherein there is an overlapping area between every two adjacent images. For example, the target image and the reference image are adjacent and there is an overlapping area between the target image and the reference image.

[0030] The following explains how to determine the overlapping region. Taking the target image and the reference image as an example, the overlapping region is determined through the following steps: obtain the camera calibration data corresponding to the target image and the reference image respectively; determine multiple discrete points of the overlapping region in the world coordinate system based on the camera calibration data; map the multiple discrete points onto the target image and the reference image to determine multiple projection points; fit the multiple projection points to determine the overlapping region.

[0031] In this example, multiple cameras capture multiple images, and the fields of view of adjacent cameras partially overlap. Camera calibration data is predetermined based on the multiple cameras. This calibration data determines the overlapping range of adjacent first and second cameras in a world coordinate system. This world coordinate system refers to the coordinate system described by the camera's extrinsic parameters. Multiple discrete points are taken within the overlapping range and mapped to the first camera's camera coordinate system according to the first camera's intrinsic, extrinsic, and distortion parameters. These discrete points are then projected onto the 2D image captured by the first camera to determine multiple projection points. These projection points are then fitted to determine the overlapping area in the 2D image captured by the first camera. Similarly, multiple discrete points are mapped to the second camera's camera coordinate system according to the second camera's intrinsic, extrinsic, and distortion parameters and then projected onto the 2D image captured by the second camera to determine multiple projection points. These projection points are then fitted to determine the overlapping area in the 2D image captured by the second camera.

[0032] In this embodiment, the color balance coefficient of the reference image can be set as needed; for example, the color balance coefficient of the reference image can be set to 1.

[0033] Step 102: Process the color balance coefficient of the reference image, the average color value of the overlapping area in the reference image, and the average color value of the overlapping area in the target image according to the preset relationship to determine the color balance coefficient of the target image.

[0034] In this embodiment, any image from multiple images is selected as a reference image, and a color balance coefficient is set for the reference image. For the reference image and an adjacent target image, the color balance coefficient of the target image is determined with the color balance of the overlapping areas in the reference image and the overlapping areas in the target image as the objective. Optionally, the color balance coefficient of the reference image is set to 1. After determining the color balance coefficient of a target image, the target image can be further used as a reference image to determine the color balance coefficient of another image adjacent to the target image. Thus, the color balance coefficients of adjacent images are determined sequentially according to a specified order, and the color balance coefficient of each image in multiple images can be determined.

[0035] Specifically, the first color value of each pixel in the overlapping region of the reference image is determined, wherein the average value of the first color value is the average color value of the overlapping region in the reference image, including the average value of the R channel, the average value of the G channel, and the average value of the B channel; and the second color value of each pixel in the overlapping region of the target image is determined, wherein the average value of the second color value is the average color value of the overlapping region in the target image, including the average value of the R channel, the average value of the G channel, and the average value of the B channel.

[0036] In this embodiment, for each overlapping region, the preset relationship includes: the product of the color equalization coefficient of the reference image and the average color value of the overlapping region in the reference image for each of the RGB three channels is equivalent to the product of the color equalization coefficient of the target image and the average color value of the overlapping region in the target image. Specifically, the product of the R-channel equalization coefficient of the reference image and the average R-channel value of the overlapping region in the reference image is equivalent to the product of the R-channel equalization coefficient of the target image and the average R-channel value of the overlapping region in the target image; the product of the G-channel equalization coefficient of the reference image and the average G-channel value of the overlapping region in the reference image is equivalent to the product of the G-channel equalization coefficient of the target image and the average G-channel value of the overlapping region in the target image; and the product of the B-channel equalization coefficient of the reference image and the average B-channel value of the overlapping region in the reference image is equivalent to the product of the B-channel equalization coefficient of the target image and the average B-channel value of the overlapping region in the target image.

[0037] The following example uses two adjacent images to illustrate how to determine the color equalization coefficient. As an example, consider sequentially adjacent images a, b, and c, where image b is the reference image. Taking the target image c and the left-adjacent reference image b as examples, the R-channel equalization coefficient of the reference image is 1, the average R-channel value of the overlapping region in the reference image is RF1, and the average R-channel value of the overlapping region in the target image is RL2. According to the formula RF1 = a... RL ×RL2 determines the R-channel equalization coefficient a of the target image. RLTaking the target image a and the adjacent reference image b on the right as an example, the R-channel equalization coefficient of the reference image is 1, the average R-channel value of the overlapping region in the reference image is RF2, and the average R-channel value of the overlapping region in the target image is RR1. According to the formula RF2 = a RR ×RR1 determines the R-channel equalization coefficient a of the target image. RR .

[0038] In one embodiment of this disclosure, the first camera and the second camera are adjacent. For a target image captured by the first camera, when its corresponding reference images are two reference images captured by the two second cameras respectively, a first color balance coefficient is determined based on the color balance coefficient of one reference image, the average color value of the overlapping area in one reference image, and the average color value of the overlapping area in the target image. A second color balance coefficient is determined based on the color balance coefficient of the other reference image, the average color value of the overlapping area in the other reference image, and the average color value of the overlapping area in the target image. If the first color balance coefficient and the second color balance coefficient satisfy a preset adjustment condition, then the color balance coefficient of the target image is determined based on the first color balance coefficient and the second color balance coefficient. Thus, by determining to perform color adjustment on each image when the first color balance coefficient and the second color balance coefficient satisfy the preset adjustment condition, compared with methods based on histograms, the color adjustment effect can be further improved when the sampling data is small.

[0039] As an example, images a, b, c, and d are stitched together end to end to form a circular image, where image b is a reference image. The color balance coefficients of images a and c are determined respectively. Then, image d is used as the target image, and images a and c are used as reference images. The first color balance coefficient of image d is determined based on the color balance coefficient of image a, and the second color balance coefficient of image d is determined based on the color balance coefficient of image c.

[0040] In this embodiment, adjustment conditions are preset. When the adjustment conditions are met, color adjustment is determined based on the color balance coefficient of each current image. The target image has a first color balance coefficient and a second color balance coefficient. The color balance coefficient of each channel of the target image used for color adjustment is determined based on the first color balance coefficient and the second color balance coefficient of each channel. The implementation methods for determining the color balance coefficient of the target image based on the first color balance coefficient and the second color balance coefficient include, but are not limited to, averaging, weighted averaging, etc.

[0041] There are various adjustment conditions, as explained below.

[0042] In one embodiment of this disclosure, detecting that the first color balance coefficient and the second color balance coefficient meet preset adjustment conditions includes: determining that the adjustment conditions are met when the ratio of the first color balance coefficient and the second color balance coefficient is within a preset range.

[0043] As an example, the first color equalization coefficient includes the first R channel equalization coefficient, the first G channel equalization coefficient, and the first B channel equalization coefficient. The second color equalization coefficient includes the second R channel equalization coefficient, the second G channel equalization coefficient, and the second B channel equalization coefficient. When the ratio of the first R channel equalization coefficient to the second R channel equalization coefficient is within a preset range, the ratio of the first G channel equalization coefficient to the second G channel equalization coefficient is within a preset range, and the ratio of the first B channel equalization coefficient to the second B channel equalization coefficient is within a preset range, the adjustment condition is determined to be met. Optionally, in this example, the preset range is (0.8, 1.25), that is, when the ratio of the first color equalization coefficient to the second color equalization coefficient is greater than 0.8 and less than 1.25, the adjustment condition is determined to be met. It should be noted that the above preset range is only an example, and the preset range can also be adjusted according to the actual application needs; no specific restrictions are imposed here.

[0044] In one embodiment of this disclosure, detecting that the first color balance coefficient and the second color balance coefficient meet preset adjustment conditions includes: determining that the adjustment conditions are met when the difference between the first color balance coefficient and the second color balance coefficient is within a preset range.

[0045] As an example, taking two reference images captured by two second cameras, the first color equalization coefficient of each channel of the target image is 1. The second color equalization coefficient includes the second R channel equalization coefficient k1, the second G channel equalization coefficient k2, and the second B channel equalization coefficient k3. When it is detected that k1-1, k2-1, and k3-1 are all within a preset range, the adjustment condition is determined to be met. Optionally, in this example, the preset range is (-0.2, 0.2), that is, when the difference between the first color equalization coefficient and the second color equalization coefficient is less than 0.2, the adjustment condition is determined to be met.

[0046] Step 103: Adjust the color of the target image according to the color balance coefficient of the target image.

[0047] In this embodiment, the color equalization coefficient includes R-channel equalization coefficient, G-channel equalization coefficient, and B-channel equalization coefficient. By determining the R-channel equalization coefficient, G-channel equalization coefficient, and B-channel equalization coefficient for each image, for each image, the R-channel value of each pixel in the image is adjusted according to the R-channel equalization coefficient, including: multiplying the R-channel value of each pixel by the R-channel equalization coefficient of the image, and using the product as the R-channel value of the pixel; and adjusting the G-channel value of each pixel in the image according to the G-channel equalization coefficient, including: multiplying the G-channel value of each pixel by the G-channel equalization coefficient of the image, and using the product as the G-channel value of the pixel; and adjusting the B-channel value of each pixel in the image according to the B-channel equalization coefficient, including: multiplying the B-channel value of each pixel by the B-channel equalization coefficient of the image, and using the product as the B-channel value of the pixel.

[0048] In one embodiment of this disclosure, taking preset adjustment conditions as an example, color adjustment can be performed according to a preset adjustment strategy if the adjustment conditions are not met. As an example, if it is detected that the first color balance coefficient and the second color balance coefficient do not meet the preset adjustment conditions, then the historical color balance coefficients are obtained to adjust the color of each image. The historical color balance coefficients were determined when the preset adjustment conditions were previously met. In this example, panoramic images are acquired at regular intervals. Multiple images are captured by multiple cameras. Within each interval, the steps of determining the color balance coefficients and determining whether the adjustment conditions are met are performed based on the current multiple images. If the adjustment conditions are met, the color of each image is adjusted according to its color balance coefficient, and the current color balance coefficient is recorded. If the adjustment conditions are not met, the historical color balance coefficients are obtained, and the color of each image is adjusted according to these historical color balance coefficients.

[0049] According to the technical solution of this disclosure, a target image and adjacent reference images are obtained from multiple images. Based on the color balance coefficient of the reference image, the average color value of the overlapping areas in the reference image, and the average color value of the overlapping areas in the target image, a color balance coefficient of the target image is determined with the goal of color balance between the overlapping areas in the reference image and the overlapping areas in the target image. Each image is then color-adjusted according to its color balance coefficient. This achieves color balance processing of the images by adjusting the colors of each image using the determined color balance coefficient, reducing color differences at the seams in the stitched panoramic images. Compared to finding the seams and performing fusion processing near them, this method reduces color differences at the seams while ensuring visual continuity in the dynamic image. Furthermore, each time the images are stitched, the current color balance coefficient is determined based on the average color value of the overlapping areas of each image. Compared to methods based on histograms, this method can adjust the colors of each image based on the current color balance coefficient when there is limited sample data, ensuring the color adjustment effect of the panoramic images at different times and reducing color differences at the seams in the stitched panoramic images.

[0050] Based on the above embodiments, the following description uses an in-vehicle surround view system as an example. In this scenario, the surround view image is generated by stitching together four images, including a first image, a second image, a third image, and a fourth image. The four images are captured by four in-vehicle cameras. For example, the first image is captured by the front in-vehicle camera, the second image is captured by the left in-vehicle camera, the third image is captured by the right in-vehicle camera, and the fourth image is captured by the rear in-vehicle camera. The surround view image is generated by stitching together the first image, the second image, the third image, and the fourth image.

[0051] Figure 2 This is a schematic flowchart of another color adjustment method for a panoramic image provided in an embodiment of this disclosure, as shown below. Figure 2 As shown, the method includes:

[0052] Step 201: Use the first image captured by the vehicle's front camera as a reference image and determine the color balance coefficient of the first image.

[0053] In this embodiment, the surround view image is generated by stitching together four images. Optionally, the first image captured by the vehicle's front camera is used as the reference image, and the color balance coefficient of the first image can be set to 1. The vehicle camera can be a fisheye camera. As an example, Figure 3 The diagram shows a top view of a vehicle surround view system, in which four vehicle cameras are respectively set around the vehicle. In the figure, mark 1 represents the front vehicle camera, mark 2 represents the left vehicle camera, mark 3 represents the right vehicle camera, and mark 4 represents the rear vehicle camera. The dashed lines are examples of the camera's field of view.

[0054] Among them, there is an overlapping area between the first image and the second image, an overlapping area between the first image and the third image, an overlapping area between the second image and the fourth image, and an overlapping area between the third image and the fourth image. The following uses the first and second images as examples to illustrate the determination of the overlapping area. For instance, based on the camera calibration data of the vehicle's front camera and left camera, the overlapping range of the vehicle's front camera and left camera in the world coordinate system can be determined. Four outer edge lines Li of a rectangle are selected on the ground within the overlapping range. Multiple discrete points are selected for each outer edge line, for example, 10 discrete points can be selected. The three-dimensional coordinates P of the discrete points in the world coordinate system are denoted as (Xi, Yi, Zi). Furthermore, based on the extrinsic, intrinsic, and distortion parameters of the vehicle's front camera and left camera, the projection point p of P on the two-dimensional image can be determined. The image coordinates of p are denoted as (xi, yi). Li corresponds to a line fi composed of multiple discrete points in the first image acquired by the vehicle's front camera, and to a line li composed of multiple discrete points in the second image acquired by the vehicle's left camera. Curve fitting is performed on line fi to obtain curve AF1, which forms the overlapping area in the first image. Curve fitting is performed on line li to obtain curve AL2, which forms the overlapping area in the second image.

[0055] Step 202: Based on the color balance coefficient of the first image and the average color value of the overlapping areas in each image, determine the first color balance coefficient and the second color balance coefficient of the fourth image in the panoramic image, as well as the color balance coefficients of the second and third images.

[0056] Optionally, the color balance coefficient of the second image is determined based on the color balance coefficient of the first image, the average color value of the overlapping region in the first image, and the average color value of the overlapping region in the second image, wherein there is an overlapping region between the first image and the second image. For example, taking the R channel as an example, RF1 = a RL ×RL2 determines the R-channel equalization coefficient a of the second image. RL Where RF1 is the average R-channel value of overlapping region A in the first image, and RL2 is the average R-channel value of overlapping region B in the second image. Furthermore, based on the color equalization coefficient of the second image, the average color value of the overlapping region in the second image, and the average color value of the overlapping region in the fourth image, a first color equalization coefficient for the fourth image is determined. This is because there is an overlapping region between the second and fourth images; for example, using a... RB1 ×RB2=a RL ×RL1 determines the equalization coefficient a for the first R channel of the fourth image. RB1Where RL1 is the average R-channel value of the overlapping region C in the second image, and RB2 is the average R-channel value of the overlapping region D in the fourth image. The implementation methods of the G-channel equalization coefficients and B-channel equalization coefficients of the second / fourth images can refer to the R-channel methods, and will not be elaborated here.

[0057] Optionally, the color balance coefficient of the third image is determined based on the color balance coefficient of the first image, the average color value of the overlapping region in the first image, and the average color value of the overlapping region in the third image, wherein there is an overlapping region between the first image and the third image. For example, taking the R channel as an example, RF2 = a RR ×RR1 determines the R-channel equalization coefficient a of the third image. RR Where RF2 is the average R-channel value of overlapping region a in the first image, and RR1 is the average R-channel value of overlapping region b in the third image. Furthermore, based on the color equalization coefficient of the third image, the average color value of the overlapping region in the third image, and the average color value of the overlapping region in the fourth image, a second color equalization coefficient for the fourth image is determined. This is because there is an overlapping region between the third and fourth images; for example, using a... RB2 ×RB1=a RR ×RR2 determines the equalization coefficient a for the second R channel of the fourth image. RB2 Where RR2 is the average R-channel value of the overlapping region c in the third image, and RB1 is the average R-channel value of the overlapping region d in the fourth image. The implementation methods of the G-channel equalization coefficients and B-channel equalization coefficients of the third / fourth images can refer to the R-channel method, and will not be elaborated here.

[0058] Step 203: If the detection shows that the first color balance coefficient and the second color balance coefficient meet the preset adjustment conditions, then the color balance coefficient of the fourth image is determined according to the first color balance coefficient and the second color balance coefficient, and the color of each image is adjusted according to the color balance coefficient of each of the four images.

[0059] In this embodiment, the adjustment condition is determined to be met when the ratio of the equalization coefficient of the first R channel and the equalization coefficient of the second R channel are within a preset range, the ratio of the equalization coefficient of the first G channel and the equalization coefficient of the second G channel are within a preset range, and the ratio of the equalization coefficient of the first B channel and the equalization coefficient of the second B channel are within a preset range. Optionally, the preset range in this example is (0.8, 1.25).

[0060] As an example, the R-channel equalization coefficients for the first, second, third, and fourth images are 1 and a, respectively. RL a RR a RB The equalization coefficient of the G channel is 1, a GL a GR aGB The equalization coefficient for channel B is 1, a BL a BR a BB , where a RB a is the average of the equalization coefficients of the first R channel and the second R channel. GB a is the average of the equalization coefficients of the first G channel and the second G channel. BB This is the average of the first B-channel equalization coefficient and the second B-channel equalization coefficient. The R-channel value of each pixel in each image is multiplied by the R-channel equalization coefficient of that image, and the G-channel value of each pixel in each image is multiplied by the G-channel equalization coefficient of that image, and the B-channel value of each pixel in each image is multiplied by the B-channel equalization coefficient of that image.

[0061] It should be noted that this embodiment uses four cameras to realize a panoramic image as an example. The panoramic image can be composed of any number of multiple images, such as five or six, and there is no specific limitation here.

[0062] In this embodiment of the disclosure, due to the different installation positions and orientations of the cameras in the vehicle surround view system, the colors of multiple images differ in the overlapping area, resulting in obvious stitching seams in the stitched surround view images. Therefore, by performing color equalization processing on the four images of the vehicle surround view system, the color difference at the seams in the stitched surround view images is reduced, thereby eliminating the stitching seams in the surround view images.

[0063] This disclosure also proposes a color adjustment device for a panoramic image.

[0064] Figure 4 This is a schematic diagram of the structure of a color adjustment device for a panoramic image provided in an embodiment of this disclosure, as shown below. Figure 4 As shown, the color adjustment device for the panoramic image includes: an acquisition module 41, a determination module 42, and an adjustment module 43.

[0065] The acquisition module 41 is used to determine a target image and a reference image adjacent to the target image from the plurality of images, wherein there is an overlapping area between the target image and the reference image.

[0066] The determining module 42 is used to process the color balance coefficient of the reference image, the average color value of the overlapping area in the reference image, and the average color value of the overlapping area in the target image according to a preset relationship, and to determine the color balance coefficient of the target image.

[0067] The adjustment module 43 is used to adjust the color of the target image according to the color balance coefficient of the target image. In one embodiment of this disclosure, the determining module 42 includes: a first determining unit, used to determine a first color balance coefficient by processing the color balance coefficient of one reference image, the average color value of the overlapping area in the one reference image, and the average color value of the overlapping area in the target image according to a preset relationship when there are two reference images; a second determining unit, used to determine a second color balance coefficient by processing the color balance coefficient of another reference image, the average color value of the overlapping area in the other reference image, and the average color value of the overlapping area in the target image according to a preset relationship; and a detection unit, used to determine the color balance coefficient of the target image according to the first color balance coefficient and the second color balance coefficient if the first color balance coefficient and the second color balance coefficient satisfy a preset adjustment condition.

[0068] In one embodiment of this disclosure, the preset relationship includes: the product of the color balance coefficient of the reference image and the average color value of the overlapping area in the reference image for each of the RGB three channels is equivalent to the product of the color balance coefficient of the target image and the average color value of the overlapping area in the target image.

[0069] In one embodiment of this disclosure, the first color equalization coefficient includes a first R-channel equalization coefficient, a first G-channel equalization coefficient, and a first B-channel equalization coefficient, and the second color equalization coefficient includes a second R-channel equalization coefficient, a second G-channel equalization coefficient, and a second B-channel equalization coefficient. The detection unit is specifically configured to: determine that the adjustment condition is met when the ratio of the first R-channel equalization coefficient to the second R-channel equalization coefficient is within a preset range, the ratio of the first G-channel equalization coefficient to the second G-channel equalization coefficient is within the preset range, and the ratio of the first B-channel equalization coefficient to the second B-channel equalization coefficient is within the preset range.

[0070] In one embodiment of this disclosure, the preset range is (0.8, 1.25).

[0071] In one embodiment of this disclosure, the color equalization coefficient includes an R-channel equalization coefficient, a G-channel equalization coefficient, and a B-channel equalization coefficient. The step of adjusting the color of each image based on the color equalization coefficient of each image in the panoramic image includes: adjusting the R-channel value of each pixel in the target image based on the R-channel equalization coefficient of the target image; adjusting the G-channel value of each pixel in the target image based on the G-channel equalization coefficient of the target image; and adjusting the B-channel value of each pixel in the target image based on the B-channel equalization coefficient of the target image.

[0072] In one embodiment of this disclosure, the apparatus further includes: a calling module, configured to, if it is detected that the first color balance coefficient and the second color balance coefficient do not meet a preset adjustment condition, obtain historical color balance coefficients to adjust the color of the target image, wherein the historical color balance coefficients were determined when the preset adjustment condition was met last time.

[0073] In one embodiment of this disclosure, the apparatus further includes: a preprocessing module, configured to acquire camera calibration data corresponding to the reference image and the target image respectively; determine multiple discrete points of the overlapping region in the world coordinate system based on the camera calibration data; map the multiple discrete points onto the reference image and the target image to determine multiple projection points; and fit the multiple projection points to determine the overlapping region.

[0074] In one embodiment of this disclosure, the apparatus further includes: a color determination module, configured to determine a first color value for each pixel in the overlapping region of the reference image, wherein the average value of the first color value is the average color value of the overlapping region in the reference image; and a module configured to determine a second color value for each pixel in the overlapping region of the target image, wherein the average value of the second color value is the average color value of the overlapping region in the target image.

[0075] The color adjustment device for a panoramic image provided in this disclosure can execute the color adjustment method for any panoramic image provided in this disclosure, and has the corresponding functional modules and beneficial effects for executing the method. Content not described in detail in the device embodiments of this disclosure can be referred to the description in any method embodiment of this disclosure.

[0076] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Figure 5 As shown, the electronic device 500 includes one or more processors 501 and memory 502.

[0077] The processor 501 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 500 to perform desired functions.

[0078] The memory 502 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 501 may execute the program instructions to implement the methods of the embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0079] In one example, the electronic device 500 may further include an input device 503 and an output device 504, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown). Furthermore, the input device 503 may include, for example, a keyboard, a mouse, etc. The output device 504 may output various information to the outside, including determined distance information, direction information, etc. The output device 504 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0080] Of course, for the sake of simplicity, Figure 5 Only some of the components of the electronic device 500 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 500 may include any other suitable components depending on the specific application.

[0081] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform any of the methods provided in the embodiments of this disclosure.

[0082] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0083] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform any of the methods provided in the embodiments of this disclosure.

[0084] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0085] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0086] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for adjusting the color of a panoramic image, characterized in that, The panoramic image is generated by stitching together multiple images, and the method includes: A target image and a reference image adjacent to the target image are determined from the plurality of images, wherein there is an overlapping region between the target image and the reference image; The color balance coefficient of the reference image, the average color value of the overlapping area in the reference image, and the average color value of the overlapping area in the target image are processed according to a preset relationship to determine the color balance coefficient of the target image. The color balance coefficient includes the R channel balance coefficient, the G channel balance coefficient, and the B channel balance coefficient. The preset relationship includes: the product of the color balance coefficient of the reference image and the average color value of the overlapping area in the reference image for each of the RGB three channels is equal to the product of the color balance coefficient of the target image and the average color value of the overlapping area in the target image. Based on the R-channel equalization coefficient, G-channel equalization coefficient and B-channel equalization coefficient of the target image, the R-channel value, G-channel value and B-channel value of each pixel in the target image are adjusted for color. Wherein, the plurality of images includes at least three images, and when there are two reference images, the color balance coefficient of the target image is determined by the following steps: The first color balance coefficient and the second color balance coefficient are determined according to a preset relationship; wherein, the first color balance coefficient is determined based on a reference image, and the second color balance coefficient is determined based on another reference image. When the first color balance coefficient and the second color balance coefficient meet the preset adjustment conditions, the color balance coefficient of the target image is determined based on the first color balance coefficient and the second color balance coefficient.

2. The method as described in claim 1, characterized in that, The first color equalization coefficient includes a first R-channel equalization coefficient, a first G-channel equalization coefficient, and a first B-channel equalization coefficient; the second color equalization coefficient includes a second R-channel equalization coefficient, a second G-channel equalization coefficient, and a second B-channel equalization coefficient; the first color equalization coefficient and the second color equalization coefficient satisfy preset adjustment conditions, including: When the ratio of the first R-channel equalization coefficient to the second R-channel equalization coefficient is within a preset range, and the ratio of the first G-channel equalization coefficient to the second G-channel equalization coefficient is within a preset range, and the ratio of the first B-channel equalization coefficient to the second B-channel equalization coefficient is within a preset range, the adjustment condition is determined to be satisfied.

3. The method as described in claim 1, characterized in that, After determining the second color balance coefficient, the following is also included: If the first color balance coefficient and the second color balance coefficient do not meet the preset adjustment conditions, then the target image is color-adjusted by obtaining the historical color balance coefficients, wherein the historical color balance coefficients were determined when the preset adjustment conditions were met last time.

4. The method as described in claim 1, characterized in that, Before processing the color balance coefficient of the reference image, the average color value of the overlapping region in the reference image, and the average color value of the overlapping region in the target image according to a preset relationship to determine the color balance coefficient of the target image, the process further includes: Obtain camera calibration data corresponding to the reference image and the target image respectively, and determine multiple discrete points of the overlapping region in the world coordinate system based on the camera calibration data; The plurality of discrete points are mapped onto the reference image and the target image to determine a plurality of projection points; The overlapping region is determined by fitting the multiple projection points.

5. The method as described in claim 1, characterized in that, Before processing the color balance coefficient of the reference image, the average color value of the overlapping region in the reference image, and the average color value of the overlapping region in the target image according to a preset relationship to determine the color balance coefficient of the target image, the method further includes: Determine the first color value of each pixel in the overlapping region of the reference image, wherein the average value of the first color value is the average color value of the overlapping region of the reference image; Determine the second color value of each pixel in the overlapping region of the target image, wherein the average value of the second color value is the average color value of the overlapping region in the target image.

6. A color adjustment device for a panoramic image, characterized in that, The panoramic image is generated by stitching together multiple images, and the device includes: An acquisition module is used to determine a target image and a reference image adjacent to the target image from the plurality of images, wherein there is an overlapping area between the target image and the reference image; The determining module is used to process the color balance coefficient of the reference image, the average color value of the overlapping area in the reference image, and the average color value of the overlapping area in the target image according to a preset relationship, and to determine the color balance coefficient of the target image; the color balance coefficient includes the R channel balance coefficient, the G channel balance coefficient, and the B channel balance coefficient; the preset relationship includes: the product of the color balance coefficient of the reference image and the average color value of the overlapping area in the reference image under each of the three RGB channels is equal to the product of the color balance coefficient of the target image and the average color value of the overlapping area in the target image; The adjustment module is used to adjust the color of the R-channel value, G-channel value and B-channel value of each pixel in the target image according to the R-channel equalization coefficient, G-channel equalization coefficient and B-channel equalization coefficient of the target image, respectively. Wherein, the plurality of images includes at least three images, and when there are two reference images, the color balance coefficient of the target image is determined by the following steps: The first color balance coefficient and the second color balance coefficient are determined according to a preset relationship; wherein, the first color balance coefficient is determined based on a reference image, and the second color balance coefficient is determined based on another reference image. When the first color balance coefficient and the second color balance coefficient meet the preset adjustment conditions, the color balance coefficient of the target image is determined based on the first color balance coefficient and the second color balance coefficient.

7. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the color adjustment method for the panoramic image as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the color adjustment method for the panoramic image as described in any one of claims 1-5.

9. A vehicle, characterized in that, This includes the color adjustment device for the panoramic image as described in claim 6 or the electronic device as described in claim 7.

Images