A panoramic system chroma adjustment method

Abstract

The application provides a panoramic system chroma adjusting method, which comprises the following steps: acquiring images of each camera; determining overlapping areas of the images of each camera; selecting a first image as a reference image of a panoramic image and a second image as a to-be-stitched image in the overlapping areas; acquiring an initial homography matrix between the reference image and the to-be-stitched image; correcting the initial homography matrix to obtain an optimized homography matrix; stitching the reference image and the to-be-stitched image through a bilinear interpolation algorithm according to the optimized homography matrix to obtain a panoramic stitched image; and judging whether the color balance degree of the panoramic stitched image meets a preset value; if yes, panoramic image stitching is completed; otherwise, the color of the panoramic stitched image is adjusted. The application realizes color balance and uniformity of the panoramic stitched image and improves imaging quality.

Description

Technical Field

[0001] This invention relates to the field of panoramic system technology, and in particular to a method for adjusting the color of a panoramic system. Background Technology

[0002] With the continuous advancement of automotive electronics and computer vision technologies, more and more high-tech intelligent configurations are being widely applied to automobiles, and products assisting drivers in vehicle control are becoming increasingly sophisticated. Previously, reversing radar, a common automotive electronic component, used ultrasonic ranging to determine the distance between objects near the rear of the vehicle and the vehicle body, then alerted the driver via display or audio devices. In contrast, in-vehicle panoramic imaging systems utilize cameras positioned around the vehicle to capture images from various directions, then stitch these images together based on the camera's own parameters and installation parameters to obtain a 360-degree panoramic image of the vehicle's surroundings, greatly facilitating the driver's monitoring of the area around the vehicle.

[0003] However, individual differences in camera sensors, image processors, and lenses can cause color differences between cameras, resulting in significant variations in color intensity and affecting the color balance of the stitched image.

[0004] Therefore, the existing technology needs further improvement. Summary of the Invention

[0005] This invention provides a method for adjusting the color of a panoramic system, aiming to overcome the shortcomings of existing technologies, achieve color balance in panoramic stitched images, and improve imaging quality.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] This invention provides a method for adjusting the chromaticity of a panoramic system, comprising:

[0008] Step 1: Acquire images from each camera;

[0009] Step 2: Determine the overlapping areas of the images from each camera;

[0010] Step 3: In each overlapping area, select the first image as the reference image for the panoramic image, and use the second image as the image to be stitched together;

[0011] Step 4: Obtain the initial homography matrix between the reference image and the image to be stitched;

[0012] Step 5: Correct the initial homography matrix to obtain the optimized homography matrix;

[0013] Step 6: Based on the optimized homography matrix, the reference image and the image to be stitched are stitched together using a bilinear interpolation algorithm to obtain a panoramic stitched image;

[0014] Step 7: Determine whether the color balance of the panoramic stitched image meets the preset value. If yes, complete the panoramic image stitching; otherwise, adjust the color of the panoramic stitched image.

[0015] Specifically, step 1 includes:

[0016] Step 101: Configure a preset image output format for the camera system in the panoramic system, and send the preset image output format to the image processor in the panoramic system;

[0017] Step 102: The image processor sets the image receiving configuration parameters according to the preset image output format, and sends the configuration parameters to the camera system;

[0018] Step 103: The camera system receives the configuration parameters and determines whether the configuration parameters match the preset image output format. If they match, the image data transmission is started and the process proceeds to the next step; otherwise, the process returns to step 101.

[0019] Step 104: The image processor receives the image data and processes it according to the configuration parameters.

[0020] Specifically, step 5 includes:

[0021] Step 501: Calculate the normalized cross-correlation objective function value corresponding to the initial homography matrix, and determine whether the normalized cross-correlation objective function value is greater than a preset threshold. If it is, no optimization adjustment is needed; otherwise, proceed to the next step.

[0022] Step 502: Starting from the initial point corresponding to the initial homography matrix, initialize the number of searches and set the initial search step size to ;

[0023] Step 503: Search along the horizontal, vertical, and diagonal directions with the current step size;

[0024] Step 504: Calculate the current normalized cross-correlation objective function value corresponding to the current homography matrix, and determine whether the current normalized cross-correlation objective function value is greater than a preset threshold. If it is, use the current homography matrix as the optimized homography matrix; otherwise, proceed to the next step.

[0025] Step 505: Increment the search count by a preset value, modify the current search step size according to a preset formula, and return to step 503.

[0026] Specifically, the preset formula is: αc'=(s+1)αc, where αc represents the current search step size, αc' represents the modified search step size, and s represents the current search number.

[0027] Specifically, step 7 includes:

[0028] Step 7A1: Set the color difference range of color sub-components in each overlapping area;

[0029] Step 7A2: Obtain the average value of the first color sub-component and the average value of the second color sub-component in each overlapping region;

[0030] Step 7A3: Calculate the absolute value of the difference between each color sub-component based on the average value of the first color sub-component and the average value of the second color sub-component. The absolute value of the difference between each color sub-component is the absolute value of the difference between the average value of the first color sub-component and the average value of the second color sub-component.

[0031] Step 7A4: Determine whether the absolute value of the difference between each color sub-component is within the color difference range. If yes, no adjustment is needed; otherwise, proceed to the next step.

[0032] Step 7A5: Receive the gain values ​​of each color sub-component;

[0033] Step 7A6: Adjust each color component according to the gain value of each color sub-component.

[0034] Specifically, step 7A2 includes:

[0035] Step A201: Obtain the first color sub-component value and the second color sub-component value within each overlapping region;

[0036] Step A202: Calculate the average value of the first color sub-component and the average value of the second color sub-component based on the first color sub-component value and the second color sub-component value.

[0037] Specifically, step 7A6 includes:

[0038] X avg (j)'=X avg (j)*X_gain(j), where X avg (j) represents the values ​​of each color sub-component before adjustment, X avg (j)' represents the adjusted values ​​of each color sub-component.

[0039] Specifically, step 7 includes:

[0040] Step 7B1: Determine the reference camera;

[0041] Step 7B2: Select a sampling area from the image captured by the reference camera;

[0042] Step 7B3: Obtain the chromaticity response value of the reference camera;

[0043] Step 7B4: Determine the chromaticity response transformation function of the reference camera based on the chromaticity response value;

[0044] Step 7B5: Calculate the correction function for the camera to be corrected based on the chromaticity response conversion function;

[0045] Step 7B6: The camera to be corrected corrects the RGB values ​​of the captured image according to the correction function.

[0046] Specifically, step 7B3 includes:

[0047] Step B301: Set up several calibration images according to preset rules;

[0048] Step B302: Capture the calibration image using the reference camera, sample the R, G, and B single-channel color values ​​at the same point in the sampling area, and calculate the chromaticity response value of the reference camera.

[0049] Specifically, the preset rule is that the R, G, and B values ​​of each calibration image increase in a fixed step size.

[0050] The beneficial effects of this invention are as follows: This invention obtains the overlapping areas of images from various cameras, selects one image as the reference image for the panoramic image, and the other image as the image to be stitched. The optimized homography matrix between the reference image and the image to be stitched is then used to stitch the images together to obtain a panoramic stitched image. When the color balance of the panoramic stitched image does not meet the preset value, the color of the panoramic stitched image is adjusted, thereby achieving color balance in the panoramic stitched image and improving the imaging quality. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the panoramic system color adjustment method of the present invention;

[0052] Figure 2 This is a schematic diagram of the panoramic system image stitching of the present invention. Detailed Implementation

[0053] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. The drawings are for reference and illustration only and do not constitute a limitation on the scope of protection of the present invention.

[0054] Example 1

[0055] The panoramic system in this embodiment includes four cameras (front, rear, left, and right) and an image processor. The image processor is used to process and stitch together the images captured by the four cameras to obtain a panoramic image.

[0056] like Figure 1 As shown, this embodiment provides a method for adjusting the brightness of a panoramic system, including:

[0057] Step 1: Acquire images from each camera.

[0058] In practice, the images captured by each camera are sent to the image processor in the panoramic system for processing.

[0059] Step 2: Determine the overlapping areas of the images from each camera.

[0060] In this embodiment, the overlapping region is denoted as Zq. The overlapping region Zq contains two overlapping camera images, denoted as the first image F(q1) and the second image F(q2), respectively, where q = 1, 2, 3, ... k, and k represents the number of overlapping regions. In this embodiment, k = 4.

[0061] like Figure 2 As shown, the panoramic system image in this embodiment includes overlapping regions Z1, Z2, Z3, and Z4 in a clockwise order.

[0062] In this embodiment, the direction of the front of the vehicle is considered forward, the direction of the rear of the vehicle is considered rear, the direction of the left rearview mirror is considered left, and the direction of the right rearview mirror is considered right.

[0063] The overlapping area Z1 represents the overlapping area of ​​the field of view of the rear camera and the left camera, which consists of the left field of view image F(11) of the rear camera and the rear field of view image F(12) of the left camera.

[0064] The overlapping area Z2 represents the overlapping area of ​​the fields of view of the left camera and the front camera, which consists of the front field of view image F(21) of the left camera and the left field of view image F(22) of the front camera.

[0065] The overlapping area Z3 represents the overlapping area of ​​the field of view of the front camera and the right camera, which consists of the right field of view image F(31) of the front camera and the front field of view image F(32) of the right camera.

[0066] The overlapping region Z4 represents the overlapping area of ​​the field of view of the right camera and the rear camera, which consists of the rear field of view image F(41) of the right camera and the right field of view image F(42) of the rear camera.

[0067] Step 3: Within each overlapping region Zq, select the first image F(q1) as the reference image for the panoramic image, and use the second image F(q2) as the image to be stitched together.

[0068] It is easy to understand that the second image F(q2) can also be selected as the reference image for the panoramic image, and the first image F(q1) can be used as the image to be stitched together.

[0069] Step 4: Obtain the initial homography matrix H between the reference image and the image to be stitched.

[0070] In this embodiment, the positions of each camera in the panoramic system are fixed, so the initial homography matrix H can be obtained through calibration.

[0071] Step 5: Correct the initial homography matrix H to obtain the optimized homography matrix H'.

[0072] During driving, the vehicle's vibrations can cause changes in the camera's position, resulting in noticeable seams in the images stitched together using the initial homography matrix. This affects the quality of the stitched images, so corrections and optimizations are needed based on the vehicle's vibrations.

[0073] Step 6: Based on the optimized homography matrix H', the reference image and the image to be stitched are stitched together using a bilinear interpolation algorithm to obtain a panoramic stitched image.

[0074] Since the coordinates of an image may be decimals after image transformation, interpolation algorithms are needed to determine which position to assign the pixel to. Bilinear interpolation uses the gray levels of the four neighboring pixels of the pixel to be identified to perform linear interpolation in two directions, ultimately making the image appear smoother.

[0075] Step 7: Determine whether the color balance of the panoramic stitched image meets the preset value. If yes, complete the panoramic image stitching; otherwise, adjust the color of the panoramic stitched image.

[0076] For panoramic stitched images under the same lighting conditions, their colors should be uniform. Therefore, it is necessary to detect the color balance of the panoramic stitched image to determine whether to adjust the colors.

[0077] Example 2

[0078] This embodiment provides a specific implementation method for step 1 in embodiment 1, including:

[0079] Step 101: Configure a preset image output format for the camera system in the panoramic system, and send the preset image output format to the image processor in the panoramic system.

[0080] The preset image output format is MIPI.

[0081] Step 102: The image processor sets the image receiving configuration parameters according to the preset image output format and sends the configuration parameters to the camera system.

[0082] In this embodiment, if the preset image output format is MIPI, the configuration parameters include: MIPI device attribute parameters (e.g., packet size, line synchronization, number of frame synchronization pixels, etc.), data output type (e.g., RGB16, RGB24 or YUV420, etc.), MIPI mode (e.g., CSI, DSI, CMD, etc.), MIPI channel (LANE) configuration, etc.

[0083] Step 103: The camera system receives the configuration parameters and determines whether the configuration parameters match the preset image output format. If they match, the image data transmission is started and the process proceeds to the next step; otherwise, it returns to step 101.

[0084] Step 104: The image processor receives the image data and processes it according to the configuration parameters.

[0085] Example 3

[0086] This embodiment provides a specific implementation method for step 5 in embodiment 1, including:

[0087] Step 501: Calculate the normalized cross-correlation objective function value F(H) corresponding to the initial homography matrix H, and determine whether the normalized cross-correlation objective function value F(H) is greater than the preset threshold T. If yes, no optimization adjustment is needed; otherwise, proceed to the next step.

[0088] Step 502: Starting from the initial point corresponding to the initial homography matrix H, initialize the search count s and set the initial search step size to α0.

[0089] Initialize the search count s, i.e., set s = 0.

[0090] Step 503: Search along the horizontal, vertical, and diagonal directions with the current step size.

[0091] Step 504: Calculate the current normalized cross-correlation objective function value F(Hc) corresponding to the current homography matrix Hc, and determine whether the current normalized cross-correlation objective function value F(Hc) is greater than a preset threshold T. If yes, use the current homography matrix Hc as the optimized homography matrix H'; otherwise, proceed to the next step.

[0092] Step 505: Increment the search count s by a preset value, modify the current search step size according to a preset formula, and return to step 503.

[0093] In this embodiment, the preset value is 1.

[0094] The preset formula is: αc'=(s+1)αc, where αc represents the current search step size, αc' represents the modified search step size, and s represents the current search number.

[0095] For example, when s = 0, αc = α0; when s = 1, αc' = (1+1)αc = 2α0; when s = 2, αc' = (2+1)αc = 3*2α0 = 6α0.

[0096] Example 4

[0097] This embodiment provides a specific implementation method for step 7 in embodiment 1, including:

[0098] Step 7A1: Set the color difference range △X for the color sub-components within each overlapping region Zq. set (q).

[0099] In this embodiment, the color sub-components include R component, G component, and B component, i.e., X = {R, G, B}, or ΔX. set (q) includes:

[0100] △R set (q)=(△R(q)min,△R(q)max),

[0101] △G set (q)=(△G(q)min,△G(q)max),

[0102] △B set (q)=(△B(q)min,△B(q)max).

[0103] Step 7A2: Obtain the average value X of the first color sub-component within each overlapping region Zq. avg (q1), Average value of the second color subcomponent X avg (q2).

[0104] In this embodiment, step 7A2 includes:

[0105] Step A201: Obtain the first color sub-component value X(q1) within each overlapping region Zq. n The second color sub-component value X(q2) n .

[0106] The first color sub-component value X(q1) n Let R, G, and B values ​​be the values ​​of each pixel in the current frame of the first image F(q1), and X(q2) be the value of the second color sub-component. n Let R, G, and B values ​​be the values ​​of each pixel in the current frame of the second image F(q2), and n represent the pixel number.

[0107] Step A202: Based on the first color sub-component value X(q1) n The second color sub-component value X(q2) n Calculate the average value X of the first color sub-component.avg (q1), Average value of the second color subcomponent X avg (q2).

[0108]

[0109]

[0110] Step 7A3: Based on the average value X of the first color sub-component avg (q1), Average value of the second color subcomponent X avg (q2) Calculate the absolute value of the difference between each color sub-component ΔX(q), where the absolute value of the difference between each color sub-component ΔX(q) is the average value X of the first color sub-component. avg (q1), Average value of the second color subcomponent X avg The absolute value of the difference between (q2).

[0111] That is: △X(q)=|X avg (q1)-X avg (q2)|.

[0112] Step 7A4: Determine whether the absolute value of the color sub-component difference ΔX(q) is within the color difference range ΔX. set If (q) is correct, no adjustment is needed; otherwise, proceed to the next step.

[0113] Step 7A5: Receive the gain value X_gain(j) of each color sub-component.

[0114] When one or more of △X(q) exceed the preset color range, the system prompts that the corresponding camera image F(j1) or F(j2) has a color deviation and needs adjustment. A dialog box pops up for the user to choose whether to make the adjustment. If the user closes the dialog box or ignores the system color adjustment, the corresponding camera color will not be adjusted. If the user chooses to adjust, the system will enter the corresponding camera's R_gain(j), B_Gain(j), and G_Gain(j) gain value adjustment interface. The user can manually adjust the R_gain(j), B_Gain(j), and G_Gain(j) values ​​to make the image color output by the camera to be adjusted consistent with other cameras.

[0115] Step 7A6: Adjust each color component according to the gain value X_gain(j) of each color sub-component.

[0116] In this embodiment, step 7A6 includes:

[0117] X avg (j)'=X avg (j)*X_gain(j), where X avg(j) represents the values ​​of each color sub-component before adjustment, X avg (j)' represents the adjusted values ​​of each color sub-component.

[0118] Example 5

[0119] This embodiment provides another specific implementation method for step 7 in embodiment 1, including:

[0120] Step 7B1: Determine the reference camera Cr.

[0121] In practical implementation, a camera with normal image color display (such as the left camera) can be selected as the reference camera Cr.

[0122] Step 7B2: Select a sampling area S from the image captured by the reference camera Cr.

[0123] In a specific embodiment, the selected sampling area S is a rectangular area.

[0124] Step 7B3: Obtain the chromaticity response value Chr of the reference camera Cr.

[0125] In this embodiment, step 7B3 includes:

[0126] Step B301: Set up several calibration images according to preset rules;

[0127] The preset rule is that the R, G, and B values ​​of each calibration image increase in a fixed step size.

[0128] For example, the RGB value of calibration screen 1 is denoted as RGB(1) = (0,0,0), and with a step size of 16, the RGB value of calibration screen 2 is denoted as RGB(2) = (16,16,16), ... and the RGB value of calibration screen 15 is denoted as RGB(15) = (255,255,255).

[0129] Step B302: Capture the calibration image using the reference camera Cr, sample the color values ​​of the R, G, and B channels at the same point in the sampling area S, and calculate the chromaticity response value Chr of the reference camera Cr.

[0130] Step 7B4: Determine the chromaticity response transformation function Fr of the reference camera Cr based on the chromaticity response value Chr.

[0131] Step 7B5: Calculate the correction function Hk of the camera Ck to be corrected based on the chromaticity response conversion function Fr.

[0132] The correction function Hk satisfies the following condition: the Euclidean distance between the value of each pixel in the image captured by the camera Ck to be corrected and the chromaticity response value Chr is the shortest.

[0133] Step 7B6: The camera Ck to be corrected corrects the RGB values ​​of the captured image according to the correction function Hk.

[0134] The above-disclosed embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for adjusting the chromaticity of a panoramic system, characterized in that, include: Step 1: Acquire images from each camera; Step 2: Determine the overlapping areas of the images from each camera; Step 3: In each overlapping area, select the first image as the reference image for the panoramic image, and use the second image as the image to be stitched together; Step 4: Obtain the initial homography matrix between the reference image and the image to be stitched; Step 5: Correct the initial homography matrix to obtain the optimized homography matrix; Step 6: Based on the optimized homography matrix, the reference image and the image to be stitched are stitched together using a bilinear interpolation algorithm to obtain a panoramic stitched image; Step 7: Determine whether the color balance of the panoramic stitched image meets the preset value. If yes, complete the panoramic image stitching; otherwise, adjust the color of the panoramic stitched image. Step 5 includes: Step 501: Calculate the normalized cross-correlation objective function value corresponding to the initial homography matrix, and determine whether the normalized cross-correlation objective function value is greater than a preset threshold. If it is, no optimization adjustment is needed; otherwise, proceed to the next step. Step 502: Starting from the initial point corresponding to the initial homography matrix, initialize the number of searches and set the initial search step size to ; Step 503: Search along the horizontal, vertical, and diagonal directions with the current step size; Step 504: Calculate the current normalized cross-correlation objective function value corresponding to the current homography matrix, and determine whether the current normalized cross-correlation objective function value is greater than a preset threshold. If it is, use the current homography matrix as the optimized homography matrix; otherwise, proceed to the next step. Step 505: Increment the search count by a preset value, modify the current search step size according to a preset formula, and return to step 503.

2. The panoramic system chromaticity adjustment method according to claim 1, characterized in that, Step 1 includes: Step 101: Configure a preset image output format for the camera system in the panoramic system, and send the preset image output format to the image processor in the panoramic system; Step 102: The image processor sets the image receiving configuration parameters according to the preset image output format, and sends the configuration parameters to the camera system; Step 103: The camera system receives the configuration parameters and determines whether the configuration parameters match the preset image output format. If they match, the image data transmission is started and the process proceeds to the next step; otherwise, the process returns to step 101. Step 104: The image processor receives the image data and processes it according to the configuration parameters.

3. The panoramic system chromaticity adjustment method according to claim 1, characterized in that, The preset formula is: αc'=(s+1)αc, where αc represents the current search step size, αc' represents the modified search step size, and s represents the current search number.

4. The panoramic system chromaticity adjustment method according to claim 1, characterized in that, Step 7 includes: Step 7A1: Set the color difference range of color sub-components in each overlapping area; Step 7A2: Obtain the average value of the first color sub-component and the average value of the second color sub-component in each overlapping region; Step 7A3: Calculate the absolute value of the difference between each color sub-component based on the average value of the first color sub-component and the average value of the second color sub-component. The absolute value of the difference between each color sub-component is the absolute value of the difference between the average value of the first color sub-component and the average value of the second color sub-component. Step 7A4: Determine whether the absolute value of the difference between each color sub-component is within the color difference range. If yes, no adjustment is needed; otherwise, proceed to the next step. Step 7A5: Receive the gain values ​​of each color sub-component; Step 7A6: Adjust each color component according to the gain value of each color sub-component.

5. The panoramic system chromaticity adjustment method according to claim 4, characterized in that, Step 7A2 includes: Step A201: Obtain the first color sub-component value and the second color sub-component value within each overlapping region; Step A202: Calculate the average value of the first color sub-component and the average value of the second color sub-component based on the first color sub-component value and the second color sub-component value.

6. The panoramic system chromaticity adjustment method according to claim 4, characterized in that, Step 7A6 includes: X avg (j)'=X avg (j)*X_gain(j), where X avg (j) represents the values ​​of each color sub-component before adjustment, X avg (j)' represents the adjusted values ​​of each color sub-component, and X_gain(j) represents the gain value of each color sub-component.

7. The panoramic system chromaticity adjustment method according to claim 1, characterized in that, Step 7 includes: Step 7B1: Determine the reference camera; Step 7B2: Select a sampling area from the image captured by the reference camera; Step 7B3: Obtain the chromaticity response value of the reference camera; Step 7B4: Determine the chromaticity response transformation function of the reference camera based on the chromaticity response value; Step 7B5: Calculate the correction function for the camera to be corrected based on the chromaticity response conversion function; Step 7B6: The camera to be corrected corrects the RGB values ​​of the captured image according to the correction function.

8. The panoramic system chromaticity adjustment method according to claim 7, characterized in that, Step 7B3 includes: Step B301: Set up several calibration images according to preset rules; Step B302: Capture the calibration image using the reference camera, sample the R, G, and B single-channel color values ​​at the same point in the sampling area, and calculate the chromaticity response value of the reference camera.

9. The panoramic system chromaticity adjustment method according to claim 8, characterized in that, The preset rule is that the R, G, and B values ​​of each calibration image increase in a fixed step size.

Images