Color calibration system and color calibration method for multi-lens panorama camera

Abstract

A color correction system and method for a multi-lens panoramic camera are provided. The color correction method includes: receiving a stitched image from the multi-lens panoramic camera; detecting the stitched image to obtain a first value corresponding to a first lens and a first color, and a second value corresponding to a second lens and the first color; in response to the first value being greater than the second value, correcting a first gain of the second lens according to the first value to generate a correction result; and outputting the correction result.

Description

Technical Field

[0001] This invention relates to a camera correction technique, and more particularly to a color correction system and method for a multi-lens panoramic camera. Prior Technology

[0002] Cameras are widely used in various fields such as environmental monitoring, road safety, video conferencing, sports broadcasting, and vehicle safety equipment. The wider the field of view of an image, the more comprehensive the information available to the user. To meet the demand for wide-angle images, cameras often use fisheye lenses to increase the field of view. However, traditional fisheye lenses often suffer from significant image distortion, reduced resolution, or low light transmission in the peripheral areas of the captured images. To improve these shortcomings, multi-lens cameras can be used to acquire multiple images corresponding to different field of view, and then stitched together to create a new image. However, to obtain a high-quality stitched image, the chromatic aberration between the lenses cannot be too high. Therefore, how to correct the chromatic aberration between different lenses is one of the important issues in this field. Summary of the Invention

[0003] This invention provides a color correction system and method for a multi-lens panoramic camera, which can reduce the color difference between multiple lenses.

[0004] The present invention discloses a color correction system for a multi-lens panoramic camera, comprising an electronic computing device. The electronic computing device is communicatively connected to the multi-lens panoramic camera, wherein the multi-lens panoramic camera includes a first lens and a second lens, wherein the electronic computing device is configured to perform: receiving a stitched image from the multi-lens panoramic camera; detecting the stitched image to obtain a first value corresponding to the first lens and a first color, and a second value corresponding to the second lens and the first color; in response to the first value being greater than the second value, correcting a first gain of the second lens according to the first value to generate a correction result; and outputting the correction result.

[0005] In one embodiment of the invention, the above-described electronic computing device is configured to further perform: increasing the gain corresponding to the brightness of the second lens to correct the first gain.

[0006] In one embodiment of the present invention, the above-described electronic computing device is configured to further perform: updating a second value according to a first gain; determining whether the difference between the first value and the updated second value is less than a threshold; and generating a correction result according to the first gain in response to the difference being less than the threshold.

[0007] In one embodiment of the present invention, the above-described electronic computing device is configured to further perform: detecting the stitched image to obtain a third value corresponding to the first lens and the second color, and a fourth value corresponding to the second lens and the second color; and correcting the second gain of the second lens on the second color based on the third value to generate a correction result.

[0008] In one embodiment of the present invention, the above-described electronic computing device is configured to further perform: updating a fourth value according to a second gain; determining whether the absolute difference between the third value and the updated fourth value is less than a threshold; and generating a correction result according to the second gain in response to the absolute difference being less than the threshold.

[0009] In one embodiment of the present invention, the aforementioned brightness gain includes a gain corresponding to the first color and a gain corresponding to the second color.

[0010] In one embodiment of the invention, the color correction system further includes a white surface. The white surface is configured to cover the first field of view of the first lens and the second field of view of the second lens.

[0011] In one embodiment of the present invention, the reflectivity of the white surface is less than a threshold.

[0012] In one embodiment of the present invention, the first distance from the first lens to the white surface is equal to the second distance from the second lens to the white surface.

[0013] In one embodiment of the present invention, the color correction system further includes a first light source and a second light source. The first light source is configured to illuminate a first portion of a first surface of a white surface with first uniform light. The second light source is configured to illuminate a second portion of the first surface with second uniform light, wherein the second portion is opposite to the first portion.

[0014] The present invention discloses a color correction method for a multi-lens panoramic camera, wherein the multi-lens panoramic camera includes a first lens and a second lens, and the color correction method includes: receiving a stitched image from the multi-lens panoramic camera; detecting the stitched image to obtain a first value corresponding to the first lens and a first color and a second value corresponding to the second lens and the first color; in response to the first value being greater than the second value, correcting a first gain of the second lens according to the first value to generate a correction result; and outputting the correction result.

[0015] In one embodiment of the present invention, the step of correcting the first gain of the second lens according to the first value to produce a correction result includes: increasing the gain corresponding to the brightness of the second lens to correct the first gain.

[0016] In one embodiment of the present invention, the step of correcting the first gain of the second lens according to the first value to generate a correction result further includes: updating the second value according to the first gain; determining whether the difference between the first value and the updated second value is less than a threshold; and generating a correction result according to the first gain in response to the difference being less than the threshold.

[0017] In one embodiment of the present invention, the above-described color correction method further includes: detecting the stitched image to obtain a third value corresponding to the first lens and the second color, and a fourth value corresponding to the second lens and the second color; and correcting the second gain of the second lens on the second color according to the third value to generate a correction result.

[0018] In one embodiment of the present invention, the step of correcting the second gain of the second lens to the second color according to the third value to generate a correction result includes: updating the fourth value according to the second gain; determining whether the absolute difference between the third value and the updated fourth value is less than a threshold; and generating a correction result according to the second gain in response to the absolute difference being less than the threshold.

[0019] In one embodiment of the present invention, the aforementioned brightness gain includes a gain corresponding to the first color and a gain corresponding to the second color.

[0020] In one embodiment of the present invention, the above-described color correction method further includes: configuring a white surface to cover the first field of view of the first lens and the second field of view of the second lens.

[0021] In one embodiment of the present invention, the reflectivity of the white surface is less than a threshold.

[0022] In one embodiment of the present invention, the first distance from the first lens to the white surface is equal to the second distance from the first lens to the white surface.

[0023] In one embodiment of the present invention, the above-described color correction method further includes: configuring a first light source to illuminate a first portion of a first surface of a white surface with first uniform light; and configuring a second light source to illuminate a second portion of the first surface with second uniform light, wherein the second portion is opposite to the first portion.

[0024] Based on the above, the color correction system of the present invention can minimize the color difference between multiple lenses of a multi-lens panoramic camera. Simple Explanation of the Diagram

[0025] Figure 1 illustrates a schematic diagram of a color correction system for a multi-lens panoramic camera according to an embodiment of the present invention. Figure 2 illustrates a flowchart of color correction for a multi-lens panoramic camera according to an embodiment of the present invention. Figure 3 illustrates a top view of a multi-lens panoramic camera with two lenses according to an embodiment of the present invention. Figure 4 illustrates a rear view of a multi-lens panoramic camera with two lenses according to an embodiment of the present invention. Figure 5 illustrates a side view of a multi-lens panoramic camera with two lenses according to an embodiment of the present invention. Figure 6 illustrates a schematic diagram of the field of view of a multi-lens panoramic camera with two lenses according to an embodiment of the present invention. Figure 7 illustrates a top view of a multi-lens panoramic camera with three lenses according to an embodiment of the present invention. Figure 8 illustrates a schematic diagram of the field of view of a multi-lens panoramic camera with three lenses according to an embodiment of the present invention. Figure 9 illustrates a top view of a multi-lens panoramic camera with four lenses according to an embodiment of the present invention. Figure 10 illustrates a schematic diagram of the field of view of a multi-lens panoramic camera with four lenses according to an embodiment of the present invention. Figure 11 illustrates a flowchart of a color correction method for a multi-lens panoramic camera according to an embodiment of the present invention. Implementation

[0026] To make the contents of this invention more readily apparent, the following specific embodiments are provided as examples on which this invention can indeed be implemented. Furthermore, wherever possible, elements / components / steps using the same reference numerals in the drawings and embodiments represent the same or similar parts.

[0027] Figure 1 illustrates a schematic diagram of a color correction system 10 for a multi-lens panoramic camera according to an embodiment of the present invention. The color correction system 10 may include an electronic processing unit 100. The electronic processing unit 100 may include a processor 110, a storage medium 120, and a transceiver 130. The multi-lens panoramic camera may have N lenses, where N is a positive integer greater than or equal to 2.

[0028] Processor 110 may be, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microcontroller (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field-programmable gate array (FPGA), or other similar elements or combinations thereof. Processor 110 may be coupled to storage medium 120 and transceiver 130, and access and execute multiple modules and various applications stored in storage medium 120.

[0029] Storage medium 120 may be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar element or combination thereof, for storing multiple modules or various applications that can be executed by processor 110.

[0030] Transceiver 130 transmits or receives signals wirelessly or via a wired connection. Transceiver 130 can also perform operations such as low-noise amplification, impedance matching, mixing, up- or down-frequency conversion, filtering, amplification, and similar functions. Processor 110 can communicate with a multi-lens panoramic camera or external electronic device via transceiver 130 to receive or transmit data.

[0031] Figure 2 illustrates a flowchart of color correction for a multi-lens panoramic camera according to an embodiment of the present invention. The flowchart in Figure 2 can be implemented by the color correction system 10 shown in Figure 1.

[0032] In step S201, the processor 110 can adjust the parameters of each lens of the multi-lens panoramic camera to preset values. For example, the processor 110 can transmit instructions to the multi-lens panoramic camera via the transceiver 130 to fix the parameters of each lens of the multi-lens panoramic camera, wherein the parameters may include, but are not limited to, shutter speed or analog gain.

[0033] In step S202, the white surface may be configured to cover the field of view (FoV) of each lens of the multi-lens panoramic camera.

[0034] Taking a multi-lens panoramic camera 500 with two lenses (i.e., lens 510 and lens 520) as an example, FIG3 shows a top view of the multi-lens panoramic camera 500 with two lenses according to an embodiment of the present invention, FIG4 shows a rear view of the multi-lens panoramic camera 500 with two lenses according to an embodiment of the present invention, FIG5 shows a side view of the multi-lens panoramic camera 500 with two lenses according to an embodiment of the present invention, and FIG6 shows a schematic diagram of the field of view of the multi-lens panoramic camera with two lenses according to an embodiment of the present invention.

[0035] In one embodiment, the color correction system 10 may further include a white surface 200. The material of the white surface 200 is, for example, paper or acrylic. The reflectivity of the white surface 200 may be less than a threshold. The white surface 200 may be configured to cover the field of view 610 of lens 510 and the field of view 620 of lens 520. The distance between all lenses of the multi-lens panoramic camera 500 and the white surface 200 may be the same. For example, the distance D1 between lens 510 and the white surface 200 may be equal to the distance D2 between lens 520 and the white surface 200.

[0036] The color correction system 10 may further include a light source 310 and a light source 320. The light source 310 may be configured to illuminate a portion 210 of one side of the white surface 200 with uniform light, and the light source 320 may be configured to illuminate another portion 220 of the white surface 200 with uniform light, for example, opposite to portion 210. For instance, if portion 210 is the upper half of the white surface 200, then portion 220 may be the lower half of said surface.

[0037] Taking a multi-lens panoramic camera 500 with three lenses (i.e., lenses 510, 520 and 530) as an example, FIG7 shows a top view of the multi-lens panoramic camera 500 with three lenses according to an embodiment of the present invention, and FIG8 shows a schematic diagram of the field of view of the multi-lens panoramic camera 500 with three lenses according to an embodiment of the present invention.

[0038] The white surface 200 can be configured to cover the field of view 610 of lens 510, the field of view 620 of lens 520, and the field of view 630 of lens 530. All lenses of the multi-lens panoramic camera 500 can be at the same distance from the white surface 200. For example, the distance D1 between lens 510 and the white surface 200 can be equal to the distance D2 between lens 520 and the white surface 200, and either distance D1 or distance D2 can be equal to the distance D3 between lens 530 and the white surface 200.

[0039] Taking a multi-lens panoramic camera 500 with four lenses (i.e., lenses 510, 520, 530 and 540) as an example, FIG9 shows a top view of the multi-lens panoramic camera 500 with four lenses according to an embodiment of the present invention, and FIG10 shows a schematic diagram of the field of view of the multi-lens panoramic camera 500 with four lenses according to an embodiment of the present invention.

[0040] The white surface 200 can be configured to cover the field of view 610 of lens 510, the field of view 620 of lens 520, the field of view 630 of lens 530, and the field of view 640 of lens 540. All lenses of the multi-lens panoramic camera 500 can be at the same distance from the white surface 200. For example, the distance D1 between lens 510 and the white surface 200 can be equal to the distance D2 between lens 520 and the white surface 200. Distance D1 or distance D2 can be equal to the distance D3 between lens 530 and the white surface 200. Distance D1, D2, or D3 can be equal to the distance D4 between lens 540 and the white surface 200.

[0041] Returning to Figure 2, in step S203, the processor 110 can select a reference lens from the multiple lenses of the multi-lens panoramic camera 500.

[0042] Specifically, after the white surface 200 is set, the multi-lens panoramic camera 500 can capture multiple images corresponding to the white surface 200 through multiple lenses, and stitch the multiple images to generate a stitched image. The processor 110 can receive the stitched image from the multi-lens panoramic camera 500 through the transceiver 130. The lenses can capture regions of interest in the field of view. Taking Figure 6 as an example, lens 510 can capture an image containing at least a portion of the white surface 200 through region of interest 61, wherein region of interest 61 can be included in the field of view 610. On the other hand, lens 520 can capture an image containing at least a portion of the white surface 200 through region of interest 62, wherein region of interest 61 can be included in the field of view 610.

[0043] In one embodiment, the region of interest 61 (or 62) and the field of view 610 (or 620) may have the same geometric center.

[0044] In one embodiment, the region of interest 61 (or 62) may partially overlap with the intersection of the fields of view 610 and 620.

[0045] After acquiring the stitched image, the processor 110 can detect the stitched image to obtain color values ​​corresponding to lens 510 (or region of interest 61) and the first color, as well as color values ​​corresponding to lens 520 (or region of interest 62) and the first color. Generally, the number of green pixels in a lens is relatively high relative to the total number of pixels, so correcting the green pixels first can speed up the correction process. Accordingly, the processor 110 can define the first color as green. That is, the processor 110 can detect the green color value corresponding to region of interest 61 and the green color value corresponding to region of interest 62 in the stitched image, respectively.

[0046] Next, the processor 110 may select a lens with the largest color value of the first color as a reference lens, wherein the reference lens can be used to correct the gain of other lenses. For example, if the green color value corresponding to lens 510 is greater than the green color value corresponding to lens 520, then the processor 110 may select lens 510 as the reference lens.

[0047] In step S204, the processor 110 may calculate the difference in a first color value between the reference lens and other lenses. For example, the processor 110 may calculate the difference in green color value between lens 510 and lens 520.

[0048] In step S205, the processor 110 determines whether the difference between the first color values ​​is less than a threshold. If the difference is less than the threshold, step S207 is executed. If the difference is greater than or equal to the threshold, step S206 is executed.

[0049] In step S206, the processor 110 can correct the gain of other lenses based on the first color value of the reference lens. Then, the processor 110 can re-execute step S204.

[0050] In one embodiment, the processor 110 can correct the gain of other lenses by increasing the gain corresponding to the luminance of the other lenses. The conversion of luminance to RGB values ​​is shown in Equation (1), where Y is the luminance gain, R is the red color gain, G is the green color gain, and B is the blue color gain. When the luminance of a lens increases, the gains of the green, red, and blue colors of the lens also increase. Y = 0.299*R + 0.587*G + 0.114*B (1)

[0051] For example, if the difference between the green color value of lens 510 and the green color value of lens 520 is greater than or equal to a threshold, processor 110 can increase the gain corresponding to the brightness of lens 520 to correct the gain of lens 520. After increasing the gain of the brightness of lens 520, the difference between the green color value of lens 510 and the green color value of lens 520 can be reduced.

[0052] After completing the correction of the first color, in step S207, the processor 110 can calculate the absolute difference of the second color value between the reference lens and other lenses.

[0053] Processor 110 can detect stitched images to obtain color values ​​corresponding to lens 510 (or region of interest 61) and the second color, as well as color values ​​corresponding to lens 520 (or region of interest 62) and the second color. Processor 110 can define the second color as red or blue. In one embodiment, the second color value can be the ratio of a red color value to a blue color value. Then, processor 110 can calculate the absolute difference between the second color values ​​of lens 510 and lens 520.

[0054] In step S208, the processor 110 determines whether the absolute difference of the second color values ​​is less than a threshold. If the absolute difference is less than the threshold, step S210 is executed. If the absolute difference is greater than or equal to the threshold, step S209 is executed.

[0055] In step S209, processor 110 can correct the gain of other lenses based on the second color value of the reference lens. Then, processor 110 can re-execute step S207.

[0056] In one embodiment, processor 110 can correct the gain of other lenses by increasing or decreasing the gain corresponding to the second color (e.g., blue or red) of other lenses. For example, processor 110 can adjust the blue color gain and / or red color gain of lens 520 so that the second color value of lens 520 (e.g., the ratio of red color value to blue color value) is closer to the second color value of lens 510.

[0057] In step S210, processor 110 may generate a correction result based on the gain obtained in step S206 and the gain obtained in step S209. The correction result may include, for example, the gain of other lenses (e.g., lens 520), wherein the gain may include a gain in brightness, a gain in a first color (e.g., green), or a gain in a second color (e.g., red or blue). Processor 110 may output the correction result via transceiver 130. For example, processor 110 may transmit the correction result to multi-lens panoramic camera 500. Multi-lens panoramic camera 500 may perform color correction based on the correction result.

[0058] Figure 11 illustrates a flowchart of a color correction method for a multi-lens panoramic camera according to an embodiment of the present invention, wherein the color correction method may be implemented by a color correction system 10. The multi-lens panoramic camera may include a first lens and a second lens. In step S111, a stitched image is received from the multi-lens panoramic camera. In step S112, the stitched image is detected to obtain a first value corresponding to the first lens and a first color, and a second value corresponding to the second lens and the first color. In step S113, in response to the first value being greater than the second value, a first gain of the second lens is corrected according to the first value to generate a correction result. In step S114, the correction result is output.

[0059] In summary, the color correction system of the present invention uses a non-reflective white surface to cover the field of view of each lens of a multi-lens panoramic camera, and illuminates the white surface with uniform light. The multi-lens panoramic camera can capture a stitched image including at least a portion of the white surface. An electronic processing unit can detect the stitched image to obtain the color values ​​of each lens, and determine whether to correct the gain of each lens based on the color difference between the multiple lenses. The electronic processing unit can sequentially adjust the gains of green and red / blue to minimize the color difference between the multiple lenses and generate a correction result. The electronic processing unit can use the correction result to configure the multi-lens panoramic camera and improve the quality of the stitched image produced by the multi-lens panoramic camera.

[0060] 10: Color Correction System 100: Electronic computing device 110: Processor 120: Storage Media 130: Transceiver 200: White surface 210, 220: White surface portion 310, 320: Light source 500: Multi-lens panoramic camera 510, 520, 530, 540: Lenses 61, 62: Regions of Interest 610, 620, 630, 640: Field of view D1, D2, D3, D4: Distance S111, S112, S113, S114, S201, S202, S203, S204, S205, S206, S207, S208, S209, S210: Steps

Claims

1. A color correction system for a multi-lens panoramic camera, comprising: White surface; The system includes an electronic computing device communicatively connected to the multi-lens panoramic camera, wherein the multi-lens panoramic camera includes a first lens and a second lens, wherein the white surface is configured to cover a first field of view of the first lens and a second field of view of the second lens, wherein a first distance from the first lens to the white surface is equal to a second distance from the second lens to the white surface, and wherein the electronic computing device is configured to perform: receiving a stitched image from the multi-lens panoramic camera; detecting the stitched image to obtain a first value corresponding to the first lens and a first color and a second value corresponding to the second lens and the first color; in response to the first value being greater than the second value, correcting a first gain of the second lens based on the first value to generate a correction result; and outputting the correction result.

2. The color correction system as claimed in claim 1, wherein the electronic processing unit is configured to further perform: increasing a gain corresponding to the brightness of the second lens to correct the first gain.

3. The color correction system of claim 2, wherein the electronic processing device is configured to further perform: updating the second value according to the first gain; determining whether the difference between the first value and the updated second value is less than a threshold; and generating the correction result according to the first gain in response to the difference being less than the threshold.

4. The color correction system as claimed in claim 2, wherein the electronic processing unit is configured to further perform: detecting the stitched image to obtain a third value corresponding to the first lens and the second color, and a fourth value corresponding to the second lens and the second color; and correcting a second gain of the second lens on the second color based on the third value to produce the correction result.

5. The color correction system of claim 4, wherein the electronic processing unit is configured to further perform: updating the fourth value according to the second gain; determining whether the absolute difference between the third value and the updated fourth value is less than a threshold; and generating the correction result according to the second gain in response to the absolute difference being less than the threshold.

6. The color correction system as claimed in claim 2, wherein the gain of luminance includes a gain corresponding to the first color and a gain corresponding to the second color.

7. The color correction system as claimed in claim 1, wherein the reflectivity of the white surface is less than a threshold.

8. The color correction system as described in claim 1, further comprising: A first light source is configured to illuminate a first portion of a first surface of the white surface with first uniform light. And a second light source, configured to illuminate a second uniform light onto a second portion of the first surface, wherein the second portion is opposite to the first portion.

9. A color correction method for a multi-lens panoramic camera, wherein the multi-lens panoramic camera includes a first lens and a second lens, and the color correction method includes: A white surface is configured to cover a first field of view of the first lens and a second field of view of the second lens, wherein a first distance from the first lens to the white surface is equal to a second distance from the second lens to the white surface; a stitched image is received from the multi-lens panoramic camera; the stitched image is detected to obtain a first value corresponding to the first lens and a first color, and a second value corresponding to the second lens and the first color; in response to the first value being greater than the second value, a first gain of the second lens is corrected according to the first value to generate a correction result; and the correction result is output.

10. The color correction method as claimed in claim 9, wherein the step of correcting the first gain of the second lens according to the first value to produce the correction result includes: Increase the gain corresponding to the brightness of the second lens to correct the first gain.

11. The color correction method as claimed in claim 10, wherein the step of correcting the first gain of the second lens according to the first value to produce the correction result further comprises: Update the second value based on the first gain; Determine whether the difference between the first value and the updated second value is less than a threshold. And in response to the difference being less than the threshold, the correction result is generated based on the first gain.

12. The color correction method as described in claim 10 further includes: The stitched image is detected to obtain a third value corresponding to the first lens and the second color, and a fourth value corresponding to the second lens and the second color; And the second gain of the second lens for the second color is corrected according to the third value to produce the correction result.

13. The color correction method as claimed in claim 12, wherein the step of correcting the second gain of the second lens for the second color according to the third value to produce the correction result includes: The fourth value is updated based on the second gain; Determine whether the absolute difference between the third value and the updated fourth value is less than a threshold. And in response to the absolute difference being less than the threshold, the correction result is generated based on the second gain.

14. The color correction method as claimed in claim 10, wherein the gain of brightness includes a gain corresponding to the first color and a gain corresponding to the second color.

15. The color correction method as described in claim 9, wherein the reflectivity of the white surface is less than a threshold.

16. The color correction method as described in claim 9 further includes: A first light source is configured to illuminate a first portion of a first surface of the white surface with first uniform light. And a second light source is configured to illuminate a second uniform light onto a second portion of the first surface, wherein the second portion is opposite to the first portion.

Images