Color chessboard calibration method and device for splicing camera and camera

By using color checkerboards and Harris corner point detection algorithms in the stitching camera, the homography transformation parameters are calculated, and the problem of corner point matching errors in the calibration of the stitching camera is solved, improving the accuracy and quality of the panoramic map.

CN113989392BActive Publication Date: 2024-12-06SHENZHEN TVT DIGITAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111424221.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-26
Publication Date
2024-12-06
Estimated Expiration
2041-11-26

AI Technical Summary

Technical Problem

In the calibration methods of existing stitching cameras, the matching of corner points of the black and white chessboard is prone to errors, resulting in image deformation problems such as misalignment, ghosting and missing field of view in the panoramic image.

Method used

The color checkerboard calibration method is used to calculate the precise homography parameters to generate a panoramic view by using a variety of background and foreground colors in the checkerboard, combining the Harris corner detection algorithm and homography transformation formula.

Benefits of technology

It improves the accuracy of corner matching, reduces the problems of misalignment, ghosting and field of view in panoramic images, and enhances the accuracy of calibration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113989392B_ABST
    Figure CN113989392B_ABST
Patent Text Reader

Abstract

The present invention discloses a color chessboard calibration method, device and camera of a spliced ​​camera, wherein the method comprises: using a spliced ​​camera to shoot a color chessboard image; extracting all corner points in two shot images; determining the background color and foreground color of each grid in the images shot by two lenses; determining the pattern of each grid in the images shot by two lenses according to the combination of foreground color and background color; calculating a rough homography transformation matrix; projecting the horizontal and vertical coordinates of all corner points in the image shot by the first lens one by one into the image shot by the second lens according to the rough homography transformation matrix to obtain the projection coordinates in the image shot by the second lens; establishing a matching point pair; calculating the precise homography transformation parameters of the camera; projecting the first lens and the second lens images onto the same surface according to the precise homography transformation parameters to generate a panoramic image. The present invention improves the accuracy of calibration.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a camera, and more particularly to a color chessboard calibration method and device for a spliced ​​camera and the camera. Background Art

[0002] A stitching camera is a camera composed of two or more lenses. It is necessary to calibrate the geometric relationship between adjacent lenses so as to project the images of all lenses together to generate a panoramic image. Currently, most stitching cameras are calibrated using the black and white checkerboard calibration method. The disadvantage of this method is that the matching between the corner points of the checkerboard is prone to errors, resulting in obvious misalignment, ghosting, missing field of view and other image deformation problems in the panoramic image. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a color chessboard calibration method, device and camera for a spliced ​​camera.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, a color chessboard calibration method for a spliced ​​camera, wherein the color chessboard comprises a plurality of grids, the pattern of each grid comprises a background color and a foreground color located at the center of the background color, the background colors in two adjacent grids are different, wherein the background colors comprise black and white, and the foreground colors comprise black, white, red, green, blue, magenta, cyan, yellow and orange; the method comprises:

[0006] A color chessboard image is captured by using a splicing camera, wherein the splicing camera includes a first lens and a second lens;

[0007] Inputting the images captured by the first lens and the second lens into a calibration program to extract all corner points in the images captured by the first lens and the second lens;

[0008] Reading grayscale values ​​from pixels of four grids around the corner point in the image captured by the first lens and the second lens to determine the background color of each grid in the image captured by the first lens and the second lens;

[0009] Reading the RGB value of the central pixel of each grid in the image taken by the first lens and the second lens to determine the foreground color of each grid in the image taken by the first lens and the second lens;

[0010] Determining the pattern of each grid in the image taken by the first lens and the second lens according to a combination of a foreground color and a background color of each grid in the image taken by the first lens and the second lens;

[0011] Calculate the rough homography transformation matrix;

[0012] According to a rough homography transformation matrix, the horizontal and vertical coordinates of all corner points in the image taken by the first lens are projected one by one into the image taken by the second lens to obtain the projection coordinates in the image taken by the second lens;

[0013] Find the same corner point that is closest to the projection coordinate position and corresponds to the corner point in the image taken by the first lens in the image taken by the second lens, and establish a matching point pair;

[0014] According to the obtained matching point pairs and homography transformation formula, the precise homography transformation parameters of the camera are calculated;

[0015] The first lens and second lens images are projected onto the same surface according to the precise homography transformation parameters to generate a panorama.

[0016] Its further technical solution is: the images captured by the first lens and the second lens are input into a calibration program to extract all corner points in the images captured by the first lens and the second lens, and the Harris corner point detection algorithm is used to extract all corner points in the images captured by the first lens and the second lens.

[0017] A further technical solution is: the grayscale values ​​of the pixels of four grids around the corner points in the images taken by the first lens and the second lens are read to determine the background color of each grid in the images taken by the first lens and the second lens, specifically including:

[0018] Extract the pixels of the four grids around the corner point;

[0019] Read the grayscale values ​​of four grids according to the pixels;

[0020] Determine whether the difference between the grayscale value of the grid and the RGB value of 0 is less than the set value;

[0021] If the difference between the grayscale value of the grid and the RGB value of 0 is less than the set value, the background color of the grid is determined to be black;

[0022] If not, determine whether the difference between the grayscale value of the grid and the RGB value of 255 is less than the set value;

[0023] If the difference between the grayscale value of the grid and the RGB value of 255 is less than the set value, the background color of the grid is determined to be white.

[0024] The further technical solution is: in the rough homography transformation matrix, the calculation formula is:

[0025] Among them, f is the focal length of the lens, a is the size of a single pixel of the sensor, and c is the x is the horizontal coordinate of the image center, cy is the ordinate of the image center, α, β, γ are the angles between the three-dimensional coordinate axes of adjacent lenses, and R is the rotation matrix.

[0026] In a second aspect, a color chessboard calibration device for a splicing camera, the color chessboard includes a plurality of grids, the pattern of each grid includes a background color and a foreground color located at the center of the background color, the background colors in two adjacent grids are different, the background colors include black and white, and the foreground colors include black, white, red, green, blue, magenta, cyan, yellow and orange; the device includes: a shooting unit, an extraction unit, a first reading unit, a second reading unit, a combination unit, a first calculation unit, a projection unit, a creation unit, a second calculation unit and a panoramic image generation unit;

[0027] The shooting unit is used to shoot a color chessboard image using a splicing camera, wherein the splicing camera includes a first lens and a second lens;

[0028] The extraction unit is used to input the images captured by the first lens and the second lens into the calibration program to extract all corner points in the images captured by the first lens and the second lens;

[0029] The first reading unit is used to read grayscale values ​​from pixels of four grids around the corner point in the image taken by the first lens and the second lens, so as to determine the background color of each grid in the image taken by the first lens and the second lens;

[0030] The second reading unit is used to read the RGB value of the central pixel of each grid in the image taken by the first lens and the second lens to determine the foreground color of each grid in the image taken by the first lens and the second lens;

[0031] The combination unit is used to determine the pattern of each grid in the image taken by the first lens and the second lens according to the combination of the foreground color and the background color of each grid in the image taken by the first lens and the second lens;

[0032] The first calculation unit is used to calculate a rough homography transformation matrix;

[0033] The projection unit is used to project the horizontal and vertical coordinates of all corner points in the image taken by the first lens one by one into the image taken by the second lens according to the rough homography transformation matrix, so as to obtain the projection coordinates in the image taken by the second lens;

[0034] The establishing unit is used to search for the same corner point that is closest to the projection coordinate position and corresponds to the corner point in the image taken by the first lens in the image taken by the second lens, and establish a matching point pair;

[0035] The second calculation unit is used to calculate the precise homography transformation parameters of the camera according to the obtained matching point pairs and the homography transformation formula;

[0036] The panoramic image generation unit is used to project the first lens image and the second lens image onto the same surface according to accurate homography transformation parameters to generate a panoramic image.

[0037] Its further technical solution is: the images captured by the first lens and the second lens are input into a calibration program to extract all corner points in the images captured by the first lens and the second lens, and the Harris corner point detection algorithm is used to extract all corner points in the images captured by the first lens and the second lens.

[0038] Its further technical solution is: the first reading unit includes an extraction module, a reading module, a first judgment module, a first determination module, a second judgment module and a second determination module;

[0039] The extraction module is used to extract pixels of four grids around the corner point;

[0040] The reading module is used to read the grayscale values ​​of four grids according to the pixels;

[0041] The first judgment module is used to judge whether the difference between the gray value of the grid and the RGB value when it is 0 is less than a set value;

[0042] The first determination module is used to determine that the background color of the grid is black;

[0043] The second judgment module is used to judge whether the difference between the gray value of the grid and the RGB value of 255 is less than a set value;

[0044] The second determination module is used to determine that the background color of the grid is white.

[0045] The further technical solution is: in the rough homography transformation matrix, the calculation formula is:

[0046] Among them, f is the focal length of the lens, a is the size of a single pixel of the sensor, and c is the x is the horizontal coordinate of the image center, c y is the ordinate of the image center, α, β, γ are the angles between the three-dimensional coordinate axes of adjacent lenses, and R is the rotation matrix.

[0047] In a third aspect, a camera comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the color chessboard calibration method for a stitched camera are implemented as described above.

[0048] In a fourth aspect, a storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the steps of the color chessboard calibration method for a stitched camera as described above.

[0049] The beneficial effects of the present invention compared with the prior art are: the present invention uses a color chessboard for calibration, which can add color information to corner point matching, improve the accuracy of calibration, reduce the problem of easy errors in matching between corner points in the chessboard during calibration, and avoid image deformation problems such as obvious misalignment, ghosting, and loss of field of view in panoramic images.

[0050] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0052] Figure 1 A flowchart of a color chessboard calibration method for a spliced ​​camera provided in a specific embodiment of the present invention;

[0053] Figure 2 A schematic block diagram of a color chessboard calibration device for a spliced ​​camera provided in a specific embodiment of the present invention;

[0054] Figure 3 A schematic block diagram of a camera provided in accordance with a specific embodiment of the present invention;

[0055] Figure 4 A state diagram of a color chessboard captured by a splicing camera used in the present invention;

[0056] Figure 5 This is a schematic diagram of an image obtained by shooting a color chessboard with a stitching camera used in the present invention. DETAILED DESCRIPTION

[0057] In order to more fully understand the technical content of the present invention, the technical solution of the present invention is further introduced and illustrated in conjunction with specific embodiments below, but is not limited thereto.

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0060] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0061] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0062] A spliced ​​camera is a camera composed of two or more lenses. It is necessary to calibrate and calculate the geometric relationship between adjacent lenses so as to project the images of all lenses together to generate a panoramic image. A chessboard refers to a plane chessboard in which each grid is a rectangle, or a spherical chessboard with longitude and latitude as the grid border. The present invention is described below through specific embodiments.

[0063] Figure 1A flow chart of a color chessboard calibration method for a splicing camera provided in a specific embodiment of the present invention. A color chessboard calibration method for a splicing camera, wherein the color chessboard comprises a plurality of rectangular grids of the same size, the pattern of each grid comprises a background color and a foreground color located at the center of the background color, in order to make a clear distinction between the foreground color and the background color, the area of ​​the foreground color can be circular or in any polygonal shape, and in order to avoid the area of ​​the foreground color covering the border and corners of the grid, the area of ​​the foreground color area occupies 20%-80% of the entire grid. The background colors in two adjacent grids are different, wherein the background colors include black and white, and the foreground colors include black, white, red, green, blue, magenta, cyan, yellow and orange. Two background colors and nine foreground colors can form 16 color patterns, namely, pure black, red on black, green on black, blue on black, magenta on black, cyan on black, yellow on black, orange on black, pure white, red on white, green on white, blue on white, magenta on white, cyan on white, yellow on white, and orange on white. When designing patterns on a chessboard, you can paste out the sixteen patterns in units of four squares * four squares, a total of sixteen squares. There is no specific requirement for the internal arrangement. Then, use this as a cycle to paste the surrounding squares so that the same pattern is four squares apart in the horizontal and vertical directions.

[0064] like Figure 1 As shown, the color chessboard calibration method of the spliced ​​camera includes the following steps:

[0065] S10, using a splicing camera to shoot a color chessboard image, where the splicing camera includes a first lens and a second lens.

[0066] like Figure 4 As shown, in this embodiment, two lenses are used to shoot the color chessboard, and it is ensured that the overlapping area of ​​the field of view of the two lenses contains at least one column of grids. The two lenses are a first lens and a second lens.

[0067] S20: Input the images captured by the first lens and the second lens into a calibration program to extract all corner points in the images captured by the first lens and the second lens.

[0068] The images captured by the first lens and the second lens are input into the calibration program, and the corner points of the chessboard are extracted using the Harris corner detection algorithm to obtain the horizontal and vertical coordinates (x, y) of all corner points. The Harris corner detection algorithm is well known to people in the field of computer vision and will not be described in detail again.

[0069] S30, reading grayscale values ​​from pixels of four grids around the corner point in the image captured by the first lens and the second lens to determine the background color of each grid in the image captured by the first lens and the second lens.

[0070] Step S30 includes the following steps:

[0071] S301, extracting pixels of four grids around the corner point.

[0072] S302, reading the grayscale values ​​of four grids according to the pixels.

[0073] S303, determine whether the difference between the grayscale value of the grid and the RGB value when it is 0 is less than the set value, if so, determine that the background color of the grid is black; if not, determine whether the difference between the grayscale value of the grid and the RGB value when it is 255 is less than the set value, if so, determine that the background color of the grid is white.

[0074] The grayscale value of the grid can be used to determine whether the background color of the grid is black or white. In addition, the setting value can be determined according to actual needs.

[0075] S40, reading the RGB value of the central pixel of each grid in the image taken by the first lens and the second lens to determine the foreground color of each grid in the image taken by the first lens and the second lens.

[0076] Read the RGB value of the central pixel of each grid and classify it into nine foreground colors: black, white, red, green, blue, magenta, cyan, yellow, and orange according to the RGB value.

[0077] S50: Determine the pattern of each grid in the image taken by the first lens and the second lens according to the combination of the foreground color and the background color of each grid in the image taken by the first lens and the second lens.

[0078] S60, calculating a rough homography transformation matrix.

[0079] According to the optical and structural parameters, a rough homography transformation matrix is ​​calculated. Specifically, the calculation formula is:

[0080] Among them, f is the focal length of the lens, a is the size of a single pixel of the sensor, and c is the x is the horizontal coordinate of the image center, c y is the ordinate of the image center, α, β, γ are the angles between the three-dimensional coordinate axes of adjacent lenses, and R is the rotation matrix.

[0081] S70. Project the horizontal and vertical coordinates of all corner points in the image taken by the first lens one by one into the image taken by the second lens according to the rough homography transformation matrix, so as to obtain the projection coordinates (x1', y1') in the image taken by the second lens.

[0082] There is the following relationship:

[0083]

[0084] S80, searching for the same corner point in the image taken by the second lens that is closest to the projection coordinate position and corresponds to the corner point in the image taken by the first lens, and establishing a matching point pair.

[0085] Find the corner point (x2, y2) that is closest to the coordinates (x1', y1') and is surrounded by the same pattern, and obtain the matching point pairs (x1, y1) and (x2, y2) of the first lens and the second lens. Figure 5 As shown, the corner point P1 is surrounded by four patterns: blue on white background, blue on black background, pure white, and pure black. Its position projected into the lens 2 is P1', and the nearest identical corner point found is P2.

[0086] Since the traditional black and white chessboard has no color pattern as matching information, it can only find the nearest corner point P3, resulting in an erroneous pair of matching points P1 and P3. This process is like matching the face of a person in the image taken by the first lens to the waist of another person in the image taken by the second lens, resulting in a distorted panorama.

[0087] S90, calculating accurate homography transformation parameters of the camera according to the obtained matching point pairs and homography transformation formula.

[0088] After obtaining the matching point pairs (x1, y1) and (x2, y2), according to the homography transformation formula:

[0089] The precise homography transformation parameters h0-h7 of the camera can be calculated.

[0090] S100 , projecting the images of the first lens and the second lens onto the same surface according to precise homography transformation parameters h0 - h7 to generate a panoramic image.

[0091] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0092] Corresponding to the above-mentioned color chessboard calibration method for a spliced ​​camera, a specific embodiment of the present invention further provides a color chessboard calibration device for a spliced ​​camera.

[0093] like Figure 2 As shown, the color chessboard calibration device 100 of the stitching camera includes a shooting unit 110, an extraction unit 120, a first reading unit 130, a second reading unit 140, a combination unit 150, a first calculation unit 160, a projection unit 170, a creation unit 180, a second calculation unit 190 and a panoramic image generation unit 200.

[0094] The shooting unit 110 is used to shoot a color chessboard image using a splicing camera, where the splicing camera includes a first lens and a second lens.

[0095] like Figure 4 As shown, in this embodiment, two lenses are used to shoot the color chessboard, and it is ensured that the overlapping area of ​​the field of view of the two lenses contains at least one column of grids. The two lenses are a first lens and a second lens.

[0096] The extraction unit 120 is used to input the images captured by the first lens and the second lens into a calibration program to extract all corner points in the images captured by the first lens and the second lens.

[0097] The images captured by the first lens and the second lens are input into the calibration program, and the corner points of the chessboard are extracted using the Harris corner detection algorithm to obtain the horizontal and vertical coordinates (x, y) of all corner points. The Harris corner detection algorithm is well known to people in the field of computer vision and will not be described in depth here.

[0098] The first reading unit 130 is used to read gray values ​​from pixels of four grids around a corner point in the image taken by the first lens and the second lens, so as to determine the background color of each grid in the image taken by the first lens and the second lens.

[0099] Specifically, the first reading unit 130 includes an extracting module, a reading module, a first judging module, a first determining module, a second judging module, and a second determining module.

[0100] The extraction module is used to extract the pixels of four grids around the corner point.

[0101] The reading module is used to read the grayscale values ​​of four grids according to the pixels.

[0102] The first judgment module is used to judge whether the difference between the gray value of the grid and the RGB value when it is 0 is less than a set value.

[0103] The first determination module is used to determine that the background color of the grid is black.

[0104] The second judgment module is used to judge whether the difference between the gray value of the grid and the RGB value of 255 is less than a set value.

[0105] The second determination module is used to determine that the background color of the grid is white.

[0106] The grayscale value of the grid can be used to determine whether the background color of the grid is black or white. In addition, the setting value can be determined according to actual needs.

[0107] The second reading unit 140 is used to read the RGB value of the central pixel of each grid in the image taken by the first lens and the second lens to determine the foreground color of each grid in the image taken by the first lens and the second lens.

[0108] Read the RGB value of the central pixel of each grid and classify it into nine foreground colors: black, white, red, green, blue, magenta, cyan, yellow, and orange according to the RGB value.

[0109] The combination unit 150 is used to determine the pattern of each grid in the image taken by the first lens and the second lens according to the combination of the foreground color and the background color of each grid in the image taken by the first lens and the second lens.

[0110] The first calculation unit 160 is used to calculate a rough homography transformation matrix.

[0111] According to the optical and structural parameters, a rough homography transformation matrix is ​​calculated. Specifically, the calculation formula is:

[0112] Among them, f is the focal length of the lens, a is the size of a single pixel of the sensor, and c is the x is the horizontal coordinate of the image center, c y is the ordinate of the image center, α, β, γ are the angles between the three-dimensional coordinate axes of adjacent lenses, and R is the rotation matrix.

[0113] The projection unit 170 is used to project the horizontal and vertical coordinates of all corner points in the image taken by the first lens one by one into the image taken by the second lens according to the rough homography transformation matrix, so as to obtain the projection coordinates in the image taken by the second lens.

[0114] There is the following relationship:

[0115]

[0116] The establishing unit 180 is used to search for the same corner point that is closest to the projection coordinate position and corresponds to the corner point in the image taken by the first lens in the image taken by the second lens, and establish a matching point pair.

[0117] Find the corner point (x2, y2) that is closest to the coordinates (x1', y1') and is surrounded by the same pattern, and obtain the matching point pairs (x1, y1) and (x2, y2) of the first lens and the second lens. Figure 5 As shown, the corner point P1 is surrounded by four patterns: blue on white background, blue on black background, pure white, and pure black. Its position projected into the lens 2 is P1', and the nearest identical corner point found is P2.

[0118] Since the traditional black and white chessboard has no color pattern as matching information, it can only find the nearest corner point P3, resulting in an erroneous pair of matching points P1 and P3. This process is like matching the face of a person in the image taken by the first lens to the waist of another person in the image taken by the second lens, resulting in a distorted panorama.

[0119] The second calculation unit 190 is used to calculate the precise homography transformation parameters of the camera according to the obtained matching point pairs and the homography transformation formula.

[0120] After obtaining the matching point pairs (x1, y1) and (x2, y2), according to the homography transformation formula:

[0121] The precise homography transformation parameters h0-h7 of the camera can be calculated.

[0122] The panoramic image generating unit 200 is used to project the first lens image and the second lens image onto the same surface according to the precise homography transformation parameters h0-h7 to generate a panoramic image.

[0123] like Figure 3 As shown, a specific embodiment of the present invention further provides a camera, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the color chessboard calibration method steps of the spliced ​​camera as described above are implemented.

[0124] The camera 700 may be a terminal or a server. The camera 700 includes a processor 720 , a memory, and a network interface 750 connected via a system bus 710 , wherein the memory may include a non-volatile storage medium 730 and an internal memory 740 .

[0125] The non-volatile storage medium 730 can store an operating system 731 and a computer program 732. When the computer program 732 is executed, the processor 720 can execute any color chessboard calibration method for a spliced ​​camera.

[0126] The processor 720 is used to provide computing and control capabilities to support the operation of the entire camera 700 .

[0127] The internal memory 740 provides an environment for the operation of the computer program 732 in the non-volatile storage medium 730. When the computer program 732 is executed by the processor 720, the processor 720 can execute any color chessboard calibration method for the spliced ​​camera.

[0128] The network interface 750 is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will appreciate that Figure 3The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the camera 700 to which the solution of the present application is applied. The specific camera 700 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. The processor 720 is used to run the program code stored in the memory to implement the following steps:

[0129] A color chessboard image is captured by using a splicing camera, wherein the splicing camera includes a first lens and a second lens;

[0130] Inputting the images captured by the first lens and the second lens into a calibration program to extract all corner points in the images captured by the first lens and the second lens;

[0131] Reading grayscale values ​​from pixels of four grids around the corner point in the image captured by the first lens and the second lens to determine the background color of each grid in the image captured by the first lens and the second lens;

[0132] Reading the RGB value of the central pixel of each grid in the image taken by the first lens and the second lens to determine the foreground color of each grid in the image taken by the first lens and the second lens;

[0133] Determining the pattern of each grid in the image taken by the first lens and the second lens according to a combination of a foreground color and a background color of each grid in the image taken by the first lens and the second lens;

[0134] Calculate the rough homography transformation matrix;

[0135] According to a rough homography transformation matrix, the horizontal and vertical coordinates of all corner points in the image taken by the first lens are projected one by one into the image taken by the second lens to obtain the projection coordinates in the image taken by the second lens;

[0136] Find the same corner point that is closest to the projection coordinate position and corresponds to the corner point in the image taken by the first lens in the image taken by the second lens, and establish a matching point pair;

[0137] According to the obtained matching point pairs and homography transformation formula, the precise homography transformation parameters of the camera are calculated;

[0138] The first lens and second lens images are projected onto the same surface according to the precise homography transformation parameters to generate a panorama.

[0139] It should be understood that in the embodiment of the present application, the processor 720 may be a central processing unit (CPU), and the processor 720 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0140] Those skilled in the art will understand that Figure 3 The structure of the camera 700 shown in the figure does not constitute a limitation on the camera 700, and the camera 700 may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0141] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0142] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0143] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0144] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0145] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A color chessboard calibration method for a spliced ​​camera, characterized in that: The color chessboard includes a plurality of grids, the pattern of each grid includes a background color and a foreground color located at the center of the background color, the background colors in two adjacent grids are different, the background colors include black and white, and the foreground colors include black, white, red, green, blue, magenta, cyan, yellow and orange; the method includes: A color chessboard image is captured by using a splicing camera, wherein the splicing camera includes a first lens and a second lens; Inputting the images captured by the first lens and the second lens into a calibration program to extract all corner points in the images captured by the first lens and the second lens; Reading grayscale values ​​from pixels of four grids around the corner point in the image captured by the first lens and the second lens to determine the background color of each grid in the image captured by the first lens and the second lens; Reading the RGB value of the central pixel of each grid in the image taken by the first lens and the second lens to determine the foreground color of each grid in the image taken by the first lens and the second lens; Determining the pattern of each grid in the image taken by the first lens and the second lens according to a combination of a foreground color and a background color of each grid in the image taken by the first lens and the second lens; Calculate the rough homography transformation matrix; According to a rough homography transformation matrix, the horizontal and vertical coordinates of all corner points in the image taken by the first lens are projected one by one into the image taken by the second lens to obtain the projection coordinates in the image taken by the second lens; Find the same corner point that is closest to the projection coordinate position and corresponds to the corner point in the image taken by the first lens in the image taken by the second lens, and establish a matching point pair; According to the obtained matching point pairs and homography transformation formula, the precise homography transformation parameters of the camera are calculated. The homography transformation formula is: Projecting the first lens image and the second lens image onto the same surface according to precise homography transformation parameters to generate a panoramic image; In the above-mentioned rough calculation of the homography transformation matrix, the calculation formula is: Among them, f is the focal length of the lens, a is the size of a single pixel of the sensor, and c is the x is the horizontal coordinate of the image center, c y is the ordinate of the image center, α, β, γ are the angles between the three-dimensional coordinate axes of adjacent lenses, and R is the rotation matrix.

2. The color chessboard calibration method of a spliced ​​camera according to claim 1, characterized in that: The images captured by the first lens and the second lens are input into the calibration program to extract all corner points in the images captured by the first lens and the second lens, and the Harris corner point detection algorithm is used to extract all corner points in the images captured by the first lens and the second lens.

3. The color chessboard calibration method of a spliced ​​camera according to claim 1, characterized in that: The step of reading grayscale values ​​of pixels of four grids around corner points in the images captured by the first lens and the second lens to determine the background color of each grid in the images captured by the first lens and the second lens specifically includes: Extract the pixels of the four grids around the corner point; Read the grayscale values ​​of four grids according to the pixels; Determine whether the difference between the grayscale value of the grid and the RGB value of 0 is less than the set value; If the difference between the grayscale value of the grid and the RGB value of 0 is less than the set value, the background color of the grid is determined to be black; If not, determine whether the difference between the grayscale value of the grid and the RGB value of 255 is less than the set value; If the difference between the grayscale value of the grid and the RGB value of 255 is less than the set value, the background color of the grid is determined to be white.

4. A color chessboard calibration device for a spliced ​​camera, characterized in that: The color chessboard includes a plurality of grids, the pattern of each grid includes a background color and a foreground color located at the center of the background color, the background colors in two adjacent grids are different, the background colors include black and white, and the foreground colors include black, white, red, green, blue, magenta, cyan, yellow and orange; the device includes a shooting unit, an extraction unit, a first reading unit, a second reading unit, a combination unit, a first calculation unit, a projection unit, a creation unit, a second calculation unit and a panoramic image generation unit; The shooting unit is used to shoot a color chessboard image using a splicing camera, wherein the splicing camera includes a first lens and a second lens; The extraction unit is used to input the images captured by the first lens and the second lens into the calibration program to extract all corner points in the images captured by the first lens and the second lens; The first reading unit is used to read grayscale values ​​from pixels of four grids around the corner point in the image taken by the first lens and the second lens, so as to determine the background color of each grid in the image taken by the first lens and the second lens; The second reading unit is used to read the RGB value of the central pixel of each grid in the image taken by the first lens and the second lens to determine the foreground color of each grid in the image taken by the first lens and the second lens; The combination unit is used to determine the pattern of each grid in the image taken by the first lens and the second lens according to the combination of the foreground color and the background color of each grid in the image taken by the first lens and the second lens; The first calculation unit is used to calculate a rough homography transformation matrix; The projection unit is used to project the horizontal and vertical coordinates of all corner points in the image taken by the first lens one by one into the image taken by the second lens according to the rough homography transformation matrix, so as to obtain the projection coordinates in the image taken by the second lens; The establishing unit is used to search for the same corner point that is closest to the projection coordinate position and corresponds to the corner point in the image taken by the first lens in the image taken by the second lens, and establish a matching point pair; The second calculation unit is used to calculate the precise homography transformation parameters of the camera according to the obtained matching point pairs and the homography transformation formula, and the homography transformation formula is: The panorama generating unit is used to project the first lens image and the second lens image onto the same surface according to the precise homography transformation parameters to generate a panorama; In the above-mentioned rough calculation of the homography transformation matrix, the calculation formula is: Among them, f is the focal length of the lens, a is the size of a single pixel of the sensor, and c is the x is the horizontal coordinate of the image center, c y is the ordinate of the image center, α, β, γ are the angles between the three-dimensional coordinate axes of adjacent lenses, and R is the rotation matrix.

5. The color chessboard calibration device for a spliced ​​camera according to claim 4, characterized in that: The images captured by the first lens and the second lens are input into the calibration program to extract all corner points in the images captured by the first lens and the second lens, and the Harris corner point detection algorithm is used to extract all corner points in the images captured by the first lens and the second lens.

6. The color chessboard calibration device for a spliced ​​camera according to claim 4, characterized in that: The first reading unit includes an extraction module, a reading module, a first judgment module, a first determination module, a second judgment module and a second determination module; The extraction module is used to extract pixels of four grids around the corner point; The reading module is used to read the grayscale values ​​of four grids according to the pixels; The first judgment module is used to judge whether the difference between the gray value of the grid and the RGB value when it is 0 is less than a set value; The first determination module is used to determine that the background color of the grid is black; The second judgment module is used to judge whether the difference between the gray value of the grid and the RGB value of 255 is less than a set value; The second determination module is used to determine that the background color of the grid is white.

7. A camera, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method steps of color chessboard calibration of a spliced ​​camera as claimed in any one of claims 1 to 3 are implemented.

8. A storage medium, characterized in that: The storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes the steps of the color chessboard calibration method for a spliced ​​camera as described in any one of claims 1 to 3.