Camera-projector calibration method, system and medium based on color chessboard

Abstract

The present invention discloses a camera-projector calibration method, system and medium based on a color chessboard, the method comprising: projecting a projection chessboard onto a reference plane pre-set with a physical chessboard to obtain a superimposed color chessboard; obtaining first image information of the superimposed color chessboard; determining a first color component and a second color component, so that the first color has the first color component but does not have the second color component, and the second color has the second color component but does not have the first color component; extracting a first grayscale image of a corresponding color channel from the first image information according to the first color component, and extracting a second grayscale image of a corresponding color channel from the first image information according to the second color component; determining a first parameter of a projector and a second parameter of a camera according to the first grayscale image and the second grayscale image. The present invention improves the efficiency of camera-projector calibration, reduces labor costs and time costs, and can be widely applied to the field of image processing technology.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a camera-projector calibration method, system, and medium based on a color checkerboard. Background Technology

[0002] In structured light systems for 3D imaging, camera-projector calibration is essential. A common challenge in calibration is the need to capture two distinct chessboards. First, a physical chessboard must be mounted on a reference plane to obtain camera parameters. Second, the physical chessboard must be removed before projecting onto the projection chessboard to obtain projector parameters, as the physical chessboard interferes with the projection pattern required for obtaining these parameters. Therefore, existing camera-projector calibration methods are complex, requiring multiple images to complete the calibration. The installation and removal of the physical chessboard also consumes considerable manpower and time, impacting the efficiency of camera-projector calibration. Summary of the Invention

[0003] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.

[0004] Therefore, one objective of this invention is to provide a camera-projector calibration method based on a color checkerboard, which improves the efficiency of camera-projector calibration based on a color checkerboard and reduces labor and time costs.

[0005] Another objective of this invention is to provide a camera-projector calibration system based on a color checkerboard.

[0006] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of the present invention include:

[0007] In a first aspect, embodiments of the present invention provide a camera-projector calibration method based on a color checkerboard, comprising the following steps:

[0008] A projected chessboard is projected onto a reference plane pre-set with a physical chessboard using a projector, resulting in a superimposed color chessboard. The first color of the projected chessboard is different from the second color of the physical chessboard, and the reference plane is white.

[0009] The first image information of the superimposed color chessboard is acquired through a camera;

[0010] Determine a first color component and a second color component such that the first color has the first color component but not the second color component, and the second color has the second color component but not the first color component.

[0011] Based on the first color component, a first grayscale image corresponding to the color channel is extracted from the first image information; based on the second color component, a second grayscale image corresponding to the color channel is extracted from the first image information.

[0012] The first parameter of the projector and the second parameter of the camera are determined based on the first grayscale image and the second grayscale image.

[0013] Furthermore, in one embodiment of the present invention, the first color is cyan, the second color is yellow, the first color component is a blue component, and the second color component is a red component.

[0014] Furthermore, in one embodiment of the present invention, both the chessboard pattern of the projected chessboard and the chessboard pattern of the physical chessboard are generated by the following formula:

[0015]

[0016] g x (x, y) = sin(πx),

[0017] g y (x, y) = sin(πy),

[0018] g r (x, y)=sin[2θ(x,y)]sin[2(n+1)θ(x,y)],

[0019] θ(x, y) = arctan(y, x),

[0020] Wherein, the y-axis of the chessboard pattern of the projected chessboard is set opposite to the y-axis of the chessboard image of the physical chessboard, sign[·] represents the sign function, x∈[-n, n+2], y∈[-n, n+1], and n represents the number of reference lines.

[0021] Furthermore, in one embodiment of the present invention, the step of determining the first parameter of the projector and the second parameter of the camera based on the first grayscale image and the second grayscale image specifically includes:

[0022] The structured light system is determined based on the first grayscale image and the second grayscale image;

[0023] The first homography matrix and the second homography matrix are determined according to the structured light system. The first homography matrix represents the correspondence matrix between points in the image plane and points in the reference plane, and the second homography matrix represents the correspondence matrix between points in the sliding plane and points in the reference plane.

[0024] The second parameters of the camera are determined based on the first homography matrix, and the first parameters of the projector are determined based on the second homography matrix.

[0025] Furthermore, in one embodiment of the present invention, the first homography matrix is ​​determined by the following formula:

[0026]

[0027] Where μ represents a point in the image plane, and ρ represents a point in the reference plane. Let G represent a homogeneous operator. i Represents the first homography matrix;

[0028] The second homography matrix is ​​determined by the following formula:

[0029]

[0030] G s =G -1 G i ,

[0031] Where ν represents a point in the sliding plane, and G represents the third homography matrix, which represents the correspondence matrix between points in the image plane and points in the sliding plane. s This represents the second homography matrix.

[0032] Furthermore, in one embodiment of the present invention, the first parameter includes a first intrinsic parameter and a first extrinsic parameter, and the second parameter includes a second intrinsic parameter and a second extrinsic parameter, wherein the first parameter and the second parameter are calculated by singular value decomposition.

[0033] Furthermore, in one embodiment of the present invention, the first parameter and the second parameter are determined by the following formula:

[0034]

[0035]

[0036]

[0037]

[0038] Among them, K s Represents the first intrinsic parameter, (R) s,k , t s,k K represents the first extrinsic parameter. i Represents the second intrinsic parameter, (R) i,k , t i,kThe ) represents the second extrinsic parameter, and x(μ), y(μ), and z(μ) represent the x, y, and z coordinates of the point μ in the image plane corresponding to the point μ in the reference plane. Let ν(μ) denote the homogeneous operator, and let ν(μ) denote the point in the image plane corresponding to point μ in the sliding plane.

[0039] Secondly, embodiments of the present invention provide a camera-projector calibration system based on a color checkerboard, comprising:

[0040] A color overlay chessboard projection module is used to project a projection chessboard onto a reference plane with a pre-set physical chessboard using a projector to obtain a color overlay chessboard. The first color of the projection chessboard is different from the second color of the physical chessboard, and the color of the reference plane is white.

[0041] The image acquisition module is used to acquire first image information of the superimposed color chessboard through a camera;

[0042] The color component determination module is used to determine a first color component and a second color component, such that the first color has the first color component but does not have the second color component, and the second color has the second color component but does not have the first color component.

[0043] A grayscale image extraction module is used to extract a first grayscale image of a corresponding color channel from the first image information based on the first color component, and to extract a second grayscale image of a corresponding color channel from the first image information based on the second color component;

[0044] The parameter determination module is used to determine the first parameter of the projector and the second parameter of the camera based on the first grayscale image and the second grayscale image.

[0045] Thirdly, embodiments of the present invention provide a camera-projector calibration device based on a color checkerboard, comprising:

[0046] At least one processor;

[0047] At least one memory for storing at least one program;

[0048] When the at least one program is executed by the at least one processor, the at least one processor implements the above-described camera-projector calibration method based on a color chessboard.

[0049] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the aforementioned camera-projector calibration method based on a color chessboard.

[0050] The advantages and beneficial effects of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention:

[0051] In this embodiment of the invention, while projecting a chessboard of the first color onto a projector, a physical chessboard of the second color is set on a reference plane, thus obtaining a superimposed color chessboard. The image information of the superimposed color chessboard is then acquired by a camera. Two suitable color components are selected based on the first and second colors, and the grayscale images of the corresponding color channels of each color component are extracted. This allows the reconstruction of the chessboard grid images of a single projected chessboard and a single physical chessboard, facilitating the acquisition of the homography matrix required for camera and projector calibration. This, in turn, allows the determination of the projector's first parameters and the camera's second parameters. The camera-projector calibration method of this embodiment is not constrained by the position and orientation of the camera, projector, and reference plane, and does not require the removal of the physical chessboard during the calibration process, thus improving the efficiency of camera-projector calibration and reducing labor and time costs. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments of the present invention are described below. It should be understood that the drawings described below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating the steps of a camera-projector calibration method based on a color checkerboard, as provided in an embodiment of the present invention;

[0054] Figure 2(a) is a schematic diagram of the chessboard grid pattern of the physical chessboard provided in an embodiment of the present invention;

[0055] Figure 2(b) is a schematic diagram of the chessboard grid pattern of the projection chessboard provided in an embodiment of the present invention;

[0056] Figure 3(a) is a schematic diagram of the first image information of the overlaid color chessboard provided in an embodiment of the present invention;

[0057] Figure 3(b) is a schematic diagram of the grayscale image of the red channel of the first image information provided in an embodiment of the present invention;

[0058] Figure 3(c) is a schematic diagram of the grayscale image of the green channel of the first image information provided in the embodiment of the present invention;

[0059] Figure 3(d) is a schematic diagram of the grayscale image of the blue channel of the first image information provided in the embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of a structured light system provided in an embodiment of the present invention;

[0061] Figure 5 A structural block diagram of a camera-projector calibration system based on a color checkerboard provided in an embodiment of the present invention;

[0062] Figure 6 This is a structural block diagram of a camera-projector calibration device based on a color checkerboard, provided in an embodiment of the present invention. Detailed Implementation

[0063] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0064] In the description of this invention, "multiple" means two or more. The use of "first" and "second" is for distinguishing technical features only and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or the order of the indicated technical features. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.

[0065] Reference Figure 1 This invention provides a camera-projector calibration method based on a color checkerboard, specifically including the following steps:

[0066] S101. Project the projection chessboard onto a reference plane with a pre-set physical chessboard using a projector to obtain an overlaid color chessboard, wherein the first color of the projection chessboard is different from the second color of the physical chessboard, and the color of the reference plane is white.

[0067] Specifically, in this embodiment of the invention, the physical chessboard and the projected chessboard use different colors, the two chessboards can be superimposed, and are simultaneously captured by a camera. In this way, the color information of different color channels can be used to detect the physical chessboard and the projected chessboard without interference, thereby capturing the chessboard grid images of the physical chessboard and the projected chessboard in one shot, which is convenient for subsequent camera-projector calibration using a structured light system.

[0068] In this embodiment of the invention, color selection is based on the additive red-green-blue (RGB) color model. The reference plane is white RGB(1,1,1) because it can reflect incident light of all colors. If a region of the reference plane has a specific color, such as red RGB(1,0,0), then all incident light will be absorbed except for the reflected red light. When the illumination light does not contain all color components, a complementary effect can be achieved. For example, if the illumination light is green RGB(0,1,0), then the entire white reference plane will be detected as green, except for the red region, which will be detected as black RGB(0,0,0), because the incident green light does not contain any color components that can be reflected by the red region.

[0069] This invention utilizes the aforementioned phenomenon to simultaneously detect overlapping physical and projected chessboards. The physical chessboard is yellow, and the projected chessboard is cyan. The yellow physical chessboard is pre-printed and placed on a reference plane, while the cyan projected chessboard is projected onto the reference plane by a projector. The resulting superimposed color chessboard can then be captured by a camera.

[0070] As a further optional implementation, both the chessboard pattern of the projected chessboard and the chessboard pattern of the physical chessboard are generated by the following formula:

[0071]

[0072] g x (x, y) = sin(πx),

[0073] g y (x, y) = sin(πy),

[0074] g r (x, y)=sin[2θ(x,y)]sin[2(n+1)θ(x,y)],

[0075] θ(x, y) = arctan(y, x),

[0076] In this context, the y-axis of the chessboard pattern of the projected chessboard is set opposite to the y-axis of the chessboard image of the physical chessboard, sign[·] represents the sign function, x∈[-n, n+2], y∈[-n, n+1], and n represents the number of reference lines.

[0077] Figures 2(a) and 2(b) show schematic diagrams of the chessboard grid patterns of the physical chessboard and the projected chessboard provided in the embodiments of the present invention, respectively. In the embodiments of the present invention, the number of reference lines is n=8, the y-axis of the physical chessboard and the projected chessboard are opposite, and the z-axis is the direction of observation (not shown).

[0078] S102. Obtain the first image information of the overlaid color chessboard through the camera.

[0079] Figure 3(a) shows a schematic diagram of the first image information of the superimposed color chessboard provided in the embodiment of the present invention. It can be seen that the physical chessboard and the projected chessboard are superimposed. Therefore, in the subsequent embodiment of the present invention, the color channels will be used to extract the physical chessboard and the projected chessboard separately.

[0080] S103. Determine the first color component and the second color component, such that the first color has the first color component but not the second color component, and the second color has the second color component but not the first color component.

[0081] As a further optional implementation, the first color is cyan, the second color is yellow, the first color component is a blue component, and the second color component is a red component.

[0082] S104. Extract the first grayscale image of the corresponding color channel from the first image information according to the first color component, and extract the second grayscale image of the corresponding color channel from the first image information according to the second color component.

[0083] Specifically, Figures 3(b), 3(c), and 3(d) show grayscale image diagrams of the red, green, and blue channels of the first image information provided in this embodiment of the invention, respectively. The red channel captures only the projected chessboard. This is because the white area of ​​the reference plane and the yellow area of ​​the physical chessboard both have red components, but the cyan area of ​​the projected chessboard does not have a red component. Therefore, the cyan area on the projected chessboard absorbs red, while other areas reflect red, resulting in the chessboard grid image of the projected chessboard as shown in Figure 3(b). Similarly, the blue channel captures only the physical chessboard because the white area of ​​the reference plane and the cyan area of ​​the projected chessboard both have blue components, but the yellow area of ​​the physical chessboard does not have a blue component, resulting in the chessboard grid image of the physical chessboard as shown in Figure 3(d).

[0084] Ideally, the green channel should be bright, since both yellow and cyan include green components, and therefore neither the physical nor the projected chessboard should be detected. However, ambient lighting factors and unbalanced color brightness on the camera and projector can lead to invalid low-contrast detection of the chessboard, resulting in a grayscale image of the green channel as shown in Figure 3(c).

[0085] S105. Determine the first parameters of the projector and the second parameters of the camera based on the first grayscale image and the second grayscale image.

[0086] Specifically, in this embodiment of the invention, the camera, projector, and reference plane are located at different unknown positions and orientations. During the calibration process, there will be multiple different equipment arrangements. For each equipment arrangement, the camera acquires image information of the overlapping color checkerboard; the grayscale images of the red and blue channels of the captured image information are extracted, thereby establishing a structured light system between the camera, projector, and reference plane, and thus completing the parameter calibration of the camera and projector. Step S105 specifically includes the following steps:

[0087] S1051. Determine the structured light system based on the first grayscale image and the second grayscale image;

[0088] S1052. Determine the first homography matrix and the second homography matrix according to the structured light system. The first homography matrix represents the correspondence matrix between points in the image plane and points in the reference plane, and the second homography matrix represents the correspondence matrix between points in the sliding plane and points in the reference plane.

[0089] S1053. Determine the second parameters of the camera based on the first homography matrix, and determine the first parameters of the projector based on the second homography matrix.

[0090] As a further optional implementation, the first homography matrix is ​​determined by the following formula:

[0091]

[0092] Where μ represents a point in the image plane, and p represents a point in the reference plane. Let G represent a homogeneous operator. i This represents the first homography matrix;

[0093] The second homography matrix is ​​determined by the following formula:

[0094]

[0095] G s =G -1 G i ,

[0096] Where ν represents a point in the sliding plane, G represents the third homography matrix, which is the matrix showing the correspondence between points in the image plane and points in the sliding plane. s This represents the second homography matrix.

[0097] Specifically, structured light systems are powerful tools for 3D surface imaging, and their applications are becoming increasingly widespread. The advantages of this modern technology include non-contact evaluation, robustness, high precision, and real-time operation. The basic working principle of a structured light system is as follows: First, a projector and camera are used to generate and detect the correspondence between points. Then, triangulation is used to determine the 3D surface of the object being measured; and during triangulation, the camera and projector must be calibrated. Furthermore, precise calibration is required for high-precision measurements. Camera-projector matching calibration can be performed using different reference objects; for example, plates, spheres, and gauge blocks. In addition, the use of auxiliary devices is recommended. Feature points are typically detected using fringe patterns, cross gratings, and speckle patterns; using a reference plane with a checkerboard pattern is one of the most efficient and robust methods for establishing the desired point correspondence.

[0098] like Figure 4 The diagram shown is a schematic of a structured light system provided in an embodiment of the present invention, and the following vectors are defined:

[0099]

[0100] Here, vectors ρ, μ, and ν represent points on the reference plane, the image plane (i.e., the image plane captured by the camera), and the sliding plane (i.e., the projection plane of the projector), respectively. The correspondence between these points is crucial for the calibration of the camera and the projector.

[0101] This invention utilizes images of the chessboard grid in a physical chessboard to calibrate a camera with high precision. The method employs the following relationship:

[0102]

[0103] Where μ represents a point in the image plane, and ρ represents a point in the reference plane. Let G represent a homogeneous operator. i Let G represent the first homography matrix, which is the matrix showing the correspondence between points in the image plane and points in the reference plane. i The correspondence between μ and ρ can be used to estimate and determine the specific correspondence, which can be determined based on the captured chessboard pattern and the measurement information of the chessboard used. This is not the focus of this embodiment of the invention and will not be elaborated here.

[0104] For projector calibration, since a projector can be considered a "reverse camera," the same calibration method can be used to calibrate projector equipment. Calibration can be performed using the following relationship:

[0105]

[0106] Where ν represents a point in the sliding plane, G sThis represents the second homography matrix, which is the matrix showing the correspondence between points in the sliding plane and points in the reference plane.

[0107] However, unlike camera calibration, an image on the reference plane cannot be acquired using only a projector. This problem can be solved by using a camera as an auxiliary device to capture attractive points; that is, by capturing an image of the reference plane with a camera, and then solving for the correspondence matrix between points in the sliding plane and points in the reference plane.

[0108] Related G i and G s The third homography matrix is ​​defined as the correspondence matrix between points in the image plane and points in the sliding plane. Based on the foregoing, the following relationship can be obtained:

[0109]

[0110]

[0111] G represents the third homography matrix.

[0112] In this embodiment of the invention, the first homography matrix G can be completed using the chessboard grid image of the physical chessboard. i Since the estimate is determined, the second homography matrix G can be calculated using the following formula. s Estimation of G: s =G -1 G i .

[0113] It can be recognized that flexible camera-projector calibration methods require simultaneous estimation of G. i and G s And without needing to fix the camera, projector, or reference plane. The difficulty lies in G i and G s The estimation involves two experiments. In the first experiment, a physical chessboard pattern is placed on a reference plane to estimate G. i In the second experiment, the physical chessboard had to be removed because it interfered with the estimation of G. s The required projection mode. In practical applications, these two steps are often impractical and prone to errors due to misalignment. This embodiment of the invention overcomes this deficiency by extracting grayscale images of different channels of the overlaid color chessboard.

[0114] As a further optional implementation, the first parameter includes a first intrinsic parameter and a first extrinsic parameter, and the second parameter includes a second intrinsic parameter and a second extrinsic parameter. The first parameter and the second parameter are calculated by singular value decomposition.

[0115] As a further optional implementation, the first parameter and the second parameter are determined by the following formula:

[0116]

[0117]

[0118]

[0119]

[0120] Among them, K s Represents the first intrinsic parameter, (R) s,k , t s,k K represents the first extrinsic parameter. i Represents the second intrinsic parameter, (R) i,k , t i,k The ) represents the second extrinsic parameter, and x(μ), y(μ), and z(μ) represent the x, y, and z coordinates of the point μ in the image plane corresponding to the point μ in the reference plane. Let ν(μ) denote the homogeneous operator, and let ν(μ) denote the point in the image plane corresponding to point μ in the sliding plane.

[0121] Specifically, the parameters of the camera-projector can be determined by directly reconstructing the 3D surface using triangulation. Considering that the absolute phases of φx(μ) and φy(μ) are obtained by demodulating the patterns generated by the vertical and horizontal fringe grating projections, the coordinates of the point ν in the sliding plane corresponding to point μ in the image plane are shown below:

[0122]

[0123] Where f x and f y This is the spatial frequency of the projection grating. For each point μ on the image plane, the coordinates (x, y, z) of the corresponding point on the surface of the object being measured can be calculated as follows:

[0124]

[0125]

[0126]

[0127] Scale factor λ i (μ) and λ s (μ) is calculated as follows:

[0128]

[0129] in,[·] T This indicates the transpose operation.

[0130] By combining the above equations and performing calculations, we can obtain the first intrinsic parameter K of the projector. s First extrinsic parameter (R) s，k , t s，k ), and the camera's second intrinsic parameter K i Second extrinsic parameter (R) i，k , t i，k It is understandable that the subscript 'k' is used to identify the kth equipment setup. By repeating the aforementioned calibration process with multiple different equipment setups, the calibration of the camera-projector system can be completed.

[0131] It should be understood that, in this embodiment of the invention, while projecting a chessboard of the first color onto a projector, a physical chessboard of the second color is set on a reference plane, thereby obtaining a superimposed color chessboard. Then, the image information of the superimposed color chessboard is acquired by a camera. Two suitable color components are selected based on the first and second colors, and the grayscale images of the color channels corresponding to the two color components are extracted respectively. This allows the recovery of the chessboard grid images of a single projected chessboard and a single physical chessboard, facilitating the acquisition of the homography matrix required for camera and projector calibration, thereby determining the first parameters of the projector and the second parameters of the camera. The camera-projector calibration method of this embodiment is not constrained by the position and orientation of the camera, projector, and reference plane, and does not require the removal of the physical chessboard during the calibration process, improving the efficiency of camera-projector calibration and reducing labor and time costs.

[0132] Reference Figure 5 This invention provides a camera-projector calibration system based on a color checkerboard, comprising:

[0133] The overlay color chessboard projection module is used to project a projection chessboard onto a reference plane with a pre-set physical chessboard through a projector to obtain an overlay color chessboard. The first color of the projection chessboard is different from the second color of the physical chessboard, and the color of the reference plane is white.

[0134] The image acquisition module is used to acquire the first image information of the overlaid color chessboard through a camera;

[0135] The color component determination module is used to determine the first color component and the second color component, such that the first color has the first color component but does not have the second color component, and the second color has the second color component but does not have the first color component.

[0136] The grayscale image extraction module is used to extract a first grayscale image of the corresponding color channel from the first image information based on the first color component, and to extract a second grayscale image of the corresponding color channel from the first image information based on the second color component;

[0137] The parameter determination module is used to determine the first parameters of the projector and the second parameters of the camera based on the first grayscale image and the second grayscale image.

[0138] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0139] Reference Figure 6 This invention provides a camera-projector calibration device based on a color checkerboard, comprising:

[0140] At least one processor;

[0141] At least one memory for storing at least one program;

[0142] When the above-mentioned at least one program is executed by the above-mentioned at least one processor, the above-mentioned at least one processor implements the above-mentioned camera-projector calibration method based on a color chessboard.

[0143] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0144] This invention also provides a computer-readable storage medium storing a processor-executable program that, when executed by a processor, performs the aforementioned camera-projector calibration method based on a color checkerboard.

[0145] This invention provides a computer-readable storage medium that can execute a camera-projector calibration method based on a color chessboard provided in the method embodiments of this invention. It can execute any combination of the implementation steps of the method embodiments and has the corresponding functions and beneficial effects of the method.

[0146] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform... Figure 1 The method shown.

[0147] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0148] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0149] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0150] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0151] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or, if necessary, processing in other suitable ways, and then stored in computer memory.

[0152] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0153] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0154] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0155] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A camera-projector calibration method based on a color chessboard, characterized in that: The following steps are involved: Projecting a projection chessboard onto a reference plane pre-set with a physical chessboard by a projector to obtain a superimposed color chessboard, wherein a first color of the projection chessboard is different from a second color of the physical chessboard, and the color of the reference plane is white; Acquiring first image information of the superimposed color chessboard through a camera; determining a first color component and a second color component so that the first color has the first color component but does not have the second color component, and the second color has the second color component but does not have the first color component; Extracting a first grayscale image of a corresponding color channel from the first image information according to the first color component, and extracting a second grayscale image of a corresponding color channel from the first image information according to the second color component; Determine a first parameter of the projector and a second parameter of the camera according to the first grayscale image and the second grayscale image; The step of determining the first parameter of the projector and the second parameter of the camera according to the first grayscale image and the second grayscale image specifically includes: determining a structured light system according to the first grayscale image and the second grayscale image; Determine a first homography matrix and a second homography matrix according to the structured light system, wherein the first homography matrix represents a correspondence matrix between points in the image plane and points in the reference plane, and the second homography matrix represents a correspondence matrix between points in the sliding plane and points in the reference plane; Determine a second parameter of the camera according to the first homography matrix, and determine a first parameter of the projector according to the second homography matrix; The image plane represents the image plane captured by the camera, and the sliding plane represents the projection plane of the projector.

2. The camera-projector calibration method based on color chessboard according to claim 1, characterized in that: The first color is cyan, the second color is yellow, the first color component is a blue component, and the second color component is a red component.

3. The camera-projector calibration method based on color chessboard according to claim 1, characterized in that: The checkerboard pattern of the projected chessboard and the checkerboard pattern of the physical chessboard are both generated by the following formula: g x (x,y)=sin(πx), g y (x,y)=sin(πy), g r (x,y)=sin[2θ(x,y)]sin[2(n+1)θ(x,y)], θ(x,y)=arctan(y,x), The y-axis of the checkerboard pattern of the projected chessboard is set opposite to the y-axis of the checkerboard image of the physical chessboard, sign[·] represents a sign function, x and y represent the horizontal coordinate and vertical coordinate of a point in the checkerboard pattern, respectively, x∈[-n,n+2], y∈[-n,n+1], and n represents the number of reference lines.

4. The camera-projector calibration method based on color chessboard according to claim 1, characterized in that: The first homography matrix is ​​determined by the following formula: Where μ represents a point in the image plane, ρ represents a point in the reference plane, represents a homogeneous operator, express The inverse matrix, G i represents the first homography matrix; The second homography matrix is ​​determined by the following formula: G s =G -1 G i , Where ν represents a point in the sliding plane, G represents the third homography matrix, G -1 represents the inverse matrix of G, and the third homography matrix represents the correspondence matrix between the points in the image plane and the points in the sliding plane, G s represents the second homography matrix.

5. The camera-projector calibration method based on color chessboard according to claim 1, characterized in that: The first parameter includes a first intrinsic parameter and a first extrinsic parameter, the second parameter includes a second intrinsic parameter and a second extrinsic parameter, and the first parameter and the second parameter are calculated by a singular value decomposition method.

6. The camera-projector calibration method based on color chessboard according to claim 5, characterized in that: The first parameter and the second parameter are determined by the following formula: Among them, K s represents the first intrinsic parameter, K s -1 K s The inverse matrix, (R s,k , t s,k ) represents the first extrinsic parameter, K i represents the second intrinsic parameter, K i -1 K i The inverse matrix, (R i,k , t i,k ) represents the second extrinsic parameter, x(μ), y(μ) and z(μ) represent the x-coordinate, y-coordinate and z-coordinate of the point μ in the image plane corresponding to the point in the reference plane, represents the homogeneous operator, ν(μ) represents the point corresponding to the point μ in the image plane in the sliding plane, and λ i (μ) and λ s (μ) represents the scaling factor.

7. A camera-projector calibration system based on a color chessboard, characterized in that: include: A superimposed color chessboard projection module is used to project a projection chessboard onto a reference plane pre-set with a physical chessboard through a projector to obtain a superimposed color chessboard, wherein a first color of the projection chessboard is different from a second color of the physical chessboard, and the color of the reference plane is white; An image acquisition module, used for acquiring first image information of the superimposed color chessboard through a camera; A color component determination module, configured to determine a first color component and a second color component, so that the first color has the first color component but does not have the second color component, and the second color has the second color component but does not have the first color component; a grayscale image extraction module, configured to extract a first grayscale image of a corresponding color channel from the first image information according to the first color component, and to extract a second grayscale image of a corresponding color channel from the first image information according to the second color component; a parameter determination module, configured to determine a first parameter of the projector and a second parameter of the camera according to the first grayscale image and the second grayscale image; The step of determining the first parameter of the projector and the second parameter of the camera according to the first grayscale image and the second grayscale image specifically includes: determining a structured light system according to the first grayscale image and the second grayscale image; Determine a first homography matrix and a second homography matrix according to the structured light system, wherein the first homography matrix represents a correspondence matrix between points in the image plane and points in the reference plane, and the second homography matrix represents a correspondence matrix between points in the sliding plane and points in the reference plane; Determine a second parameter of the camera according to the first homography matrix, and determine a first parameter of the projector according to the second homography matrix; The image plane represents the image plane captured by the camera, and the sliding plane represents the projection plane of the projector.

8. A camera-projector calibration device based on a color chessboard, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the camera-projector calibration method based on the color chessboard as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to execute the camera-projector calibration method based on a color chessboard as described in any one of claims 1 to 6 when executed by the processor.

Images