Multi-projector calibration method, device and equipment and storage medium

Abstract

The invention discloses a multi-projector calibration method, device and equipment and a storage medium, which are applied to the field of projector calibration, and are characterized in that projectors are divided into a plurality of projector calibration groups according to the number of projector projection ranges which can be simultaneously covered by the view field range of a binocular vision system, and the binocular vision system is sequentially arranged in each projector calibration group; each projector projects a fixed calibration pattern, a binocular vision system shoots calibration images of the calibration patterns projected by the projectors to different attitude receiving plates, three-dimensional coordinates of feature points of the calibration images in a world coordinate system are determined, and calibration of a single projector is completed according to two-dimensional coordinates of the corresponding calibration patterns. And according to the transformation matrix between each projector and the binocular vision system, calculating the transformation matrix between the projectors, and completing multi-projector calibration. By realizing calibration of a plurality of projectors, the problem that only calibration of a single projector exists in the prior art and the single projector cannot realize ultra-large field-of-view imaging is solved.

Description

Technical Field

[0001] The present invention relates to the field of projector calibration, and in particular to a multi-projector calibration method, device, equipment and computer-readable storage medium. Background Art

[0002] With the development of projector-related technologies, the diversity and performance of projectors have reached unprecedented heights. It has become more convenient and easier to use projectors to design and implement various imaging systems, and it can ensure high imaging effects and imaging quality. Therefore, more and more imaging systems use projectors as the core device for projecting images. Due to the new demand for imaging systems to have ultra-large field of view, multi-viewing angles and other imaging functions, multi-projector imaging systems consisting of more than two projectors have emerged.

[0003] In order to achieve seamless connection between projectors, overlapping imaging, multi-viewing imaging, etc., and to achieve the best imaging effect, the multi-projector imaging system needs to be calibrated jointly to obtain the internal parameters, distortion coefficients, and relative position relationship of each projector. The current projector calibration is only for a single projector, and there is no calibration method for multiple projectors. Summary of the invention

[0004] The purpose of the present invention is to provide a multi-projector calibration method, device, equipment and computer-readable storage medium, which are applied to the field of projector calibration. The method proposes a multi-projector calibration method to realize the calibration of multiple projectors, thereby solving the problem that only a single projector can be calibrated in the prior art, and a single projector cannot achieve ultra-large field of view imaging.

[0005] In order to solve the above technical problems, the present invention provides a multi-projector calibration method, comprising:

[0006] Calibrate the binocular vision system, and group multiple projectors based on the field of view of the binocular vision system to obtain multiple projector groups; wherein the projectors in each of the projector groups are adjacent projectors, and there is a common projector between adjacent projector groups;

[0007] Determine a target projector group from the projector group, determine a target projector from the target projector group, and shoot a plurality of calibration images of the target projector projecting a calibration pattern onto a receiving plate with different postures through the binocular vision system;

[0008] Acquire the three-dimensional coordinates of the feature points in the calibration image in the world coordinate system, and calibrate the target projector based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points in the calibration pattern;

[0009] The target projector and the left eye camera form a temporary binocular vision system, and the temporary binocular vision system is calibrated to obtain a coordinate system transformation matrix of the target projector and the left eye camera;

[0010] Re-execute the step of determining the target projector from the target projector group until the coordinate system transformation matrix of the left-eye camera and all the projectors in the target projector group is obtained; re-execute the step of determining the target projector group from the projector group until the coordinate system transformation matrix of the left-eye camera and all the projectors is obtained;

[0011] Based on the coordinate system transformation matrix between the left-eye camera and all the projectors and the coordinate system transformation matrix between the common projector in different projector groups, the coordinate system transformation matrix between the multiple projectors is determined to complete the calibration of the multiple projectors.

[0012] Optionally, the calibrating the target projector based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern includes:

[0013] Setting the initial internal parameters and distortion coefficients of the target projector based on the theoretical values ​​of the projector parameters;

[0014] Determine a plurality of homography matrices based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern;

[0015] Determining a plurality of groups of initial external parameters based on the initial internal parameters and the homography matrix;

[0016] The Levenberg-Marquardt optimization algorithm is used to minimize the reprojection error, and the initial internal parameters, the distortion coefficients and the external parameters are globally optimized to complete the calibration of the target projector.

[0017] Optionally, the multiple projectors are grouped based on the field of view of the binocular vision system to obtain multiple projector groups, including:

[0018] Determining the maximum number of projectors that can be accommodated based on the field of view of the binocular vision system and the projection range of the projector;

[0019] The number of projectors in a group is determined based on the maximum number of projectors that can be accommodated, and the multiple projectors are grouped according to the number of projectors in the group to obtain a plurality of projector groups.

[0020] Optionally, the method further includes:

[0021] When the projection range of two adjacent projectors is larger than the field of view of the binocular vision system, an auxiliary projector is added between the two adjacent projectors so that the projection range of at least every two adjacent projectors can be within the field of view of the binocular vision system at the same time.

[0022] Optionally, obtaining three-dimensional coordinates of feature points in the calibration image in a world coordinate system includes:

[0023] Based on the binocular vision system, the feature points in the calibration image are identified, and according to the triangulation principle, the three-dimensional coordinates of the feature points to be converted in the binocular vision system coordinate system of the feature points in the calibration image are obtained;

[0024] The three-dimensional coordinates of the feature points to be converted are converted based on the coordinate system transformation matrix between the binocular vision system coordinate system and the world coordinate system to obtain the three-dimensional coordinates of the feature points in the world coordinate system.

[0025] Optionally, the calibrating the binocular vision system includes:

[0026] The binocular vision system is calibrated using Zhang Zhengyou calibration method;

[0027] Accordingly, the calibrating the temporary binocular vision system includes:

[0028] The temporary binocular vision system is calibrated using the Zhang Zhengyou calibration method.

[0029] Optionally, the world coordinate system is the left-eye camera coordinate system of the left-eye camera, and correspondingly, the binocular vision system coordinate system is the left-eye camera coordinate system.

[0030] In order to solve the above technical problems, the present invention provides a multi-projector calibration device, comprising:

[0031] The first module is used to calibrate the binocular vision system, and group multiple projectors based on the field of view of the binocular vision system to obtain multiple projector groups; wherein the projectors in each of the projector groups are adjacent projectors, and there is a common projector between adjacent projector groups;

[0032] The second module is used to determine a target projector group from the projector group, determine a target projector from the target projector group, and shoot a plurality of calibration images of the target projector projecting a calibration pattern onto a receiving plate with different postures through the binocular vision system;

[0033] The third module is used to obtain the three-dimensional coordinates of the feature points in the calibration image in the world coordinate system, and calibrate the target projector based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern;

[0034] The fourth module is used to form a temporary binocular vision system with the target projector and the left camera, calibrate the temporary binocular vision system, and obtain the coordinate system transformation matrix of the target projector and the left camera;

[0035] The fifth module is used to re-execute the step of determining the target projector from the target projector group until the coordinate system transformation matrix of the left eye camera and all the projectors in the target projector group is obtained; re-execute the step of determining the target projector group from the projector group until the coordinate system transformation matrix of the left eye camera and all the projectors is obtained;

[0036] The sixth module is used to determine the coordinate system transformation matrix between multiple projectors based on the coordinate system transformation matrix between the left-eye camera and all the projectors and the coordinate system transformation matrix between the common projector in different projector groups, so as to complete the calibration of multiple projectors.

[0037] In order to solve the above technical problems, the present invention provides a multi-projector calibration device, comprising:

[0038] Memory, for storing computer programs;

[0039] A processor is used to implement the multi-projector calibration method described above when executing the computer program.

[0040] In order to solve the above technical problem, the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the above-mentioned multi-projector calibration method is implemented.

[0041] It can be seen that the method of the present invention divides the projectors into multiple projector calibration groups according to the number of projector projection ranges that can be covered by the binocular vision system field of view at the same time, and arranges the binocular vision system in each projector calibration group in turn. Each projector projects a fixed calibration pattern, and the binocular vision system is used to shoot the calibration images of each projector projecting the calibration pattern onto the receiving board with different postures, and the three-dimensional coordinates of the feature points of the calibration image in the world coordinate system are determined, and the single projector calibration is completed according to the corresponding two-dimensional coordinates of the calibration pattern, and the transformation matrix between each projector is calculated according to the transformation matrix between each projector and the binocular vision system, and the multi-projector calibration is completed. By realizing the calibration of multiple projectors, the problem that only a single projector calibration exists in the prior art, and a single projector cannot achieve ultra-large field imaging is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0043] Figure 1 A flowchart of a multi-projector calibration method provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of a calibration pattern provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of a binocular vision system provided by an embodiment of the present invention arranged in a projector calibration group;

[0046] Figure 4 A schematic diagram of a binocular vision system provided by an embodiment of the present invention arranged in another projector calibration group;

[0047] Figure 5 A schematic diagram of adding an auxiliary projector for calibration provided by an embodiment of the present invention;

[0048] Figure 6 A structural block diagram of a multi-projector calibration device provided in an embodiment of the present invention.

[0049] Description of reference numerals:

[0050] 1- Binocular vision system; 2- First projector; 3- Second projector; 4- Third projector; 5- Field of view of binocular vision system; 6- Projection range of first projector; 7- Projection range of second projector; 8- Projection range of third projector; 9- Auxiliary projector; 10- Projection range of auxiliary projector. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] With the development of projector-related technologies, the diversity and performance of projectors have reached unprecedented heights. It has become more convenient and easier to use projectors to design and implement various imaging systems, and it can ensure higher imaging effects and quality. Therefore, more and more imaging systems use projectors as the core device for projecting images.

[0053] Due to the continuous improvement of the requirements for the use of various imaging systems in all walks of life, or the demand for the use of imaging systems in special application scenarios in certain fields, there have been some new demands for imaging systems with ultra-large field of view, multi-viewing angles and other imaging functions; obviously, to achieve ultra-large field of view imaging at a certain distance, the projection range of a single projector is limited, and it is difficult to cover the area of ​​ultra-large field of view imaging; this is even more true for multi-viewing angle imaging, a single projector can only provide one viewing angle, and each additional viewing angle requires an additional projector; therefore, a multi-projector imaging system consisting of two or more projectors has the ability to solve the above-mentioned demand problems.

[0054] In order to achieve seamless connection between projectors, overlapping imaging, multi-viewing imaging, etc., and to achieve the best imaging effect, the multi-projector imaging system needs to be calibrated jointly to obtain the internal parameters, distortion coefficients, and relative position relationship of each projector. The current projector calibration is only for a single projector, and there is no calibration method for multiple projectors.

[0055] The following combination Figure 1 , Figure 1 A flowchart of a multi-projector calibration method provided by an embodiment of the present invention includes:

[0056] S101: Calibrate the binocular vision system, and group multiple projectors based on the field of view of the binocular vision system to obtain multiple projector groups; wherein the projectors in each projector group are adjacent projectors, and there are common projectors between adjacent projector groups.

[0057] In this embodiment, the calibration of multiple projectors can be completed through a binocular vision system. The binocular vision system can include a left camera and a right camera. In this embodiment, the binocular vision system can be first calibrated to obtain the internal parameters and distortion coefficients of the left camera and the right camera, and the coordinate system transformation matrix between the left camera and the right camera.

[0058] This embodiment does not limit the specific method of calibrating the binocular vision system. Generally, the binocular vision system can be calibrated by Zhang Zhengyou calibration method.

[0059] In this embodiment, before performing multi-projector calibration, the multi-projectors can be grouped and grouped based on the field of view of the binocular vision system to obtain multiple projector groups; the projectors in each projector group are adjacent projectors, and there are common projectors between adjacent projector groups. This embodiment does not limit the number of common projectors between groups, which is generally one.

[0060] This embodiment does not limit the specific method of grouping projectors. Generally, the maximum number of projectors to be accommodated is determined based on the field of view of the binocular vision system and the projection range of the projector; the number of projectors in a group is determined based on the maximum number of projectors to be accommodated, and multiple projectors are grouped according to the number of projectors in the group to obtain multiple projector groups. Generally, the maximum number of projectors to be accommodated can be determined as the number of projectors in the group for grouping.

[0061] In the multi-projector calibration process, generally, the projection range of at least every two adjacent projectors among the multiple projectors can be within the field of view of the binocular vision system at the same time; therefore, in this embodiment, when the projection range of two adjacent projectors is larger than the field of view of the binocular vision system, an auxiliary projector is added between the two adjacent projectors so that the projection range of at least every two adjacent projectors can be within the field of view of the binocular vision system at the same time. This embodiment does not limit the number of auxiliary projectors to be added, which can be determined based on actual applications.

[0062] S102: determining a target projector group from the projector group, determining a target projector from the target projector group, and photographing a plurality of calibration images of the target projector projecting a calibration pattern onto a receiving plate with different postures through a binocular vision system.

[0063] In this embodiment, the projectors in each projector group can be calibrated in turn through the binocular vision system. First, the target projector group can be determined from the projector group, and the target projector can be determined from the target projector group. The binocular vision system is used to capture multiple calibration images of the target projector projecting a calibration pattern onto a receiving plate with different postures.

[0064] In this embodiment, the binocular vision system can be arranged near the target projector group. After the calibration pattern projected by the projector in the target projector group is received by the receiving board, the effective information of the calibration pattern is imaged within the field of view of the binocular vision system. When calibrating the projector in each target projector group, the position of the binocular vision system generally does not change.

[0065] The target projector in the target projector calibration group projects a fixed calibration pattern onto the receiving plate. The calibration pattern has multiple feature points with known two-dimensional coordinates, such as Figure 2As shown; the receiving plate is moved so that the receiving plate receives the calibration pattern projected by the target projector in different postures relative to the target projector, and a calibration image is formed on the receiving plate; each receiving plate posture corresponds to a calibration image, and this embodiment can sequentially capture the calibration images on the receiving plate in different postures through a binocular vision system.

[0066] S103: Acquire the three-dimensional coordinates of the feature points in the calibration image in the world coordinate system, and calibrate the target projector based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points in the calibration pattern.

[0067] This embodiment can obtain the three-dimensional coordinates of the feature points in the calibration image in the world coordinate system. This embodiment does not limit the specific method of obtaining the three-dimensional coordinates of the feature points. Generally, the feature points in the calibration image can be identified based on the binocular vision system, and the three-dimensional coordinates of the feature points to be converted in the binocular vision system coordinate system of the calibration image can be obtained according to the principle of triangulation; the three-dimensional coordinates of the feature points to be converted are converted based on the coordinate system transformation matrix between the binocular vision system coordinate system and the world coordinate system to obtain the three-dimensional coordinates of the feature points in the world coordinate system.

[0068] This embodiment does not limit the setting method of the world coordinate system. In order to reduce the amount of calculation, this embodiment can determine the left camera coordinate system of the left camera in the binocular vision system as the world coordinate system. Correspondingly, the left camera coordinate system is determined as the binocular vision system coordinate system. The three-dimensional coordinates of the feature points to be converted in the binocular vision system coordinate system can be directly determined as the three-dimensional coordinates of the feature points in the world coordinate system, without the need for conversion through the coordinate system transformation matrix.

[0069] The binocular vision system identifies the feature points in each calibration image and calculates the spatial three-dimensional coordinates of the feature points of the calibration image when the receiving board is in different positions based on the principle of triangulation, that is, the three-dimensional coordinates of the feature points in the world coordinate system.

[0070] This embodiment can determine the two-dimensional coordinates of the feature points corresponding to the feature points in the calibration pattern according to the characteristics of the feature points; under each calibration image, there is a correspondence between a set of two-dimensional coordinates of the feature points and the three-dimensional coordinates of the feature points. Every time a set of three-dimensional coordinates of the feature points is obtained, a correspondence between a set of three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points can be formed. Multiple sets of correspondences are formed under multiple different receiving plate postures. Therefore, this embodiment can obtain the correspondence between multiple sets of three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern.

[0071] This embodiment can calibrate the target projector based on multiple groups of correspondences. This embodiment does not limit the specific calibration method. Generally, the target projector can be calibrated by referring to the method of monocular camera calibration. Specifically, the initial internal parameters and distortion coefficients of the target projector are set based on the theoretical values ​​of the projector parameters; multiple homography matrices are determined based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern; multiple groups of initial external parameters are determined based on the initial internal parameters and the homography matrix; the Levenberg-Marquardt optimization algorithm is used to minimize the reprojection error, and the initial internal parameters, distortion coefficients and external parameters are globally optimized to complete the calibration of the target projector. The Levenberg-Marquardt algorithm is a widely used nonlinear least squares optimization algorithm. It combines the advantages of the Gauss-Newton method and the gradient descent method, and balances the two methods by introducing an adjustment parameter, so as to select the appropriate optimization strategy at different stages.

[0072] S104: The target projector and the left eye camera form a temporary binocular vision system, and the temporary binocular vision system is calibrated to obtain a coordinate system transformation matrix of the target projector and the left eye camera.

[0073] After completing the calibration of the target projector, the present embodiment can form a temporary binocular vision system with the target projector and the left-eye camera, calibrate the temporary binocular vision system, and obtain the coordinate system transformation matrix of the target projector and the left-eye camera. The present embodiment does not limit the specific method of calibrating the temporary binocular vision system. Accordingly, the present embodiment can use Zhang Zhengyou calibration method to calibrate the temporary binocular vision system.

[0074] S105: Re-execute the step of determining the target projector from the target projector group until the coordinate system transformation matrix of the left eye camera and all the projectors in the target projector group is obtained; re-execute the step of determining the target projector group from the projector group until the coordinate system transformation matrix of the left eye camera and all the projectors is obtained.

[0075] This embodiment can re-execute the step of determining the target projector from the target projector group until the coordinate system transformation matrix of the left eye camera and all projectors in the target projector group is obtained; re-execute the step of determining the target projector group from the projector group until the coordinate system transformation matrix of the left eye camera and all projectors is obtained.

[0076] After obtaining the coordinate system transformation matrix of the target projector and the left eye camera, that is, completing the calibration of the target projector and the left eye camera, this embodiment can re-determine the target projector from the target projector group, and perform a new calibration of the target projector and the left eye camera again to obtain the coordinate system transformation matrix between the new target projector and the left eye camera, until the coordinate system transformation matrix of all projectors in the left eye camera and the target projector group is obtained, that is, completing the calibration of all projectors in the left eye camera and the target projector group; in the process of re-determining the target projector, the calibrated projector is no longer used as the target projector.

[0077] Furthermore, after obtaining the coordinate system transformation matrix of the left-eye camera and all projectors in the target projector group, the present embodiment can re-determine the target projector group from the projector group, and calibrate the projectors in the new target projector group with the left-eye camera until the coordinate system transformation matrix of the left-eye camera and all projectors is obtained, that is, until the calibration of the left-eye camera and all projectors is completed.

[0078] S106: Determine the coordinate system transformation matrix between multiple projectors based on the coordinate system transformation matrix between the left-eye camera and all projectors and the coordinate system transformation matrix between the common projector and different projector groups, and complete the calibration of the multiple projectors.

[0079] This embodiment can determine the coordinate system transformation matrix between multiple projectors based on the coordinate system transformation matrix between the left-eye camera and all projectors and the coordinate system transformation matrix between the common projector and different projector groups, thereby completing the calibration of multiple projectors.

[0080] In this embodiment, the coordinate system transformation matrix between each projector in each projector group can be determined based on the coordinate system transformation matrix between the left-eye camera and all projectors. Specifically, if the projector group includes n projectors, the coordinate system transformation matrix between the left-eye camera coordinate system and any two projectors in the projector calibration group in the binocular vision system is respectively and , where the range of i and j is 1~n, and i and j are not equal, then the coordinate system transformation matrix between the j-th projector and the i-th projector is:

[0081] ;

[0082] In the formula, is the coordinate system transformation matrix between the j-th projector and the i-th projector, is the coordinate system transformation matrix between the left camera and the i-th projector, is the coordinate system transformation matrix between the left camera and the j-th projector.

[0083] Similarly, this embodiment can obtain the coordinate system transformation matrix of the public projector between different projector groups based on the coordinate system transformation matrix of the left-eye camera and the public projector. Using the coordinate system transformation matrix of the public projector between different projector groups as a bridge, the coordinate system transformation matrix of the projectors between different projector groups can be obtained, that is, the coordinate system transformation matrix of any two projectors among all projectors can be obtained.

[0084] Therefore, this embodiment can obtain the internal parameters and distortion coefficients of each projector, as well as the coordinate system transformation matrix between all projectors, to complete the multi-projector calibration.

[0085] Based on the above embodiments, the present invention proposes a multi-projector calibration method to achieve calibration of multiple projectors, solving the problem in the prior art that only a single projector can be calibrated but a single projector cannot achieve ultra-large field of view imaging.

[0086] Figure 3 and Figure 4 A schematic diagram of a binocular vision system provided by an embodiment of the present invention arranged in a projector calibration group is shown. In the schematic diagram, every two adjacent projectors can be divided into a group. The first projector 2 and the second projector 3 form a first projector group, and the second projector 3 and the third projector 4 form a second projector group. First, the binocular vision system 1 can be arranged in the first projector group, such as Figure 3 As shown, the binocular vision system field of view 5 of the binocular vision system 1 can simultaneously accommodate the first projector projection range 6 and the second projector projection range 7. After completing the calibration of the left-eye camera and the first projector 2 and the second projector 3, the binocular vision system can be arranged in the second projector group, such as Figure 4 As shown, the binocular vision system field of view 5 of the binocular vision system 1 can accommodate the second projector projection range 7 and the third projector projection range 8 at the same time.

[0087] Figure 5 A schematic diagram of adding an auxiliary projector for calibration provided in an embodiment of the present invention, if after the binocular vision system 1 is arranged in the first projector group, the binocular vision system field of view 5 of the binocular vision system 1 cannot accommodate the projection range 6 of the first projector and the projection range 7 of the second projector at the same time, then an auxiliary projector 9 can be added to the first projector 2 and the second projector 3, and the first projector group can be re-divided into a first-first projector group including the first projector 2 and the auxiliary projector 9 and a first-second projector group including the second projector 3 and the auxiliary projector 9. At this time, when the binocular vision system 1 is arranged in the first-first projector group or the first-second projector group, the binocular vision system field of view 5 of the binocular vision system 1 can accommodate the projection range 6 of the first projector and the projection range 10 of the auxiliary projector at the same time, or can accommodate the projection range 7 of the second projector and the projection range 10 of the auxiliary projector at the same time.

[0088] The following combination Figure 6 , FIG. 1 is a structural block diagram of a multi-projector calibration device provided by an embodiment of the present invention, and the device may include:

[0089] The first module 100 is used to calibrate the binocular vision system, and group multiple projectors based on the field of view of the binocular vision system to obtain multiple projector groups; wherein the projectors in each of the projector groups are adjacent projectors, and there is a common projector between adjacent projector groups;

[0090] The second module 200 is used to determine a target projector group from the projector group, determine a target projector from the target projector group, and photograph a plurality of calibration images of the target projector projecting a calibration pattern onto a receiving plate with different postures through the binocular vision system;

[0091] The third module 300 is used to obtain the three-dimensional coordinates of the feature points in the calibration image in the world coordinate system, and calibrate the target projector based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern;

[0092] The fourth module 400 is used to form a temporary binocular vision system with the target projector and the left-eye camera, calibrate the temporary binocular vision system, and obtain a coordinate system transformation matrix of the target projector and the left-eye camera;

[0093] The fifth module 500 is used to re-execute the step of determining the target projector from the target projector group until the coordinate system transformation matrix of the left eye camera and all the projectors in the target projector group is obtained; re-execute the step of determining the target projector group from the projector group until the coordinate system transformation matrix of the left eye camera and all the projectors is obtained;

[0094] The sixth module 600 is used to determine the coordinate system transformation matrix between multiple projectors based on the coordinate system transformation matrix between the left-eye camera and all the projectors and the coordinate system transformation matrix between the common projector in different projector groups, so as to complete the calibration of multiple projectors.

[0095] Based on the above embodiments, the present invention proposes a multi-projector calibration method to achieve calibration of multiple projectors, solving the problem in the prior art that only a single projector can be calibrated but a single projector cannot achieve ultra-large field of view imaging.

[0096] Based on the above embodiment, the third module 300 may include:

[0097] The first unit is used to set the initial internal parameters and distortion coefficients of the target projector based on the theoretical values ​​of the projector parameters;

[0098] A second unit is used to determine a plurality of homography matrices based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern;

[0099] A third unit is used to determine multiple groups of initial external parameters based on the initial internal parameters and the homography matrix;

[0100] The fourth unit is used to adopt the Levenberg-Marquardt optimization algorithm to minimize the reprojection error, perform global optimization on the initial internal parameters, the distortion coefficients and the external parameters, and complete the calibration of the target projector.

[0101] Based on the above embodiments, the first module 100 may include:

[0102] A fifth unit is used to determine a maximum number of projectors to be accommodated based on the field of view of the binocular vision system and the projection range of the projector;

[0103] The sixth unit is used to determine the number of projectors in the group based on the maximum number of projectors that can be accommodated, and group the multiple projectors according to the number of projectors in the group to obtain a plurality of projector groups.

[0104] Based on the above embodiments, the device may further include:

[0105] The seventh module is used to add an auxiliary projector between two adjacent projectors when the projection range of the two adjacent projectors is larger than the field of view of the binocular vision system, so that the projection range of at least every two adjacent projectors can be within the field of view of the binocular vision system at the same time.

[0106] Based on the above embodiments, obtaining the third module 300 may include:

[0107] A seventh unit is used to identify feature points in the calibration image based on the binocular vision system, and obtain three-dimensional coordinates of the feature points to be converted in the binocular vision system coordinate system of the feature points in the calibration image according to the triangulation principle;

[0108] The eighth unit is used to transform the three-dimensional coordinates of the feature points to be transformed based on the coordinate system transformation matrix between the binocular vision system coordinate system and the world coordinate system to obtain the three-dimensional coordinates of the feature points in the world coordinate system.

[0109] Based on the above embodiments, the calibrating of the binocular vision system includes:

[0110] A ninth unit is used for calibrating the binocular vision system by using Zhang Zhengyou calibration method;

[0111] Accordingly, the calibrating the temporary binocular vision system includes:

[0112] The tenth unit is used to calibrate the temporary binocular vision system by adopting the Zhang Zhengyou calibration method.

[0113] Based on the above embodiments, the world coordinate system is the left-eye camera coordinate system of the left-eye camera, and correspondingly, the binocular vision system coordinate system is the left-eye camera coordinate system.

[0114] Based on the above embodiments, the present invention further provides a multi-projector calibration device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiments may be implemented. Of course, the device may also include various necessary network interfaces, power supplies, and other components.

[0115] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by an execution terminal or a processor, the multi-projector calibration method provided in the embodiment of the present invention can be implemented; the storage medium may include: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0116] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. A multi-projector calibration method, characterized in that: include: Calibrate the binocular vision system, and group multiple projectors based on the field of view of the binocular vision system to obtain multiple projector groups; wherein the projectors in each of the projector groups are adjacent projectors, and there is a common projector between adjacent projector groups; Determine a target projector group from the projector group, determine a target projector from the target projector group, and shoot a plurality of calibration images of the target projector projecting a calibration pattern onto a receiving plate with different postures through the binocular vision system; Acquire the three-dimensional coordinates of the feature points in the calibration image in the world coordinate system, and calibrate the target projector based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points in the calibration pattern; The target projector and the left eye camera form a temporary binocular vision system, and the temporary binocular vision system is calibrated to obtain a coordinate system transformation matrix of the target projector and the left eye camera; Re-execute the step of determining the target projector from the target projector group until the coordinate system transformation matrix of the left-eye camera and all the projectors in the target projector group is obtained; re-execute the step of determining the target projector group from the projector group until the coordinate system transformation matrix of the left-eye camera and all the projectors is obtained; Based on the coordinate system transformation matrix of the left-eye camera and all the projectors and the coordinate system transformation matrix of the common projector between different projector groups, the coordinate system transformation matrix between the multiple projectors is determined to complete the calibration of the multiple projectors.

2. The multi-projector calibration method according to claim 1, characterized in that: The calibrating the target projector based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern includes: Setting the initial internal parameters and distortion coefficients of the target projector based on the theoretical values ​​of the projector parameters; Determine a plurality of homography matrices based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern; Determining a plurality of groups of initial external parameters based on the initial internal parameters and the homography matrix; The Levenberg-Marquardt optimization algorithm is used to minimize the reprojection error, and the initial internal parameters, the distortion coefficients and the external parameters are globally optimized to complete the calibration of the target projector.

3. The multi-projector calibration method according to claim 1, characterized in that: The multiple projectors are grouped based on the field of view of the binocular vision system to obtain multiple projector groups, including: Determining the maximum number of projectors that can be accommodated based on the field of view of the binocular vision system and the projection range of the projector; The number of projectors in a group is determined based on the maximum number of projectors that can be accommodated, and the multiple projectors are grouped according to the number of projectors in the group to obtain a plurality of projector groups.

4. The multi-projector calibration method according to claim 1, characterized in that: Also includes: When the projection range of two adjacent projectors is larger than the field of view of the binocular vision system, an auxiliary projector is added between the two adjacent projectors so that the projection range of at least every two adjacent projectors can be within the field of view of the binocular vision system at the same time.

5. The multi-projector calibration method according to claim 1, characterized in that: Obtaining the three-dimensional coordinates of the feature points in the calibration image in the world coordinate system includes: Based on the binocular vision system, the feature points in the calibration image are identified, and according to the triangulation principle, the three-dimensional coordinates of the feature points to be converted in the binocular vision system coordinate system of the feature points in the calibration image are obtained; The three-dimensional coordinates of the feature points to be converted are converted based on the coordinate system transformation matrix between the binocular vision system coordinate system and the world coordinate system to obtain the three-dimensional coordinates of the feature points in the world coordinate system.

6. The multi-projector calibration method according to claim 1, characterized in that: The calibrating of the binocular vision system comprises: The binocular vision system is calibrated using Zhang Zhengyou calibration method; Accordingly, the calibrating the temporary binocular vision system includes: The temporary binocular vision system is calibrated using the Zhang Zhengyou calibration method.

7. The multi-projector calibration method according to claim 1, characterized in that: The world coordinate system is the left eye camera coordinate system of the left eye camera, and correspondingly, the binocular vision system coordinate system is the left eye camera coordinate system.

8. A multi-projector calibration device, characterized in that: include: The first module is used to calibrate the binocular vision system, and group multiple projectors based on the field of view of the binocular vision system to obtain multiple projector groups; wherein the projectors in each of the projector groups are adjacent projectors, and there is a common projector between adjacent projector groups; The second module is used to determine a target projector group from the projector group, determine a target projector from the target projector group, and shoot a plurality of calibration images of the target projector projecting a calibration pattern onto a receiving plate with different postures through the binocular vision system; The third module is used to obtain the three-dimensional coordinates of the feature points in the calibration image in the world coordinate system, and calibrate the target projector based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern; The fourth module is used to form a temporary binocular vision system with the target projector and the left camera, calibrate the temporary binocular vision system, and obtain the coordinate system transformation matrix of the target projector and the left camera; The fifth module is used to re-execute the step of determining the target projector from the target projector group until the coordinate system transformation matrix of the left eye camera and all the projectors in the target projector group is obtained; re-execute the step of determining the target projector group from the projector group until the coordinate system transformation matrix of the left eye camera and all the projectors is obtained; The sixth module is used to determine the coordinate system transformation matrix between multiple projectors based on the coordinate system transformation matrix between the left-eye camera and all the projectors and the coordinate system transformation matrix between the common projector in different projector groups, so as to complete the calibration of multiple projectors.

9. A multi-projector calibration device, characterized in that: include: Memory, for storing computer programs; A processor, configured to implement the multi-projector calibration method as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the multi-projector calibration method according to any one of claims 1 to 7 is implemented.

Images