A multi-projector calibration method, device, equipment and storage medium

Abstract

The application discloses a kind of multi-projector calibration method, device, equipment and storage medium, applied to projector calibration field, according to the number of projector projection range that binocular vision system field of view range can cover simultaneously, it is divided into multiple projector calibration groups for projector, binocular vision system is sequentially arranged in each projector calibration group, each projector projects fixed calibration pattern, and the calibration image that each projector projects calibration pattern is photographed on different posture receiving plate by binocular vision system, the three-dimensional coordinates of the feature point world coordinate system of calibration image are determined, and according to corresponding map pattern two-dimensional coordinates, single projector calibration is completed, according to the transformation matrix between each projector and binocular vision system, the transformation matrix between each projector is calculated, and multi-projector calibration is completed. By realizing the calibration of multiple projectors, the problem that only single-projector calibration exists in the prior art and single-projector cannot realize super-large field of view imaging is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of projector calibration, in particular to a multi-projector calibration method, device, equipment and computer readable storage medium. BACKGROUND

[0002] With the development of projector related technology, the diversity and performance of the projector have reached an unprecedented height, and the use of projectors for various imaging system design and implementation has become more convenient and easy, and can ensure high imaging effect and imaging quality, therefore, more and more imaging systems use projectors as the core device of projecting images. Due to the new demand for imaging systems to have super large field of view, multi-view and other imaging functions, a multi-projector imaging system composed of two or more projectors appears.

[0003] In order to make the imaging seamless between projectors, or to realize overlapping cooperation imaging, and multi-view cooperation imaging, and to form the best imaging effect, the multi-projector imaging system needs to jointly calibrate the multiple projectors of the imaging system to obtain the respective internal parameters, distortion coefficients of the multiple projectors, and the relative position relationship between the projectors. The current projector calibration is only for the calibration of a single projector, and there is no multi-projector calibration method. SUMMARY

[0004] The purpose of the present application is to provide a multi-projector calibration method, device, equipment and computer readable storage medium, which is applied to the field of projector calibration, and the method realizes the calibration of multiple projectors by proposing a multi-projector calibration method, solving the problem that only single projector calibration exists in the prior art, and single projector cannot realize super large field of view imaging.

[0005] To solve the above technical problems, the present application provides a multi-projector calibration method, comprising:

[0006] Calibrate the binocular vision system, group the multiple projectors based on the field of view range of the binocular vision system, and obtain multiple projector groups; wherein the projectors in each projector group are adjacent projectors, and there are common projectors between adjacent projector groups;

[0007] Determine a target projector group from the projector groups, determine a target projector from the target projector group, and capture multiple calibration images of the target projector projecting a calibration pattern onto different posture receiving plates through the binocular vision system;

[0008] Obtain the three-dimensional coordinates of the feature points in the world coordinate system in the calibration images, and calibrate the target projector based on the corresponding relationship between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern;

[0009] composing a temporary binocular vision system with the target projector and the left eye camera, calibrating the temporary binocular vision system to obtain a coordinate system transformation matrix of the target projector and the left eye camera;

[0010] re-executing the step of determining the target projector from the target projector group until coordinate system transformation matrices of the left eye camera and all the projectors in the target projector group are obtained; and re-executing the step of determining the target projector group from the projector group until coordinate system transformation matrices of the left eye camera and all the projectors are obtained;

[0011] determining coordinate system transformation matrices among the multiple projectors based on the coordinate system transformation matrices of the left eye camera and all the projectors and the coordinate system transformation matrices of the common projector among different projector groups, to complete the calibration of the multiple projectors.

[0012] Optionally, the calibration of the target projector based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern comprises:

[0013] setting initial internal parameters and distortion coefficients of the target projector based on theoretical values of projector parameters;

[0014] determining a plurality of homography matrices based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern;

[0015] determining a plurality of groups of initial external parameters based on the initial internal parameters and the homography matrices;

[0016] performing global optimization on the initial internal parameters, the distortion coefficients and the external parameters by using a Levenberg-Marquardt optimization algorithm to minimize the re-projection error, to complete the calibration of the target projector.

[0017] Optionally, the grouping of the multiple projectors based on the field of view range of the binocular vision system to obtain a plurality of projector groups comprises:

[0018] determining a maximum number of accommodated projectors based on the field of view range of the binocular vision system and the projection range of the projector;

[0019] determining a number of projectors in a group based on the maximum number of accommodated projectors, grouping the multiple projectors according to the number of projectors in the group to obtain the plurality of projector groups.

[0020] Optionally, the method further comprises:

[0021] When the projection range of two adjacent projectors is greater than the field of view range of the binocular vision system, an auxiliary projector is added between the two adjacent projectors, so that the projection range of at least each adjacent projector can be in the field of view range of the binocular vision system at the same time.

[0022] Optionally, the feature point three-dimensional coordinates of the feature points in the calibration image in the world coordinate system are acquired, including:

[0023] The feature points in the calibration image are identified based on the binocular vision system, and the to-be-converted feature point three-dimensional coordinates of the feature points in the calibration image in the binocular vision system coordinate system are acquired according to the principle of triangulation;

[0024] The to-be-converted feature point three-dimensional coordinates are converted based on the coordinate system conversion matrix of the binocular vision system coordinate system and the world coordinate system, so as to obtain the feature point three-dimensional coordinates in the world coordinate system.

[0025] Optionally, the calibration of the binocular vision system includes:

[0026] The Zhang Zhengyou calibration method is used to calibrate the binocular vision system.

[0027] Correspondingly, the calibration of the temporary binocular vision system includes:

[0028] The Zhang Zhengyou calibration method is used to calibrate the temporary binocular vision system.

[0029] Optionally, the world coordinate system is a left camera coordinate system of the left camera, and correspondingly, the binocular vision system coordinate system is the left camera coordinate system.

[0030] To solve the above technical problems, the application provides a multi-projector calibration device, which comprises:

[0031] A first module is configured to calibrate a binocular vision system, group multiple projectors based on the field of view range of the binocular vision system, and obtain multiple projector groups; wherein the projectors in each projector group are adjacent projectors, and there are common projectors between adjacent projector groups.

[0032] A second module is configured to determine a target projector group from the projector groups, determine a target projector from the target projector group, and capture multiple calibration images of the target projector projecting a calibration pattern onto different posture receiving plates through the binocular vision system.

[0033] A third module is configured to acquire feature point three-dimensional coordinates of feature points in the calibration image in the world coordinate system, and calibrate the target projector based on the corresponding relationship between the feature point three-dimensional coordinates and the feature point two-dimensional coordinates of the calibration pattern.

[0034] a fourth module configured to form a temporary binocular vision system by combining the target projector and the left-eye camera, calibrate the temporary binocular vision system, and obtain a coordinate system transformation matrix of the target projector and the left-eye camera;

[0035] a fifth module configured to re-perform the step of determining the target projector from the target projector group until coordinate system transformation matrices of the left-eye camera and all the projectors in the target projector group are obtained, and re-perform the step of determining the target projector group from the projector group until coordinate system transformation matrices of the left-eye camera and all the projectors are obtained;

[0036] a sixth module configured to determine coordinate system transformation matrices among the multiple projectors based on the coordinate system transformation matrices of the left-eye camera and all the projectors and coordinate system transformation matrices of the common projector among different projector groups, and complete calibration of the multiple projectors.

[0037] To solve the above technical problems, the present application provides a multiple projector calibration device, which comprises:

[0038] a memory configured to store a computer program;

[0039] a processor configured to execute the computer program to implement the multiple projector calibration method.

[0040] To solve the above technical problems, the present application provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are executed by a processor to implement the multiple projector calibration method.

[0041] It can be seen that, in the method, the projectors are divided into multiple projector calibration groups according to the number of the projection ranges of the projectors that can be simultaneously covered by the field of view of the binocular vision system, the binocular vision system is arranged in each projector calibration group in turn, each projector projects a fixed calibration pattern, the binocular vision system captures calibration images of the calibration patterns projected by each projector onto different posture receiving plates, determines three-dimensional coordinates of feature points in a world coordinate system of the calibration images, and completes calibration of a single projector according to corresponding two-dimensional coordinates of the calibration patterns, calculates transformation matrices among the projectors according to the transformation matrices between each projector and the binocular vision system, and completes calibration of the multiple projectors. The calibration of multiple projectors is realized, and the problem that only single projector calibration exists in the prior art and single projector cannot realize super large field of view imaging is solved. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below only illustrate a part of the embodiments of the present application, and not all the embodiments. Based on the embodiments of the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the protection scope of the present application.

[0043] Figure 1 A flow chart of a multi-projector calibration method provided by an embodiment of the present application;

[0044] Figure 2 A calibration pattern schematic diagram provided by an embodiment of the present application;

[0045] Figure 3 A schematic diagram of a binocular vision system arranged in a projector calibration group provided by an embodiment of the present application;

[0046] Figure 4 A schematic diagram of a binocular vision system arranged in another group of projector calibration groups provided by an embodiment of the present application;

[0047] Figure 5 A schematic diagram of an auxiliary projector added for calibration provided by an embodiment of the present application;

[0048] Figure 6 A structural block diagram of a multi-projector calibration device provided by an embodiment of the present application.

[0049] Explanation of reference signs:

[0050] 1-binocular vision system; 2-first projector; 3-second projector; 4-third projector; 5-binocular vision system field of view range; 6-first projector projection range; 7-second projector projection range; 8-third projector projection range; 9-auxiliary projector; 10-auxiliary projector projection range. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only illustrate a part of the embodiments of the present application, and not all the embodiments. Based on the embodiments of the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the protection scope of the present application.

[0052] With the development of the projector related technology, the diversity and performance of the projector have reached an unprecedented height, and the use of the projector for various imaging system design and implementation becomes more convenient and easy, and can ensure high imaging effect and imaging quality, therefore, more and more imaging systems use the projector as the core device of the projected image.

[0053] Due to the continuous improvement of the use requirements of various industries for various imaging systems, or the use requirements of some fields for the imaging system in special application scenarios, some new requirements for the imaging system with super large field of view, multi-view angle and other imaging functions appear; obviously, in a certain distance, the projection range of a single projector is limited, and it is difficult to cover the area of super large field of view imaging; the multi-view angle imaging function is more so, a single projector can only provide one view angle, and each additional view angle requires an additional projector; therefore, the multi-projector imaging system composed of two or more projectors has the ability to solve the above problems.

[0054] In order to make the imaging of the projectors seamless, or realize the overlapping cooperation imaging, and the multi-view angle cooperation imaging, and form the best imaging effect, the multiple projectors of the imaging system need to be jointly calibrated to obtain the respective internal parameters, distortion coefficients of the multiple projectors, and the relative position relationship between the projectors. The current projector calibration is only for the calibration of a single projector, and there is no calibration method for multiple projectors.

[0055] The following will be described in detail Figure 1 , Figure 1 The flowchart of the multi-projector calibration method provided by the embodiment of the present application comprises:

[0056] S101: calibrate the binocular vision system, group the multiple projectors based on the field of view range of the binocular vision system, and obtain multiple projector groups; wherein the projectors in each projector group are adjacent projectors, and there are common projectors between adjacent projector groups.

[0057] The embodiment can complete the calibration of the multiple projectors through the binocular vision system, which can include a left eye camera and a right eye camera. The embodiment can first calibrate the binocular vision system to obtain the respective internal parameters, distortion coefficients of the left eye camera and the right eye camera, and the coordinate system transformation matrix between the left eye camera and the right eye camera.

[0058] The embodiment does not limit the specific way of calibrating the binocular vision system, which can generally be calibrated by Zhang Zhengyou calibration method.

[0059] The embodiment can group the multiple projectors before calibrating the multiple projectors. The multiple projectors are grouped based on the field of view range of the binocular vision system to obtain multiple projector groups. The projectors in each projector group are adjacent projectors, and there are common projectors between adjacent projector groups. The embodiment does not limit the number of common projectors between groups, which is generally one.

[0060] The embodiment does not limit the specific way of grouping projectors. Generally, the maximum number of accommodated projectors is determined based on the field of view range of the binocular vision system and the projection range of the projector. The number of projectors in a group is determined based on the maximum number of accommodated projectors, and the multiple projectors are grouped according to the number of projectors in a group to obtain multiple projector groups. Generally, the maximum number of accommodated projectors can be determined as the number of projectors in a group for grouping.

[0061] In the calibration process of the multiple projectors, the projection ranges of at least every two adjacent projectors in the multiple projectors can be in the field of view range of the binocular vision system at the same time. Therefore, when the projection ranges of the adjacent two projectors are greater than the field of view range of the binocular vision system in the embodiment, an auxiliary projector is added between the adjacent two projectors, so that the projection ranges of at least every two adjacent projectors can be in the field of view range of the binocular vision system at the same time. The embodiment does not limit the number of auxiliary projectors to be added, which can be determined based on actual application.

[0062] S102: Determine a target projector group from the projector groups, determine a target projector from the target projector group, and capture multiple calibration images of the target projector projecting a calibration pattern onto different posture receiving plates through the binocular vision system.

[0063] The embodiment can calibrate the projectors in each projector group through the binocular vision system. First, a target projector group is determined from the projector groups, a target projector is determined from the target projector group, and multiple calibration images of the target projector projecting a calibration pattern onto different posture receiving plates are captured through the binocular vision system.

[0064] The embodiment can arrange the binocular vision system near the target projector group. After the calibration pattern projected by the projectors in the target projector group is received by the receiving plate, the effective information of the calibration pattern is imaged in the field of view range of the binocular vision system. The position of the binocular vision system generally does not change when calibrating the projectors in each target projector group.

[0065] The target projectors in the target projector calibration group project fixed calibration patterns onto the receiving plate, and the calibration patterns have multiple feature points with known two-dimensional coordinates, such as Figure 2The receiving plates are moved so that the receiving plates receive the calibration patterns projected by the target projector to form calibration images on the receiving plates in different poses relative to the target projector. Each pose of the receiving plates corresponds to a calibration image. The calibration images in different poses can be captured by the binocular vision system.

[0066] S103: Obtain the feature point three-dimensional coordinates of the feature points in the calibration images in the world coordinate system, and calibrate the target projector based on the correspondence between the feature point three-dimensional coordinates and the feature point two-dimensional coordinates of the calibration patterns.

[0067] The embodiment can obtain the feature point three-dimensional coordinates of the feature points in the calibration images in the world coordinate system. The embodiment is not limited to the specific manner of obtaining the feature point three-dimensional coordinates. Generally, the feature points in the calibration images can be recognized based on the binocular vision system, and the feature point three-dimensional coordinates of the feature points in the calibration images in the binocular vision system coordinate system can be obtained according to the triangulation principle. The feature point three-dimensional coordinates to be converted in the binocular vision system coordinate system are converted based on the coordinate system conversion matrix between the binocular vision system coordinate system and the world coordinate system to obtain the feature point three-dimensional coordinates in the world coordinate system.

[0068] The embodiment is not limited to the setting manner of the world coordinate system. In order to reduce the calculation amount, the left camera coordinate system of the left camera in the binocular vision system can be determined as the world coordinate system. Correspondingly, the left camera coordinate system is determined as the binocular vision system coordinate system. Then, the feature point three-dimensional coordinates to be converted in the binocular vision system coordinate system can be directly determined as the feature point three-dimensional coordinates in the world coordinate system, without conversion through the coordinate system conversion matrix.

[0069] The binocular vision system recognizes the feature points in each calibration image, and calculates the spatial three-dimensional coordinates of the feature points in the calibration images when the receiving plates are in different pose states, that is, the feature point three-dimensional coordinates of the feature points in the world coordinate system according to the triangulation principle.

[0070] The embodiment can determine the feature point two-dimensional coordinates of the corresponding feature points in the calibration patterns according to the features of the feature points. Each calibration image has a corresponding relationship between a group of feature point two-dimensional coordinates and feature point three-dimensional coordinates. Each group of feature point three-dimensional coordinates can form a corresponding relationship between the feature point three-dimensional coordinates and the feature point two-dimensional coordinates. A plurality of different receiving plate pose states form a plurality of corresponding relationships. Therefore, the embodiment can obtain a plurality of corresponding relationships between the feature point three-dimensional coordinates and the feature point two-dimensional coordinates of the calibration patterns.

[0071] The embodiment can calibrate the target projector based on multiple sets of corresponding relationships. The embodiment is not limited to a specific calibration manner. Generally, the target projector can be calibrated by a monocular camera calibration method. Specifically, initial internal parameters and distortion coefficients of the target projector are set based on theoretical values of projector parameters; multiple homography matrices are determined based on corresponding relationships between three-dimensional coordinates of feature points and two-dimensional coordinates of feature points of a calibration pattern; multiple sets of initial external parameters are determined based on the initial internal parameters and the homography matrices; and the Levenberg-Marquardt optimization algorithm is used to globally optimize the initial internal parameters, the distortion coefficients, and the external parameters to minimize the reprojection error, thereby completing calibration of the target projector. The Levenberg-Marquardt algorithm is a widely used nonlinear least squares optimization algorithm. It combines the advantages of the Gauss-Newton method and the gradient descent method, balances the two methods by introducing an adjustment parameter, and thus selects a suitable optimization strategy at different stages.

[0072] S104: The target projector and the left-eye camera form a temporary binocular vision system, the temporary binocular vision system is calibrated, and a coordinate system transformation matrix of the target projector and the left-eye camera is obtained.

[0073] After the calibration of the target projector is completed, the embodiment can form a temporary binocular vision system with the target projector and the left-eye camera, calibrate the temporary binocular vision system, and obtain a coordinate system transformation matrix of the target projector and the left-eye camera. The embodiment is not limited to a specific manner of calibrating the temporary binocular vision system. Accordingly, the embodiment can calibrate the temporary binocular vision system by using the Zhang Zhengyou calibration method.

[0074] S105: The step of determining the target projector from the target projector group is re-executed until the coordinate system transformation matrix of the left-eye camera and all projectors in the target projector group is obtained; and the step of determining the target projector group from the projector group is re-executed until the coordinate system transformation matrix of the left-eye camera and all projectors is obtained.

[0075] The embodiment can re-execute the step of determining the target projector from the target projector group until the coordinate system transformation matrix of the left-eye camera and all projectors in the target projector group is obtained; and re-execute the step of determining the target projector group from the projector group until the coordinate system transformation matrix of the left-eye camera and all projectors is obtained.

[0076] After the coordinate system transformation matrix between the target projector and the left eye camera is obtained, i.e., after the calibration of the target projector and the left eye camera is completed, the target projector can be determined again from the target projector group, and the calibration of the new target projector and the left eye camera is performed again to obtain the coordinate system transformation matrix between the new target projector and the left eye camera, until the coordinate system transformation matrix between the left eye camera and all projectors in the target projector group is obtained, i.e., until the calibration of the left eye camera and all projectors in the target projector group is completed; in the process of determining the target projector again, the calibrated projector is no longer used as the target projector.

[0077] Further, after the coordinate system transformation matrix between the left eye camera and all projectors in the target projector group is obtained, the target projector group can be determined again from the projector group, and the calibration of the projectors in the new target projector group and the left eye camera is performed, until the coordinate system transformation matrix between the left eye camera and all projectors is obtained, i.e., until the calibration of the left eye camera and all projectors is completed.

[0078] S106: determining the coordinate system transformation matrix between the multiple projectors based on the coordinate system transformation matrix between the left eye camera and all projectors and the coordinate system transformation matrix between the common projector and different projector groups, and completing the calibration of the multiple projectors.

[0079] The embodiment can determine the coordinate system transformation matrix between the multiple projectors based on the coordinate system transformation matrix between the left eye camera and all projectors and the coordinate system transformation matrix between the common projector and different projector groups, and complete the calibration of the multiple projectors.

[0080] The embodiment can determine the coordinate system transformation matrix between each projector in each projector group based on the coordinate system transformation matrix between the left eye camera and all projectors. Specifically, if the projector group contains n projectors, the coordinate system transformation matrix between the left eye camera coordinate system in the binocular vision system and any two projectors in the calibration group is respectively and wherein the range of i and j is 1-n, and i and j are not equal, and the coordinate system transformation matrix between the jth projector and the ith projector is

[0081] ;

[0082] In the formula, is the coordinate system transformation matrix between the jth projector and the ith projector, is the coordinate system transformation matrix between the left eye camera and the ith projector, is the coordinate system transformation matrix between the left eye camera and the jth projector.

[0083] Similarly, the embodiment can obtain the coordinate transformation matrix of the common projector between different projector groups based on the coordinate transformation matrix of the left eye camera and the common projector, and can obtain the coordinate transformation matrix of the projectors between different projector groups by taking the coordinate transformation matrix of the common projector between different projector groups as a bridge, that is, obtain the coordinate transformation matrix of any two projectors in all projectors.

[0084] Therefore, the embodiment can obtain the self internal parameter and the distortion coefficient of each projector, and the coordinate transformation matrix between all projectors, and complete the multi-projector calibration.

[0085] Based on the above embodiment, the application provides a multi-projector calibration method to realize the calibration of multiple projectors, and solves the problem that only single-projector calibration exists in the prior art, and single-projector cannot realize super large field imaging.

[0086] Figure 3 and Figure 4 A schematic diagram of a binocular vision system provided by the embodiment of the application is arranged in a projector calibration group, in which every two adjacent projectors can be divided into a group, the first projector 2 and the second projector 3 form a first projector group, the second projector 3 and the third projector 4 form a second projector group, the binocular vision system 1 can be arranged in the first projector group first, as shown in FIG. 2, so that the binocular vision system field range 5 of the binocular vision system 1 can simultaneously accommodate the first projector projection range 6 and the second projector projection range 7. Figure 3 After the calibration of the left eye camera and the first projector 2 and the second projector 3 is completed, the binocular vision system can be arranged in the second projector group, as shown in FIG. 3, so that the binocular vision system field range 5 of the binocular vision system 1 can simultaneously accommodate the second projector projection range 7 and the third projector projection range 8. Figure 4 After the calibration of the left eye camera and the first projector 2 and the second projector 3 is completed, the binocular vision system can be arranged in the second projector group, as shown in FIG. 3, so that the binocular vision system field range 5 of the binocular vision system 1 can simultaneously accommodate the second projector projection range 7 and the third projector projection range 8.

[0087] Figure 5 If the binocular vision system 1 cannot simultaneously accommodate the first projector projection range 6 and the second projector projection range 7 after the binocular vision system 1 is arranged in the first projector group, an auxiliary projector 9 can be added to the first projector 2 and the second projector 3, and the first projector group can be re-divided into a first one projector group containing the first projector 2 and the auxiliary projector 9 and a first two projector group containing the second projector 3 and the auxiliary projector 9. At this time, the binocular vision system 1 can be arranged in the first one projector group or the first two projector group, so that the binocular vision system field range 5 of the binocular vision system 1 can simultaneously accommodate the first projector projection range 6 and the auxiliary projector projection range 10, or can simultaneously accommodate the second projector projection range 7 and the auxiliary projector projection range 10.

[0088] The application is described below Figure 6 , Fig. is a structural block diagram of a multi-projector calibration device provided by an embodiment of the application, which can include:

[0089] The first module 100 is configured to calibrate a binocular vision system, group multiple projectors based on a field of view range of the binocular vision system, and obtain multiple projector groups; wherein the projectors in each of the projector groups are adjacent projectors, and there are common projectors between adjacent projector groups;

[0090] The second module 200 is configured to determine a target projector group from the projector groups, determine a target projector from the target projector group, and capture multiple calibration images of the target projector projecting a calibration pattern onto different posture receiving plates through the binocular vision system;

[0091] The third module 300 is configured to obtain three-dimensional coordinates of feature points of the calibration images in a world coordinate system, and calibrate the target projector based on a corresponding relationship between the three-dimensional coordinates of the feature points and two-dimensional coordinates of feature points of the calibration pattern;

[0092] The fourth module 400 is configured to form a temporary binocular vision system with the target projector and a left-eye camera, calibrate the temporary binocular vision system, and obtain a coordinate system transformation matrix of the target projector and the left-eye camera;

[0093] The fifth module 500 is configured to re-perform the step of determining a target projector from the target projector group until the coordinate system transformation matrix of the left-eye camera and all the projectors in the target projector group is obtained, and re-perform the step of determining a target projector group from the projector groups until the coordinate system transformation matrix of the left-eye camera and all the projectors is obtained;

[0094] The sixth module 600 is configured to determine a coordinate system transformation matrix between multiple projectors based on the coordinate system transformation matrix of the left-eye camera and all the projectors and the coordinate system transformation matrix of the common projectors between different projector groups, and complete the calibration of the multiple projectors.

[0095] Based on the above embodiment, the application proposes a calibration method for multiple projectors to realize the calibration of multiple projectors, and solves the problem that only single-projector calibration exists in the prior art, and single-projector cannot realize super-large field imaging.

[0096] Based on the above embodiment, the third module 300 can include:

[0097] The first unit is configured to set initial internal parameters and distortion coefficients of a target projector based on a theoretical value of a projector parameter.

[0098] a second unit configured to determine a plurality of homography matrices based on a correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern;

[0099] a third unit configured to determine a plurality of sets of initial extrinsic parameters based on the initial intrinsic parameters and the homography matrices;

[0100] a fourth unit configured to globally optimize the initial intrinsic parameters, the distortion coefficients, and the extrinsic parameters by using a Levenberg-Marquardt optimization algorithm to minimize a re-projection error, thereby completing the calibration of the target projector.

[0101] According to the above embodiments, the first module 100 can include:

[0102] a fifth unit configured to determine a maximum number of projectors that can be accommodated based on a field of view range of the binocular vision system and a projection range of the projector;

[0103] a sixth unit configured to determine a number of projectors in a group based on the maximum number of projectors that can be accommodated, and group a plurality of projectors according to the number of projectors in the group, thereby obtaining a plurality of groups of projectors.

[0104] According to the above embodiments, the device can further include:

[0105] a seventh module configured to add an auxiliary projector between two adjacent projectors when the projection ranges of the two adjacent projectors are greater than the field of view range of the binocular vision system, so that the projection ranges of at least every two adjacent projectors can be simultaneously within the field of view range of the binocular vision system.

[0106] According to the above embodiments, the third module 300 can include:

[0107] a seventh unit configured to identify feature points in the calibration image based on the binocular vision system, and obtain three-dimensional coordinates of the feature points in the calibration image in a binocular vision system coordinate system according to a triangulation principle;

[0108] an eighth unit configured to convert the three-dimensional coordinates of the feature points in the binocular vision system coordinate system to the three-dimensional coordinates of the feature points in the world coordinate system based on a coordinate system conversion matrix between the binocular vision system coordinate system and the world coordinate system.

[0109] According to the above embodiments, the calibration of the binocular vision system includes:

[0110] a ninth unit configured to calibrate the binocular vision system by using Zhang Zhengyou's calibration method;

[0111] Correspondingly, the calibration of the temporary binocular vision system comprises:

[0112] The tenth unit is configured to calibrate the temporary binocular vision system by using the Zhang Zhengyou calibration method.

[0113] According to the above embodiments, the world coordinate system is a left camera coordinate system of the left camera, and correspondingly, the binocular vision system coordinate system is the left camera coordinate system.

[0114] According to the above embodiments, the present application further provides a multi-projector calibration device, which can comprise a memory and a processor, wherein the memory has a computer program stored therein, and the processor can realize the steps provided in the above embodiments when calling the computer program in the memory. Of course, the device can further comprise various necessary network interfaces, power supplies and other components.

[0115] The present application further provides a computer readable storage medium having a computer program stored thereon, and the computer program can realize the multi-projector calibration method provided in the embodiments of the present application when executed by a terminal or a processor. The storage medium can include a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media capable of storing program codes.

[0116] In this document, the terms "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A multi-projector calibration method, characterized in that: include: Calibrate the binocular vision system, and group multiple projectors based on the field of view of the binocular vision system to obtain multiple projector groups; wherein the projectors in each projector group are adjacent projectors, and there is a common projector between adjacent projector groups; Determining a target projector group from the projector group, determining a target projector from the target projector group, and photographing a plurality of calibration images of the target projector projecting a calibration pattern onto receiving plates with different postures through the binocular vision system; Acquire three-dimensional coordinates of feature points in the calibration image in a world coordinate system, and calibrate the target projector based on a correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points in the calibration pattern; The target projector and the left-eye camera form a temporary binocular vision system, and the temporary binocular vision system is calibrated to obtain a coordinate system transformation matrix of the target projector and the left-eye camera; Re-performing the step of determining the target projector from the target projector group until the coordinate system transformation matrix between the left-eye camera and all the projectors in the target projector group is obtained; re-performing the step of determining the target projector group from the projector group until the coordinate system transformation matrix between the left-eye camera and all the projectors is obtained; The coordinate system transformation matrix between multiple projectors is determined based on the coordinate system transformation matrix between the left-eye camera and all the projectors and the coordinate system transformation matrix between the common projector and different projector groups, thereby completing the calibration of the multiple projectors.

2. The multi-projector calibration method according to claim 1, characterized in that: The calibrating the target projector based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern includes: Set the initial internal parameters and distortion coefficients of the target projector based on the theoretical values ​​of the projector parameters; determining a plurality of homography matrices based on a correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern; Determining multiple sets of initial extrinsic parameters based on the initial intrinsic parameters and the homography matrix; The Levenberg-Marquardt optimization algorithm is used to minimize the reprojection error, and the initial internal parameters, the distortion coefficients and the external parameters are globally optimized to complete the calibration of the target projector.

3. The multi-projector calibration method according to claim 1, characterized in that: The multiple projectors are grouped based on the field of view of the binocular vision system to obtain multiple projector groups, including: Determining the maximum number of projectors that can be accommodated based on the field of view of the binocular vision system and the projection range of the projector; The number of projectors in a group is determined based on the maximum number of projectors that can be accommodated, and the multiple projectors are grouped according to the number of projectors in the group to obtain a plurality of projector groups.

4. The multi-projector calibration method according to claim 1, characterized in that: Also includes: When the projection range of two adjacent projectors is larger than the field of view of the binocular vision system, an auxiliary projector is added between the two adjacent projectors so that the projection range of at least every two adjacent projectors can be within the field of view of the binocular vision system at the same time.

5. The multi-projector calibration method according to claim 1, characterized in that: Obtaining the three-dimensional coordinates of the feature points in the calibration image in the world coordinate system includes: Identifying feature points in the calibration image based on the binocular vision system, and obtaining three-dimensional coordinates of the feature points to be converted in the binocular vision system coordinate system of the feature points in the calibration image according to the triangulation principle; The three-dimensional coordinates of the feature points to be converted are converted based on the coordinate system transformation matrix of the binocular vision system coordinate system and the world coordinate system to obtain the three-dimensional coordinates of the feature points in the world coordinate system.

6. The multi-projector calibration method according to claim 1, characterized in that: The calibrating of the binocular vision system includes: The binocular vision system is calibrated using Zhang Zhengyou calibration method; Accordingly, the calibrating the temporary binocular vision system includes: The temporary binocular vision system is calibrated using the Zhang Zhengyou calibration method.

7. The multi-projector calibration method according to claim 1, characterized in that: The world coordinate system is the left-eye camera coordinate system of the left-eye camera, and correspondingly, the binocular vision system coordinate system is the left-eye camera coordinate system.

8. A multi-projector calibration device, characterized in that: include: A first module is configured to calibrate a binocular vision system, group multiple projectors based on the field of view of the binocular vision system to obtain multiple projector groups; wherein the projectors in each projector group are adjacent projectors, and there is a common projector between adjacent projector groups; The second module is configured to determine a target projector group from the projector group, determine a target projector from the target projector group, and capture a plurality of calibration images of the target projector projecting a calibration pattern onto a receiving plate with different postures through the binocular vision system; A third module is used to obtain the three-dimensional coordinates of the feature points in the calibration image in the world coordinate system, and calibrate the target projector based on the correspondence between the three-dimensional coordinates of the feature points and the two-dimensional coordinates of the feature points of the calibration pattern; A fourth module is used to form a temporary binocular vision system with the target projector and the left-eye camera, calibrate the temporary binocular vision system, and obtain a coordinate system transformation matrix of the target projector and the left-eye camera; A fifth module is configured to re-execute the step of determining the target projector from the target projector group until the coordinate system transformation matrix between the left-eye camera and all the projectors in the target projector group is obtained; and re-execute the step of determining the target projector group from the projector group until the coordinate system transformation matrix between the left-eye camera and all the projectors is obtained. The sixth module is used to determine the coordinate system transformation matrix between multiple projectors based on the coordinate system transformation matrix of the left-eye camera and all the projectors and the coordinate system transformation matrix of the common projector between different projector groups to complete the multi-projector calibration.

9. A multi-projector calibration device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the multi-projector calibration method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the multi-projector calibration method according to any one of claims 1 to 7 is implemented.

Images