Multi-projection correction method and device based on binocular vision camera

Abstract

The invention relates to the field of projection correction, in particular to a multi-projection correction method and equipment based on a binocular vision camera. According to the method, the binocular vision camera is calibrated, the radial distortion and eccentric distortion of the camera are reduced, then the projector projects the coded structured light stripes, and after the binocular vision camera collects the deformed stripes on the projection screen, high-precision projection screen three-dimensional structure information can be reconstructed according to the phase relation. Therefore, a sub-pixel-level mapping relation of the screen-projector is established, and high-precision geometric correction is realized. According to the method, the multi-projection splicing geometric correction precision is greatly improved, the adaptability is wide, and the method has high application and popularization value.

Description

Technical Field

[0001] The present invention relates to the field of projection correction, and in particular to a multi-projection correction method and device based on a binocular vision camera. Background Art

[0002] With the development and progress of science and technology, information interaction through display terminals has received more and more attention. In the process of displaying images, in order to give users a better audio-visual experience, stereoscopic projection technology came into being. However, when the projection screen is relatively large, a single projector cannot meet the requirements of the projection screen, and multiple projectors are needed for splicing.

[0003] The existing multi-projection stitching and fusion methods mainly use a single camera to collect feature images. The images collected by a single camera lose spatial depth information, which makes it impossible to accurately adapt to projection screens of different shapes, thereby affecting the accuracy of correction and fusion.

[0004] Therefore, a more accurate multi-projection correction method and device are needed nowadays. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned deficiencies in the prior art and to provide a multi-projection correction method based on a binocular vision camera.

[0006] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions: A multi-projection correction method based on a binocular vision camera comprises the following steps: S1: Use two projectors to be calibrated to project horizontal phase-shifted structured light coded images and vertical phase-shifted structured light coded images onto a projection screen, and use two sets of binocular vision cameras to collect them respectively to obtain several sets of feature images; S2: reconstructing the projection screen in three dimensions according to the characteristic image, and outputting three-dimensional reconstruction data of the projection screen; S3: transferring the three-dimensional reconstruction data of the projection screen of the two projectors to be calibrated to the set point cloud image unified coordinate system, completing the complete point cloud alignment, and outputting the aligned point cloud image; S4: converting the aligned point cloud image from the point cloud image unified coordinate system to the spherical coordinate system, and calculating the longitude and latitude data corresponding to each pixel point in each frame buffer image of the projector to be corrected; S5: Calculate the perspective projection frustum parameters of the projector to be corrected according to the longitude and latitude data, convert the frame buffer image into a pre-deformed image according to the perspective projection frustum parameters, and complete the geometric correction.

[0007] As a preferred solution of the present invention, S1 further includes: Calibrate two sets of binocular vision cameras; The camera coordinate system of one set of binocular vision cameras is selected as the unified coordinate system of the point cloud image, and the feature image is synchronously collected by another set of binocular vision cameras.

[0008] As a preferred embodiment of the present invention, S2 comprises the following steps: The characteristic images corresponding to the two groups of binocular vision cameras are calculated respectively by using a structured light measurement method to obtain horizontal phase data and vertical phase data; Generate corresponding points of binocular stereo matching according to points with the same phase value in the horizontal phase data and the vertical phase data; Calculating the parallax value of the projection screen according to the corresponding points, and solving the spatial coordinates of the projection screen according to the parallax value; The calculation formula of phase is: , , Where φ(x, y) is the phase information, I i (x, y) is the phase data of the i-th phase shift, A(x, y) is the background light intensity, and B(x, y) is the modulation factor; The calculation formula of spatial coordinates is: , , , d=u L -u R , Where X, Y, and Z are the three-dimensional coordinate data of the spatial coordinates, B is the baseline distance between the lens centers of the left and right binocular vision cameras, f is the principal distance of the camera lens, (u L , v L )(u R , v R ) are the phase data of the current corresponding point combination, is the coordinate of the principal point after camera calibration, and d is the parallax value.

[0009] As a preferred embodiment of the present invention, S3 comprises the following steps: S31: randomly selecting a point in the projection overlap area of ​​the projection screen, and calculating its coordinates in the camera coordinate systems of the two sets of binocular vision cameras according to the horizontal and vertical phase data corresponding to the selected point; S32: establishing a conversion relationship according to the coordinates of the selected points in the camera coordinate systems of the two sets of binocular vision cameras, and calculating coordinate conversion parameters; S33: According to the coordinate conversion parameters, the three-dimensional reconstructed data is converted to a set point cloud image unified coordinate system, and an aligned point cloud image is output.

[0010] As a preferred embodiment of the present invention, the expression of the conversion relation in S32 is: , Among them, the coordinate transformation parameters include the rotation matrix R and the translation variable T, is the coordinate of point i in the camera coordinate system of the binocular vision camera that is not selected as the unified coordinate system of the point cloud image, is the coordinate of point i in the unified coordinate system of the point cloud image.

[0011] As a preferred embodiment of the present invention, S4 comprises the following steps: S41: Obtaining the coordinates of each pixel point in each frame buffer image of the projector to be calibrated in the unified coordinate system of the point cloud image; S42: Convert the unified coordinate system of the point cloud image to the spherical coordinate system according to the coordinate conversion formula, and output the longitude and latitude data corresponding to each pixel point.

[0012] As a preferred solution of the present invention, the coordinate conversion formula in S42 is expressed as: , Among them, Lo is the longitude, La is the latitude, (x, y, z) is the coordinate of the pixel point, and R is the radius of the sphere in the spherical coordinate system.

[0013] As a preferred embodiment of the present invention, S5 comprises the following steps: S51: Calculating the perspective projection cone parameters of the projector to be calibrated according to the longitude and latitude data; the perspective projection cone parameters include up, down, left, right, and right viewing angles and an hpr deflection angle; S52: converting the longitude and latitude data corresponding to each pixel of the frame buffer image into texture coordinates according to the perspective projection frustum parameters to generate a pre-deformed image; S53: The projector to be corrected projects the pre-deformed image to complete the geometric correction.

[0014] As a preferred solution of the present invention, the calculation formula of the perspective projection frustum parameters is: , , , in, is the longitude range of the projector to be calibrated, is the latitude range of the projector to be calibrated, and Respectively represent the left and right viewing angles of the projector to be calibrated, and Respectively represent the upper and lower viewing angles of the projector to be calibrated, is the heading rotation offset angle.

[0015] A multi-projection correction device based on a binocular vision camera comprises at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the methods described above.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention calibrates the binocular vision camera to reduce the radial distortion and eccentric distortion of the camera, and then the projector projects the coded structured light stripes. After the deformed stripes on the projection screen are collected by the binocular vision camera, the high-precision three-dimensional structure information of the projection screen can be reconstructed according to the phase relationship, thereby establishing a sub-pixel mapping relationship between the screen and the projector, and realizing high-precision geometric correction. The method of the present invention has greatly improved the accuracy of multi-projection splicing geometric correction, and has wide adaptability and high promotion and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a multi-projection correction method based on a binocular vision camera according to Embodiment 1 of the present invention; FIG2 (a) is a schematic diagram of the projection and shooting areas of a dual-channel projector in a multi-projection correction method based on a binocular vision camera according to Embodiment 2 of the present invention; FIG2 (b) is a schematic diagram of the projection and shooting area of ​​a binocular vision camera in a multi-projection correction method based on a binocular vision camera according to Embodiment 2 of the present invention; Figure 3 A flowchart of three-dimensional reconstruction in a multi-projection correction method based on a binocular vision camera according to Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of the projector P1 projecting and the binocular vision cameras C1 and C2 collecting images in a multi-projection correction method based on binocular vision cameras according to Embodiment 2 of the present invention; Figure 5 A vertical phase and horizontal phase grayscale distribution diagram in a multi-projection correction method based on a binocular vision camera described in Example 2 of the present invention; Figure 6 A spherical definition diagram of longitude and latitude in a multi-projection correction method based on a binocular vision camera described in Example 2 of the present invention; Figure 7This is a structural schematic diagram of a multi-projection correction device based on a binocular vision camera described in Example 3 of the present invention, which utilizes the multi-projection correction method based on a binocular vision camera described in the previous embodiment. DETAILED DESCRIPTION

[0018] The present invention is further described in detail below in conjunction with test examples and specific implementation methods. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0019] Example 1 like Figure 1 As shown, a multi-projection correction method based on a binocular vision camera includes the following steps: S1: Use two projectors to be calibrated to project horizontal phase-shifted structured light coded images and vertical phase-shifted structured light coded images onto a projection screen, and use two groups of binocular vision cameras to collect them respectively to obtain several groups of feature images.

[0020] S2: Perform three-dimensional reconstruction of the projection screen according to the characteristic images, calculate the horizontal and vertical phase data corresponding to each group of characteristic images, and output the three-dimensional reconstruction data on the projection screen.

[0021] S3: Transfer the 3D reconstruction data of the two projectors to be calibrated on the projection screen to the set point cloud image unified coordinate system, complete the complete point cloud alignment, and output the aligned point cloud image.

[0022] S4: converting the aligned point cloud image from the point cloud image unified coordinate system to the spherical coordinate system, and calculating the longitude and latitude data corresponding to each pixel point in each frame buffer image of the projector to be calibrated.

[0023] S5: Calculate the perspective projection frustum parameters of the projector to be corrected according to the longitude and latitude data, convert the frame buffer image into a pre-deformed image according to the perspective projection frustum parameters, and complete the geometric correction.

[0024] Example 2 This embodiment is a specific implementation of the multi-projection correction method based on a binocular vision camera described in Embodiment 1, comprising the following steps: S1: Use two projectors to be calibrated to project horizontal phase-shifted structured light coded images and vertical phase-shifted structured light coded images onto a projection screen, and use two groups of binocular vision cameras to collect them respectively to obtain several groups of feature images.

[0025] Among them, S1 also includes calibrating two groups of binocular vision cameras; and selecting the camera coordinate system of one group of binocular vision cameras as the unified coordinate system of the point cloud image, and synchronously collecting feature images through another group of binocular vision cameras.

[0026] Binocular vision camera is a binocular camera that simulates the human eye. There is parallax between the two eyes of a person, and this parallax can form stereoscopic vision after being synthesized in the brain. In the usual sense, binocular vision theory is a method in which a binocular camera takes images from different positions, and then uses the parallax principle to calculate the position deviation between corresponding points of the image, thereby obtaining the stereoscopic geometric information of the object. Binocular vision camera calibration includes two aspects: one is the calibration of each camera, that is, single-target calibration, and the other is the calibration of the relative position between the two cameras. Only when the relative position relationship between the two cameras is determined can the distance calculation be performed to complete the three-dimensional reconstruction. This embodiment uses calibration objects such as a chessboard to shoot multiple groups of binocular camera images at different angles to calibrate the binocular vision cameras C1 and C2.

[0027] The existing multi-projection stitching and fusion methods mainly use a single camera to collect feature images. The images collected by a single camera lose spatial depth information, which makes it impossible to accurately adapt to projection screens of different shapes, thereby affecting the accuracy of correction and fusion. The present invention adopts a binocular stereo vision camera to better solve this problem, which mainly applies the principle of triangulation. For the same object in space, due to the position deviation between the two cameras, there will also be deviations in imaging. Through camera calibration and polar constraint relationship, the corresponding relationship between the two cameras can be established. The pixel coordinate position of the object to be measured in the two cameras can be obtained through the collected coded image, and the spatial coordinates of the object can be obtained according to the principle of triangulation, thereby realizing three-dimensional reconstruction of the space.

[0028] As shown in Figure 2 (a), projectors P1 and P2 are used to project two sets of horizontal and vertical phase-shifted structured light coded images onto the screen, and then binocular vision cameras C1 and C2 are used to collect each set of fringe images, as shown in Figure 2 (b). At the same time, one set of camera coordinate systems is set as the unified coordinate system of the point cloud image, and the other corresponding projector is used to project the fringe pattern, and the camera group collects images synchronously.

[0029] S2: Perform three-dimensional reconstruction of the projection screen according to the characteristic images, calculate the horizontal and vertical phase data corresponding to each group of characteristic images, and output the three-dimensional reconstruction data on the projection screen.

[0030] The present invention uses phase-coded fringe projection to achieve high-quality measurement accuracy, fast measurement speed, sufficiently high point density, and low cost. The core of binocular stereo vision is to solve the image parallax of the same object being measured by two cameras, and solve the object's spatial coordinates based on the parallax value. Combining the binocular stereo vision method with the three-frequency four-step phase-shift structured light encoding method to complete three-dimensional reconstruction can obtain the precise three-dimensional coordinates of the target object.

[0031] like Figure 3As shown, the 3D reconstruction includes the following steps: a. The characteristic image is calculated by using the structured light measurement method to obtain the phase information corresponding to the two sets of binocular vision cameras; after phase unwrapping in this step, the horizontal phase and vertical phase According to this parameter, the three-dimensional coordinates can be solved by combining the calibration parameters (projector-binocular vision camera relationship) .

[0032] The process of solving the horizontal and vertical phases is similar. Here we take the horizontal phase solution as an example. For each projector, 12 frames of encoded horizontal structured light stripes are projected. These 12 frames of stripes are divided into 3 groups. The stripes in different groups have different frequencies. The stripes in the same group are phase shifted in 4 steps in sequence ( ), the calculation formula of phase information is: , , Where φ(x, y) is the phase information, I i (x, y) is the phase data of the i-th phase shift, A(x, y) is the background light intensity, and B(x, y) is the modulation factor.

[0033] b. Generate corresponding points for binocular stereo matching based on the points with the same phase value in the two sets of phase information.

[0034] After calculating the horizontal and vertical absolute phase values ​​of each pixel on the camera, the phase consistency principle is used to match the corresponding points of the left and right cameras. The process is as follows: Set a pixel point on the left camera The absolute phase value is and , find the point in the right camera where the phase value is equal to this phase value These two points are used as the corresponding points of binocular stereo matching.

[0035] c. Calculate the disparity value of the projection screen according to the corresponding points, and solve the spatial coordinates of the projection screen according to the disparity value, that is, use the triangulation principle, combine the above-mentioned matching pixel pairs and calibration parameters to calculate its three-dimensional coordinates.

[0036] The calculation formula of the spatial coordinates is: , , , d=u L -u R , Where X, Y, and Z are the three-dimensional coordinate data of the spatial coordinates, B is the baseline distance between the lens centers of the left and right binocular vision cameras, f is the principal distance of the camera lens, (u L , v L )(u R , v R ) are the phase data of the current corresponding point combination, is the coordinate of the principal point after camera calibration, and d is the parallax value.

[0037] S3: Transfer the 3D reconstruction data of the two projectors to be calibrated on the projection screen to the set point cloud image unified coordinate system, complete the complete point cloud alignment, and output the aligned point cloud image.

[0038] S31: randomly selecting a point in the projection overlap area of ​​the projection screen, and calculating its coordinates in the camera coordinate systems of the two sets of binocular vision cameras according to the horizontal and vertical phase data corresponding to the selected point; S32: establishing a conversion relationship according to the coordinates of the selected points in the camera coordinate systems of the two sets of binocular vision cameras, and calculating coordinate conversion parameters; The expression of the conversion relation is: , Among them, the coordinate transformation parameters include the rotation matrix R and the translation variable T, is the coordinate of point i in the camera coordinate system of the binocular vision camera that is not selected as the unified coordinate system of the point cloud image, is the coordinate of point i in the unified coordinate system of the point cloud image.

[0039] S33: According to the coordinate conversion parameters, the three-dimensional reconstructed data is converted to a set point cloud image unified coordinate system, and an aligned point cloud image is output.

[0040] Furthermore, based on the above-mentioned 3D reconstruction method, this embodiment can solve the spatial coordinates of the projection surface when the binocular vision cameras are placed at two different positions and the corresponding horizontal phase and vertical phase of the left and right cameras. The core goal of point cloud alignment is to transform the spatial coordinate points of projectors P1 and P2 into the same coordinate system, and use the left camera coordinate system of the binocular vision camera as the alignment coordinate system of the point cloud. Since the transformation relationship between the two binocular vision cameras C1 and C2 cannot be obtained from the outside world when they are located at two different positions, the structured light feature fringe image is studied here. Figure 4 As shown, due to the overlapping area between two adjacent projectors, when projector P1 projects horizontal and vertical structured light fringe patterns, only part of the image projected by projector P1 can be captured by using binocular vision cameras C1 and C2 to capture images, while C1 in the binocular vision camera can capture the complete image projected by P1.

[0041] Here, the three-frequency four-step phase shift method is used to solve the absolute phase of each group of collected images in the horizontal and vertical directions. If there is a point P projected on the screen by projector P1, then the position of this point in the phase distribution diagram is as follows: Figure 5 shown.

[0042] The horizontal phase and vertical phase represent the directions of two coordinate axes, which intersect at a unique point in the image. Then the phase and the coordinates of the only spatial point in the three-dimensional space establish a mapping relationship, that is, , Then, using the interpolation function, we can Find the phase pair The corresponding spatial coordinate point is express.

[0043] and Represents the point cloud coordinates of the same target object in different coordinate systems. There are N sets of such spatial corresponding points. The relationship between the two is as follows: , Where R is the rotation matrix and T is the translation variable. These two parameters represent the positional relationship of points in three-dimensional space. 21 ,Y 21 ,Z 21 ,1]T and [X temp ,Y temp ,Z temp ,1] T are all 4×N matrices. Using the R and T parameters, (X 21 ,Y 21 ,Z 21 ) to (X 22 ,Y 22 ,Z 22 ) coordinate system to complete the point cloud alignment.

[0044] S4: converting the aligned point cloud image from the point cloud image unified coordinate system to the spherical coordinate system, and calculating the longitude and latitude data corresponding to each pixel point in each frame buffer image of the projector to be calibrated.

[0045] S41: Obtaining the coordinates of each pixel point in each frame buffer image of the projector to be calibrated in the unified coordinate system of the point cloud image; S42: Convert the unified coordinate system of the point cloud image to the spherical coordinate system according to the coordinate conversion formula, and output the longitude and latitude data corresponding to each pixel point.

[0046] The coordinate transformation formula is: , Among them, Lo is the longitude, La is the latitude, (x, y, z) is the coordinate of the pixel point, and R is the radius of the sphere in the spherical coordinate system.

[0047] In this embodiment, if Figure 6 As shown, longitude and latitude form a geographic coordinate system, which is used in spherical mapping.

[0048] If the coordinates of a point on the sphere are (x, y, z) and the radius of the sphere is R, then the following formula can be used to solve the corresponding longitude and latitude values. Longitude is represented by Lo and latitude is represented by La.

[0049] , Known projectors : , Projector : The spatial coordinate points can be converted from Cartesian coordinates to spherical coordinates to obtain the longitude and latitude values ​​of each point. However, what needs to be obtained is the longitude and latitude values ​​corresponding to each projector pixel point so that the image projected by the projector can be geometrically aligned.

[0050] With projector : For example, its latitude and longitude values ​​are expressed as: According to the corresponding relationship: The projector projects a phase-shifted fringe pattern, and the phase value of each pixel point at the projector Known. For a group Perform surface fitting on the corresponding values ​​and insert The longitude and latitude values ​​corresponding to each projector pixel can be obtained. : Similarly, the longitude and latitude values ​​corresponding to each frame buffer pixel in the two channel projectors are obtained.

[0051] S5: Calculate the perspective projection frustum parameters of the projector to be corrected according to the longitude and latitude data, convert the frame buffer image into a pre-deformed image according to the perspective projection frustum parameters, and complete the geometric correction.

[0052] S51: Calculate the perspective projection cone parameters of the projector to be calibrated according to the longitude and latitude data; the perspective projection cone parameters include up, down, left, right viewing angles and hpr deflection angles; after the longitude and latitude values ​​corresponding to each pixel point on the projector, generate seven parameters of the cone according to computer graphics, namely, up, down, left, right viewing angle values ​​and H (heading), P (pitch), R (roll) deflection angles, so as to guide the scene to be rendered according to the cone parameters; the calculation formula of the perspective projection cone parameters is: , , , in, is the longitude range of the projector to be calibrated, is the latitude range of the projector to be calibrated, and Respectively represent the left and right viewing angles of the projector to be calibrated, and Respectively represent the upper and lower viewing angles of the projector to be calibrated, is the heading rotation offset angle, and the other two deflection angles and Usually set to 0, where is the pitch rotation offset angle, is the roll angle.

[0053] S52: converting the longitude and latitude data corresponding to each pixel of the frame buffer image into texture coordinates according to the perspective projection frustum parameters to generate a pre-deformed image; S53: The projector to be corrected projects the pre-deformed image to complete the geometric correction.

[0054] Example 3 like Figure 7 As shown, a multi-projection correction device based on a binocular vision camera includes at least one processor, a memory connected to the at least one processor, and at least one input-output interface connected to the at least one processor; the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the multi-projection correction method based on a binocular vision camera described in the above embodiment. The input-output interface may include a display, a keyboard, a mouse, and a USB interface for inputting and outputting data.

[0055] Furthermore, the multi-projection correction device based on a binocular vision camera can be a desktop, mobile phone, tablet computer, wearable multi-projection correction device based on a binocular vision camera, etc., which can perform depth information recognition.

[0056] Furthermore, the processor may include one or more processing cores. The processor uses various interfaces and lines to connect the various parts of the multi-projection correction device based on the binocular vision camera, and executes various functions and processes data of the multi-projection correction device based on the binocular vision camera by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but may be implemented separately through a communication chip.

[0057] The memory may include a random access memory (RAM) or a read-only memory (ROM). The memory may be used to store instructions, programs, codes, code sets or instruction sets, such as instructions or code sets for implementing a multi-projection correction method based on a binocular vision camera provided in an embodiment of the present application. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also be based on data created during use by the multi-projection correction device of the binocular vision camera (such as a mapping table of modulation sequence and depth, image data, spectrum diagram data), etc.

[0058] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), disks or optical disks, etc. Various media that can store program codes.

[0059] When the above-mentioned integrated unit of the present invention is implemented in the form of a software functional unit and sold or used as an independent product, it can also be stored in a computer-readable storage medium, in which a program code is stored, and the program code can be called by a processor to execute the method described in the above method embodiment. Based on such an understanding, the technical solution of the embodiment of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: electronic storage such as flash memory, EEPROM (electrically erasable programmable read-only memory), EPROM, hard disk or ROM. Optionally, the computer-readable storage medium includes a non-transitory computer-readable storage medium. The computer-readable storage medium has a storage space for the program code that executes any method step in the above method. These program codes can be read from or written to one or more computer program products. And the program code can be compressed, for example, in an appropriate form.

[0060] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-projection correction method based on a binocular vision camera, characterized in that: The following steps are involved: S1: Use two projectors to be calibrated to project horizontal phase-shifted structured light coded images and vertical phase-shifted structured light coded images onto a projection screen, and use two sets of binocular vision cameras to collect them respectively to obtain several sets of feature images; S2: reconstructing the projection screen in three dimensions according to the characteristic image, and outputting three-dimensional reconstruction data of the projection screen; S3: transferring the three-dimensional reconstruction data of the projection screen of the two projectors to be calibrated to the set point cloud image unified coordinate system, completing the complete point cloud alignment, and outputting the aligned point cloud image; S4: converting the aligned point cloud image from the point cloud image unified coordinate system to the spherical coordinate system, and calculating the longitude and latitude data corresponding to each pixel point in each frame buffer image of the projector to be corrected; S5: Calculate the perspective projection frustum parameters of the projector to be corrected according to the longitude and latitude data, convert the frame buffer image into a pre-deformed image according to the perspective projection frustum parameters, and complete the geometric correction.

2. The multi-projection correction method based on a binocular vision camera according to claim 1, characterized in that: The S1 further comprises: Calibrate two sets of binocular vision cameras; The camera coordinate system of one set of binocular vision cameras is selected as the unified coordinate system of the point cloud image, and the feature image is synchronously collected by another set of binocular vision cameras.

3. The multi-projection correction method based on a binocular vision camera according to claim 2, characterized in that: The S2 comprises the following steps: The structured light measurement method is used to calculate the characteristic images corresponding to the two groups of binocular vision cameras respectively to obtain horizontal phase data and vertical phase data; Generate corresponding points of binocular stereo matching according to points with the same phase value in the horizontal phase data and the vertical phase data; Calculating the parallax value of the projection screen according to the corresponding points, and solving the spatial coordinates of the projection screen according to the parallax value; The calculation formula of phase is: ， ， Where φ(x, y) is the phase information, I i (x, y) is the phase data of the i-th phase shift, A(x, y) is the background light intensity, and B(x, y) is the modulation factor; The calculation formula of spatial coordinates is: ， ， ， d=u L -u R , Where X, Y, and Z are the three-dimensional coordinate data of the spatial coordinates, B is the baseline distance between the lens centers of the left and right binocular vision cameras, f is the principal distance of the camera lens, (u L , v L )(u R , v R ) are the phase data of the current corresponding point combination, is the coordinate of the principal point after camera calibration, and d is the parallax value.

4. The multi-projection correction method based on a binocular vision camera according to claim 3 is characterized in that: The S3 comprises the following steps: S31: randomly selecting a point in the projection overlap area of ​​the projection screen, and calculating its coordinates in the camera coordinate systems of the two sets of binocular vision cameras according to the horizontal and vertical phase data corresponding to the selected point; S32: establishing a conversion relationship according to the coordinates of the selected points in the camera coordinate systems of the two sets of binocular vision cameras, and calculating coordinate conversion parameters; S33: According to the coordinate conversion parameters, the three-dimensional reconstructed data is converted to a set point cloud image unified coordinate system, and an aligned point cloud image is output.

5. The multi-projection correction method based on a binocular vision camera according to claim 4 is characterized in that: The expression of the conversion relation in S32 is: ， Among them, the coordinate transformation parameters include the rotation matrix R and the translation variable T, is the coordinate of point i in the camera coordinate system of the binocular vision camera that is not selected as the unified coordinate system of the point cloud image, is the coordinate of point i in the unified coordinate system of the point cloud image.

6. The multi-projection correction method based on a binocular vision camera according to claim 5, characterized in that: The S4 comprises the following steps: S41: Obtaining the coordinates of each pixel point in each frame buffer image of the projector to be calibrated in the unified coordinate system of the point cloud image; S42: Convert the unified coordinate system of the point cloud image to the spherical coordinate system according to the coordinate conversion formula, and output the longitude and latitude data corresponding to each pixel point.

7. The multi-projection correction method based on a binocular vision camera according to claim 6, characterized in that: The coordinate conversion formula in S42 is expressed as: ， Among them, Lo is the longitude, La is the latitude, (x, y, z) is the coordinate of the pixel point, and R is the radius of the sphere in the spherical coordinate system.

8. The multi-projection correction method based on a binocular vision camera according to claim 1, characterized in that: The S5 comprises the following steps: S51: Calculating the perspective projection cone parameters of the projector to be calibrated according to the longitude and latitude data; the perspective projection cone parameters include up, down, left, right, and right viewing angles and an hpr deflection angle; S52: converting the longitude and latitude data corresponding to each pixel of the frame buffer image into texture coordinates according to the perspective projection frustum parameters to generate a pre-deformed image; S53: The projector to be corrected projects the pre-deformed image to complete the geometric correction.

9. The multi-projection correction method based on a binocular vision camera according to claim 8, characterized in that: The calculation formula of the perspective projection frustum parameters is: ， ， ， in, is the longitude range of the projector to be calibrated, is the latitude range of the projector to be calibrated, and Respectively represent the left and right viewing angles of the projector to be calibrated, and Respectively represent the upper and lower viewing angles of the projector to be calibrated, is the heading rotation offset angle.

10. A multi-projection correction device based on a binocular vision camera, characterized in that: It includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of claims 1 to 9.

Images