A binocular camera calibration system and method based on optical module

Abstract

The present invention discloses a binocular camera calibration system and method based on an optical module, the system comprising: an optical waveguide for generating two parallel images, and the two images are focused at infinity; a binocular camera, arranged in front of the optical waveguide, for acquiring the image focused at infinity of the optical waveguide; the binocular camera comprises a left camera and a right camera; an adjustment mechanism for adjusting the spatial position and angle of the binocular camera; a binocular fusion module, connected to the binocular camera, and acquiring images from the binocular camera to determine whether the images in the binocular camera completely overlap. The present invention is based on the optical waveguide emitting two absolutely parallel images, using a binocular camera to observe the image focused at infinity in the optical waveguide, and adjusting the binocular camera through an adjustment mechanism to make the optical axes of the binocular camera parallel, at which time the two images of the image have no pixel difference, and the image is the clearest, so that the binocular camera calibration accuracy can be improved and the binocular camera calibration difficulty can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of AR testing technology, and in particular to a binocular camera calibration system and method based on an optical module. Background Art

[0002] AR / VR devices with binocular display can display 3D effects, giving people a dreamy stereoscopic visual experience. One way to achieve stereoscopic display is to first make the left and right eye images overlap at infinity (equivalent to the center lines of the images being parallel), and then use hardware or software to offset the two images by a certain pixel / angle to form a visual difference between the left and right eyes, ultimately giving people a stereoscopic sense.

[0003] In the above-mentioned stereoscopic display principle, the technology to achieve the overlap of the left and right eye images at infinity is the key, but it is relatively difficult. The light propagation path is invisible to the naked eye, and a binocular camera with absolutely parallel optical axes must be used to assist in correction. There is currently no such equipment on the market. Make the left and right images of AR / VR overlap at infinity, and use this as a basis for pixel offset to form visual difference, and finally achieve stereoscopic display. Therefore, the correction method and device for the parallel binocular optical axes are the key to realizing stereoscopic display technology. In order to calibrate the AR / VR projected virtual images to overlap at infinity, the optical axes of the binocular cameras used for calibration must be absolutely parallel to ensure the accuracy of calibration.

[0004] At present, the method of binocular parallelism calibration is to use a reflector or mechanical translation for binocular imaging calibration, but both methods have the problems of low calibration accuracy and inconvenient operation. Summary of the invention

[0005] The embodiments of the present invention provide a binocular camera calibration system and method based on an optical module, aiming to improve the binocular camera calibration accuracy and reduce the binocular camera calibration difficulty.

[0006] An embodiment of the present invention provides a binocular camera calibration system based on an optical module, comprising:

[0007] An optical waveguide for producing two parallel images that are focused at infinity;

[0008] A binocular camera, arranged in front of the optical waveguide, for acquiring an image focused at infinity of the optical waveguide; the binocular camera comprises a left camera and a right camera;

[0009] An adjustment mechanism, used to adjust the spatial position and angle of the binocular camera;

[0010] The binocular fusion module is connected to the binocular camera and obtains images from the binocular camera to determine whether the images in the binocular camera completely overlap.

[0011] Furthermore, the optical waveguide includes an optical machine for emitting an image, an entrance pupil grating for diffracting the image into the optical waveguide, and an exit pupil grating for diffracting the image out of the optical waveguide and focusing it into a virtual image at infinity.

[0012] Furthermore, the spatial position of the binocular camera and the captured image signal form a closed-loop control circuit.

[0013] Furthermore, the adjustment mechanism is a six-axis adjustment platform, and the six-axis adjustment platform is arranged below the binocular camera.

[0014] Furthermore, the six-axis adjustment platform includes adjustable structures of X-axis, Y-axis, Z-axis, Rx-axis, Ry-axis, and Rz-axis.

[0015] Furthermore, the initial position deviations of the binocular cameras in the X-axis direction and the Z-axis direction are both within the range of 30 mm, wherein the spacing between the left camera and the right camera is 65±25 mm, and the angle between the left camera and / or the right camera and the optical waveguide is 0°±30°.

[0016] Furthermore, the binocular fusion module is used to compare the pixels of the images in the left camera and the right camera, and calculate the distance between the pixels to determine whether the optical axes of the binocular cameras are parallel in space;

[0017] When the pixel deviations in the X and Y directions are both 0, it is determined that the binocular cameras are absolutely parallel in the optical axis space.

[0018] Furthermore, it also includes a smooth bottom plate, and the optical waveguide, binocular camera and adjustment mechanism are all arranged on the smooth bottom plate.

[0019] The embodiment of the present invention also provides a binocular camera calibration method based on an optical module, comprising:

[0020] Connecting a binocular camera to a binocular fusion module, and using the binocular fusion module to acquire an image of the binocular camera;

[0021] Generate two parallel images at infinity through an optical waveguide;

[0022] Adjusting the focal length of the binocular camera so that the binocular camera captures the image generated by the optical waveguide;

[0023] The positions of the binocular cameras are adjusted by an adjusting mechanism so that the images of the binocular fusion module overlap.

[0024] Furthermore, the adjustment mechanism is a six-axis adjustment platform;

[0025] The adjusting mechanism is used to adjust the position of the binocular camera so that the images of the binocular fusion module overlap, including:

[0026] The position and angle of the binocular camera in the horizontal direction and the vertical direction are adjusted respectively by the six-axis adjustment platform;

[0027] The binocular fusion module compares the pixels of the image and calculates the distance between pixels to determine whether the optical axes of the binocular cameras are parallel in space;

[0028] When the deviations of the pixels in the X-axis direction and the Y-axis direction are both 0, it is determined that the binocular cameras 200 are absolutely parallel in the optical axis space, and it is determined that the images in the binocular fusion module completely overlap in the horizontal direction and the vertical direction.

[0029] The embodiment of the present invention provides a binocular camera calibration system and method based on an optical module, the system comprising: an optical waveguide for generating two parallel images, and the two images are focused at infinity; a binocular camera, arranged in front of the optical waveguide, for acquiring the image focused at infinity of the optical waveguide; the binocular camera comprises a left camera and a right camera; an adjustment mechanism for adjusting the spatial position and angle of the binocular camera; a binocular fusion module connected to the binocular camera and acquiring images from the binocular camera to determine whether the images in the binocular camera completely overlap. The embodiment of the present invention is based on the optical waveguide emitting two absolutely parallel images, using the binocular camera to observe the image focused at infinity in the optical waveguide, and adjusting the spatial position and angle of the binocular camera through the adjustment mechanism so that the optical axes of the binocular camera are parallel. At this time, there is no pixel difference between the two images of the fused image, and the image is the clearest, so that the calibration accuracy of the binocular camera can be improved and the calibration difficulty of the binocular camera can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0031] Figure 1 A schematic diagram of the structure of a binocular camera calibration system based on an optical module provided in an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of a binocular camera calibration system based on an optical module before calibration provided in an embodiment of the present invention;

[0033] Figure 3 A schematic diagram of a binocular camera calibration system based on an optical module after calibration provided by an embodiment of the present invention;

[0034] Figure 4A schematic diagram of the structure of an optical waveguide in a binocular camera calibration system based on an optical module provided by an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of an image generated by an optical waveguide in a binocular camera calibration system based on an optical module provided by an embodiment of the present invention;

[0036] Figure 6 A schematic diagram of the principle of a binocular camera calibration system based on an optical module provided in an embodiment of the present invention;

[0037] Figure 7 A schematic flow chart of a binocular camera calibration method based on an optical module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0040] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0041] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0042] See below Figure 1 , Figure 1 A schematic diagram of a binocular camera calibration system based on an optical module provided by an embodiment of the present invention, including:

[0043] The optical waveguide 100 is used to generate two parallel images, and the two images are focused at infinity;

[0044] A binocular camera 200 is disposed in front of the optical waveguide 100 and is used to obtain an image focused at infinity of the optical waveguide 100; the binocular camera 200 includes a left camera 201 and a right camera 202;

[0045] An adjustment mechanism 300, used to adjust the spatial position and angle of the binocular camera 200;

[0046] The binocular fusion module 400 is connected to the binocular camera 200 and acquires images from the binocular camera 200 to determine whether the images in the binocular camera 200 completely overlap.

[0047] In this embodiment, the binocular camera calibration system includes an optical waveguide 100, a binocular camera 200, an adjustment mechanism 300 and a binocular fusion module 400. The implementation process thereof can be specifically: connecting the binocular camera 200 with the binocular fusion module 400, and using the binocular fusion module 400 to obtain an image of the binocular camera 200; generating two parallel images at infinity through the optical waveguide 100; adjusting the focal length of the binocular camera 200 so that the binocular camera 200 captures the image generated by the optical waveguide 100; and adjusting the position of the binocular camera 200 through the adjustment mechanism 300 so that the images of the binocular fusion module 400 overlap.

[0048] This embodiment is based on the optical waveguide 100 emitting two absolutely parallel images, and using the binocular camera 200 to observe the image focused at infinity in the optical waveguide 100. In the initial position, since the optical axes of the cameras are not parallel, the image in the binocular fusion module 400 is likely to have serious ghosting. The spatial position of the camera and the image signal captured by the camera form a closed-loop control loop. The spatial position and angle of the binocular camera 200 are adjusted by the adjustment mechanism 300 to make the optical axes of the binocular camera 200 parallel. At this time, there is no pixel difference between the two images of the fused image, and the image is the clearest, which can improve the calibration accuracy of the binocular camera 200 and reduce the calibration difficulty of the binocular camera 200.

[0049] This embodiment is applied in the optical axis calibration process of the binocular camera 200. Since the camera optical axis has no mechanical positioning point, the mechanical method cannot achieve the high-precision optical axis parallelism specification requirement of 0°. By adopting the method of the present invention, the optical waveguide 100 that can emit absolutely parallel light is used as the feedback signal for camera calibration, so that the high-precision binocular camera 200 optical axis parallelism calibration can be achieved, and the operation is simple and the cost is low.

[0050] In one embodiment, in combination Figure 2 and Figure 3, because the optical axes of the binocular cameras 200 are not parallel, the image in the binocular camera 200 has a deviation angle θ relative to the center of the binocular camera 200. The function of the binocular fusion module 400 is to overlap the pixels of the images in the left and right cameras one by one to form a left and right fused picture. However, since the images in the left camera 201 and the right camera 202 have an angle deviation of θ, the fused image 43 has serious ghosting. When the binocular position is adjusted by the adjustment mechanism 300, for example, the binocular camera 200 is driven to rotate / translate by the adjustment mechanism 300. When the center of the binocular camera 200 coincides with the image, the fused image is clearest. At this time, the angle of the optical axis of the binocular camera 200 is exactly 0°.

[0051] In one embodiment, the optical waveguide 100 includes an optical engine 101 for emitting an image, an entrance pupil grating 102 for diffracting the image into the optical waveguide 100 , and an exit pupil grating 103 for diffracting the image out of the optical waveguide 100 and focusing the image into a virtual image at infinity.

[0052] In this embodiment, combined with Figure 4 and Figure 5 , after the optical machine 101 on the optical waveguide 100 generates an image, the image is transmitted into the optical waveguide 100 through the entrance pupil grating 102, and then the image is emitted through the exit pupil grating 103. The entire structure is made on a piece of transparent glass, through which the left eye image and the right eye image at infinity can be observed. And the image is focused at infinity. The optical waveguide 100 is processed by semiconductor process, and the optical axis parallelism is high, which is the best calibration light source for binocular calibration system. In a specific embodiment, the optical machine 101 is located at the upper center position of the optical waveguide 100, the entrance pupil grating 102 is located below the optical machine 101, and the exit pupil grating 103 is located on both sides of the entrance pupil grating 102. The optical waveguide 100 also includes pupil expansion gratings 104 located on both sides of the optical machine 101 and above the exit pupil grating 103.

[0053] This embodiment uses internal light splitting of the optical waveguide 100, and the optical axes of the two images are absolutely parallel in space. The binocular camera 200 respectively captures the images in the exit pupil grating 103 of the optical waveguide 100, so that the optical axis of the binocular camera 200 is parallel to the optical axis of the optical waveguide 100, and the optical axis of the binocular camera 200 is also adjusted to be parallel through the adjustment mechanism 300, so that the image centers of the binocular camera 200 are aligned, that is, the binocular calibration is completed. The parallelism satisfies the following formula:

[0054]

[0055] Among them, θ is the parallelism of the optical axis, Δx and Δy represent the pixel difference of the binocular camera in the x direction and y direction, and ∞ represents the imaging distance.

[0056] The optical waveguide 100 is a light emitted by an optical machine 101, which enters the entrance pupil grating 102 at 90 degrees to the waveguide. After passing through the waveguide, the light is emitted at 90 degrees to the optical waveguide 100, and the centers of the two images of the exit pupil are absolutely parallel. Using this component, the binocular camera 200 is placed at the image receiving end, and the center of the binocular camera 200 is adjusted to align with the center of the image presented in the optical waveguide 100, so that the center of the binocular camera 200 is aligned. It should be noted that the binocular camera calibration system based on the optical module described in this embodiment does not need to consider whether the initial position of the binocular camera 200 and the optical waveguide 100 is perpendicular. Figure 6 As shown, the initial position and angle of the binocular camera 200 and the optical waveguide 100 do not affect the parallelism of the camera optical axis, but only affect the left and right relative position of the binocular camera 200. Since the binocular camera 200 observes an infinite image, the left and right position deviation has no effect on the test result. Therefore, the binocular calibration device made by the above principle has the characteristics of simple operation and low cost.

[0057] In one embodiment, the adjustment mechanism 300 is a six-axis adjustment platform, and the six-axis adjustment platform is disposed below the binocular camera 200 .

[0058] In this embodiment, a six-axis adjustment platform is used as an adjustment mechanism 300, which is arranged below the binocular camera 200, and the six-axis adjustment platform includes adjustable structures of X-axis, Y-axis, Z-axis, Rx-axis, Ry-axis, and Rz-axis, which are used to realize three-axis translation and three-axis rotation of the binocular camera 200.

[0059] In one embodiment, the initial position deviation of the binocular camera 200 in the X-axis direction and the Z-axis direction is within the range of 30 mm, wherein the spacing between the left camera 201 and the right camera 202 is 65±25 mm, and the angle between the left camera 201 and / or the right camera 202 and the optical waveguide 100 is 0°±30°.

[0060] In this embodiment, the initial position of the binocular camera 200 can have a certain deviation, for example, the deviation in the X-axis direction can be within ±30mm, and the position in the Y-axis direction is not limited, as long as the binocular camera 200 receives a sufficiently strong picture, and the deviation in the Z-axis direction can be ±30mm. At the same time, the spacing between the left eye camera 201 and the right eye camera 202 in the binocular camera 200 is 65±25mm. In addition, there is no special restriction on the angle between the left eye camera 201 and / or the right eye camera 202 and the optical waveguide 100, for example, 0 degrees or ±30 degrees are both possible. During the calibration process, the optical waveguide 100 is stationary relative to the binocular camera 200, and the optical waveguide 100 can also be axially translated along the optical axis for alignment. This solution has the advantages of large tolerance, high calibration accuracy and low cost.

[0061] In one embodiment, the binocular fusion module 400 is used to compare the pixels of the images in the left camera 201 and the right camera 202, and calculate the distance between the pixels to determine whether the optical axes of the binocular cameras 200 are spatially parallel;

[0062] When the deviations of the pixels in the X direction and the Y direction are both 0, it is determined that the binocular cameras 200 are absolutely parallel in the optical axis space.

[0063] In this embodiment, the binocular fusion module 400 compares the pixels of all or part of the corresponding positions of the images in the left camera 201 and the right camera 202, and calculates the distance between the pixels, so as to calculate whether the optical axes of the binocular cameras 200 are spatially parallel. When the deviations of the pixels in the X / Y directions are both 0, it can be determined that the binocular cameras 200 are absolutely parallel in the optical axis space.

[0064] In one embodiment, a smooth bottom plate 500 is further included, and the optical waveguide 100 , the binocular camera 200 and the adjustment mechanism 300 are all disposed on the smooth bottom plate 500 .

[0065] In this embodiment, the optical waveguide 100, the binocular camera 200 and the adjustment mechanism 300 are all installed on the smooth bottom plate 500 to avoid the uneven placement and affect the final calibration result. In a specific application scenario, the adjustment mechanism 300 can be first installed on the smooth bottom plate 500 with screws, and then the binocular camera 200 is installed on the adjustment mechanism 300, and then the binocular camera 200 is connected to the binocular fusion module 400 by means of cables, etc., and the optical waveguide 100 is fixed to the bottom plate 500 by screws or glue.

[0066] Figure 7 A schematic flow chart of a binocular camera calibration method based on an optical module provided in an embodiment of the present invention specifically includes: steps S701 to S704.

[0067] S701, connecting the binocular camera 200 to the binocular fusion module 400, and using the binocular fusion module 400 to acquire an image of the binocular camera 200;

[0068] S702, generating two parallel images at infinity through the optical waveguide 100;

[0069] S703, adjusting the focal length of the binocular camera 200 so that the binocular camera 200 can capture the image generated by the optical waveguide 100;

[0070] S704, adjusting the position of the binocular camera 200 by the adjusting mechanism 300 so that the images of the binocular fusion module 400 overlap.

[0071] In one embodiment, the adjustment mechanism 300 is a six-axis adjustment platform;

[0072] The adjusting mechanism 300 is used to adjust the position of the binocular camera 200 so that the images of the binocular fusion module 400 overlap, including:

[0073] The position and angle of the binocular camera 200 in the horizontal direction and the vertical direction are adjusted respectively by the six-axis adjustment platform;

[0074] The binocular fusion module 400 compares the pixels of the image and calculates the distance between the pixels to determine whether the optical axes of the binocular cameras 200 are parallel in space;

[0075] When the deviations of the pixels in the X-axis direction and the Y-axis direction are both 0, it is determined that the binocular cameras 200 are absolutely parallel in the optical axis space, and it is determined that the images in the binocular fusion module 400 completely overlap in the horizontal direction and the vertical direction.

[0076] Since the embodiments of the method part correspond to the embodiments of the device part, please refer to the description of the embodiments of the device part for the embodiments of the method part, which will not be repeated here.

[0077] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0078] The embodiment of the present invention also provides a computer device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the computer device may also include various network interfaces, power supplies and other components.

[0079] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0080] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A binocular camera calibration system based on an optical module, characterized in that: include: An optical waveguide for producing two parallel images that are focused at infinity; A binocular camera, arranged in front of the optical waveguide, for acquiring an image focused at infinity of the optical waveguide; the binocular camera comprises a left camera and a right camera; An adjustment mechanism, used to adjust the spatial position and angle of the binocular camera; A binocular fusion module is connected to the binocular camera and acquires images from the binocular camera to determine whether the images in the binocular camera completely overlap; The binocular fusion module is used to compare the pixels of the images in the left camera and the right camera and calculate the distance between the pixels to determine whether the optical axes of the binocular cameras are parallel in space; When the pixel deviations in the X and Y directions are both 0, it is determined that the binocular cameras are absolutely parallel in the optical axis space.

2. The binocular camera calibration system based on optical module according to claim 1, characterized in that: The optical waveguide includes an optical machine for emitting an image, an entrance pupil grating for diffracting the image into the optical waveguide, and an exit pupil grating for diffracting the image out of the optical waveguide and focusing the image into a virtual image at infinity.

3. The binocular camera calibration system based on optical module according to claim 1, characterized in that: The spatial position of the binocular camera and the captured image signal form a closed-loop control circuit.

4. The binocular camera calibration system based on optical module according to claim 1, characterized in that: The adjustment mechanism is a six-axis adjustment platform, and the six-axis adjustment platform is arranged below the binocular camera.

5. The binocular camera calibration system based on optical module according to claim 4, characterized in that: The six-axis adjustment platform includes adjustable structures of X-axis, Y-axis, Z-axis, Rx-axis, Ry-axis and Rz-axis.

6. The binocular camera calibration system based on optical module according to claim 1, characterized in that: The initial position deviations of the binocular cameras in the X-axis direction and the Z-axis direction are both within a range of 30 mm, wherein the spacing between the left camera and the right camera is 65±25 mm, and the angle between the left camera and / or the right camera and the optical waveguide is 0°±30°.

7. The binocular camera calibration system based on optical module according to claim 1, characterized in that: It also includes a smooth bottom plate, on which the optical waveguide, binocular camera and adjustment mechanism are all arranged.

8. A binocular camera calibration method based on an optical module, characterized in that: include: Connecting a binocular camera to a binocular fusion module, and using the binocular fusion module to acquire an image of the binocular camera; Generate two parallel images at infinity through an optical waveguide; Adjusting the focal length of the binocular camera so that the binocular camera captures the image generated by the optical waveguide; The position of the binocular camera is adjusted by an adjustment mechanism so that the images of the binocular fusion module overlap; the adjustment mechanism is a six-axis adjustment platform; The adjusting mechanism is used to adjust the position of the binocular camera so that the images of the binocular fusion module overlap, including: The position and angle of the binocular camera in the horizontal direction and the vertical direction are adjusted respectively by the six-axis adjustment platform; The binocular fusion module compares the pixels of the image and calculates the distance between pixels to determine whether the optical axes of the binocular cameras are parallel in space; When the deviations of the pixels in the X-axis direction and the Y-axis direction are both 0, it is determined that the binocular cameras 200 are absolutely parallel in the optical axis space, and it is determined that the images in the binocular fusion module completely overlap in the horizontal direction and the vertical direction.

Images