Collaborative positioning point calibration method and device for multi-aperture imaging system

Abstract

The present invention provides a method and device for calibrating collaborative positioning points of a multi-aperture imaging system, including: designing a far-field equivalent optical path to provide a far-field straight line target for the multi-aperture optical imaging system to be calibrated, and each aperture of the multi-aperture optical imaging system images the far-field straight line target; calibrating the rotation angle error between the image planes of each aperture through the slope of the straight line image obtained by each aperture; establishing a positioning point correction model according to the rotation angle error between the image planes of each aperture, and realizing the correction of the positioning points in each aperture image. The present invention can realize the correction of positioning points during collaborative tracking of a multi-aperture optical imaging system, and meets the high-precision collaborative tracking and positioning requirements of the multi-aperture optical imaging system for any point of the target.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for collaborative positioning point calibration of a multi-aperture imaging system. Background Art

[0002] Target positioning is a prerequisite for the optical imaging detection system to achieve stable closed-loop tracking of the target. The optical imaging detection system generally extracts a point on the target image as a reference positioning point, and then uses the tracking control subsystem to close the loop on the positioning point, thereby achieving high-precision and stable tracking of the target. This positioning point is generally the center of mass or centroid of the target image. Since in most cases, the targets that the optical imaging detection system needs to detect are mostly extended targets of a certain size, there is often a need to change the positioning point for offset tracking in application scenarios with high-precision tracking requirements. The general method is to give a bias vector based on the initial positioning point such as the center of mass or centroid, and re-determine the new positioning point based on this vector, and then switch the closed-loop tracking position of the target.

[0003] However, for multi-aperture imaging detection systems, the above method may have certain problems. A multi-aperture imaging system refers to an imaging system with multiple independent imaging and closed-loop control channels. Since the multi-aperture optoelectronic imaging detection system not only has the ability of each aperture to work together, but also can back up each other, it is increasingly used in the field of imaging detection. For a multi-aperture detection system, all apertures often need to be based on the same offset vector given by the initial positioning point when performing closed-loop tracking of any point on the target. Ignoring the influence of each aperture parallax on the target imaging configuration, the best initial positioning point for each aperture is the centroid or centroid of the target image. Obviously, the consistency of the offset vectors of each aperture is the key factor affecting the consistency of the collaborative positioning of each aperture. In order to simplify the control logic, the multi-aperture detection system generally uses a unified offset vector to switch the positioning point of any point on the target. Since there may be rotation errors between different aperture imaging planes due to factors such as the camera installation angle, the calibration of the rotation errors between different aperture imaging planes is a necessary prerequisite for the high-precision collaborative closed loop of each aperture at the same target point. However, there is currently no suitable method disclosed for calibrating the rotation angle error between imaging surfaces with different apertures. Summary of the invention

[0004] In view of the fact that there is no public report on the solution for calibrating the inter-aperture rotation angle error of a multi-aperture optical imaging detection system in the prior art, the present invention provides a method and device for collaborative positioning point calibration of a multi-aperture imaging system.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] In one aspect, the present invention provides a method for collaborative positioning point calibration of a multi-aperture imaging system, comprising the following steps:

[0007] Design a far-field equivalent optical path to provide a far-field linear target for the multi-aperture optical imaging system to be calibrated. Each aperture of the multi-aperture optical imaging system images the far-field linear target.

[0008] The slope of the straight line image obtained by each aperture is used to calibrate the rotation angle error between the image planes of each aperture;

[0009] According to the rotation angle error between the image planes of each aperture, a positioning point correction model is established to realize the correction of the positioning points in the images of each aperture.

[0010] Furthermore, the far-field equivalent optical path includes a DMD spatial light modulator, a laser, a collimating lens, a beam expanding lens group, and a multi-aperture optical imaging system to be calibrated; a straight line image is loaded on the DMD spatial light modulator; the laser is used to emit a beam of laser light to the DMD spatial light modulator; a collimating lens is provided on the reflection path of the DMD spatial light modulator, and the modulated image plane of the DMD spatial light modulator is located at the focal plane position of the collimating lens. The laser is modulated and reflected by the DMD spatial light modulator, and then collimated by the collimating lens, and then expanded by the beam expanding lens group before entering the multi-aperture imaging system. The beam expanding lens group adopts a Galilean telescope structure, thereby shortening the length of the beam expanding optical path. At this time, each aperture of the multi-aperture optical imaging system images a far-field straight line target.

[0011] On the other hand, a multi-aperture imaging system collaborative positioning point calibration device is provided, comprising:

[0012] The far-field equivalent optical path is used to provide a far-field linear target for the multi-aperture optical imaging system to be calibrated, and each aperture of the multi-aperture optical imaging system images the far-field linear target;

[0013] A rotation angle error calibration module is used to calibrate the rotation angle error between the image planes of each aperture by using the slope of the straight line image obtained by each aperture;

[0014] The positioning point correction module is used to establish a positioning point correction model according to the rotation angle error between the image planes of each aperture, so as to realize the positioning point correction in the image of each aperture.

[0015] The beneficial technical effects that the present invention can produce are:

[0016] The present invention designs a far-field equivalent optical path and provides a far-field straight line target for the multi-aperture imaging detection system to be calibrated. By calculating the slope of the straight line image in each aperture, the image plane rotation error between the apertures is calibrated; according to the image plane rotation error of each aperture, a positioning point correction model is established to correct the positioning point in each aperture. The present invention realizes the positioning point correction during the collaborative tracking of the multi-aperture optical imaging system, and meets the high-precision collaborative tracking and positioning requirements of the multi-aperture optical imaging system for any point of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0018] Figure 1 A calibration principle diagram of a far-field equivalent optical path is provided for an embodiment;

[0019] Figure 2 FIG. 1 is a diagram showing the calibration result of the rotation angle error of a three-aperture optical imaging detection system in an embodiment, wherein Figure 2 (a) Figure 2 (b) Figure 2 (c) are the imaging effects of the linear calibration object in the closed-loop state of the 1st, 2nd and 3rd apertures in the three-aperture optical imaging detection system. Figure 2 (d) is a comparison result of performing LSD line detection on three aperture line images and placing them in the same image;

[0020] Figure 3 FIG. 1 is a schematic diagram of a positioning point correction principle of a multi-aperture optical imaging system in an embodiment, wherein Figure 3 (a) Schematic diagram of extracting bias vector in basic aperture; Figure 3 (b) Schematic diagram of the new positioning points after correction in the aperture.

[0021] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] In one embodiment, a method for collaborative positioning point calibration of a multi-aperture imaging system is provided, comprising:

[0024] Design a far-field equivalent optical path to provide a far-field linear target for the multi-aperture optical imaging system to be calibrated. Each aperture of the multi-aperture optical imaging system images the far-field linear target.

[0025] The slope of the straight line image obtained by each aperture is used to calibrate the rotation angle error between the image planes of each aperture;

[0026] According to the rotation angle error between the image planes of each aperture, a positioning point correction model is established to realize the correction of the positioning points in the images of each aperture.

[0027] For a multi-aperture optical imaging system with a larger aperture, the calibration object often needs to be placed more than ten kilometers or even dozens of kilometers away, which makes it difficult to calibrate the image plane rotation angle. The present invention calibrates the image plane rotation angle error by designing a far-field equivalent optical path and providing a straight calibration object in the equivalent optical path. Figure 1 , a calibration principle diagram of a far-field equivalent optical path is provided for an embodiment, wherein the far-field equivalent optical path includes a DMD spatial light modulator 1, a laser 3, a collimating lens 4, a beam expanding lens group 5, and a multi-aperture optical imaging system 6 to be calibrated; the DMD spatial light modulator 1 is loaded with a straight line image 2; the laser 3 is used to emit a laser beam to the DMD spatial light modulator 1; a collimating lens 4 is provided on the reflection path of the DMD spatial light modulator 1, and the modulated image plane of the DMD spatial light modulator 1 is located at the focal plane position of the collimating lens 4. The laser is modulated and reflected by the DMD spatial light modulator 1, and then collimated by the collimating lens 4, and then expanded by the beam expanding lens group 5 before entering the multi-aperture imaging system. The beam expanding lens group 5 adopts a Galilean telescope structure, thereby shortening the length of the beam expanding optical path. At this time, each aperture of the multi-aperture optical imaging system 6 images a far-field straight line target; wherein the distance between the modulated image plane of the DMD spatial light modulator 1 and the main surface of the collimating lens 4 is the focal length of the collimating lens 4 f , this optical path is equivalent to the target being at infinity in the imaging system.

[0028] The present invention calibrates the rotation angle error between each aperture image plane by comparing the slopes of the straight line images obtained by the multi-aperture optical imaging system 6 with different apertures. However, affected by factors such as the MTF of the actual optical system, the straight line images obtained by each aperture camera have a certain width compared to the ideal straight line. In a preferred embodiment of the present invention, the method for calibrating the rotation angle error of each aperture image plane includes:

[0029] The straight line image edges in the image obtained by each aperture are extracted by using the LSD straight line detection algorithm in the image processing technology;

[0030] The rotation angle error between the image planes of each aperture is calibrated by the upper slope of the straight image edge corresponding to each aperture.

[0031] It should be noted that in the above process of calibrating the rotation angle error of each aperture image plane, each aperture closes the centroid position of the linear image. At this time, the centroid of the linear image in each aperture will be located at the center of the image field of view, thereby eliminating the influence of the parallax between apertures on the image position.

[0032] In a preferred embodiment, the method for correcting the positioning points in each aperture image includes:

[0033] Each aperture performs a closed loop on the centroid of the target image. In the closed loop state, the centroid of the target image in all apertures coincides with the center point of the imaging field of view. The target image is the straight line image observed by each aperture.

[0034] The initial positioning point of each aperture is the centroid of the target image. Taking a certain aperture in the multi-aperture imaging system as the reference aperture, the coordinates of the new offset positioning point are determined in the reference aperture, and the offset vector is extracted.

[0035] For the remaining apertures except the reference aperture, a positioning point correction model is established based on the initial positioning point of the aperture, the rotation angle error of the aperture relative to the reference aperture, and the offset vector;

[0036] Based on the positioning point correction model, a new positioning point closed loop is switched for the target image in the aperture to achieve high-precision collaborative tracking.

[0037] Specifically, the positioning point correction model is established as follows:

[0038] ;

[0039] In the above formula, the reference aperture is assumed to be the first aperture of the multi-aperture optical imaging system. Represents the first The coordinates of the new positioning points after correction in the aperture are: It is The rotation angle error of each aperture relative to the reference aperture, It is The coordinates of the initial positioning point in the aperture; represents the length of the bias vector, is the vector angle of the offset vector.

[0040] Figure 2 FIG. 1 is a diagram showing the calibration result of the rotation angle error of a three-aperture optical imaging detection system in an embodiment, wherein Figure 2 (a) Figure 2 (b) Figure 2 (c) are the imaging effects of the linear calibration object in the closed-loop state of the 1st, 2nd and 3rd apertures in the three-aperture optical imaging detection system. Figure 2(d) is a comparison result diagram of three aperture straight line images subjected to LSD straight line detection and placed in the same image, wherein the red, green and blue lines correspond to the edges of the straight line images obtained by the 1st, 2nd and 3rd apertures in the three-aperture optical imaging detection system, respectively. Obviously, although the three apertures image the same straight line calibration target, there are certain differences in the angles of the straight lines. The present invention uses the slope of the edge on the straight line image to calibrate the rotation angle error of each aperture image. During the calibration process, an image coordinate system is established with the center of the image as the origin, the horizontal right direction as the X direction, and the vertical downward direction as the Y direction. At this time, the center point of the image is the center of mass of the straight line image. Figure 2 In the three-aperture optical imaging detection system, the angles of the straight lines in the first, second, and third apertures relative to the X direction are 26.3631°, 25.9570°, and 27.2553°, respectively. If the first aperture is taken as the reference aperture, the rotation angle error of the third aperture relative to the reference aperture is 0.8922°, and the rotation angle error of the second aperture relative to the reference aperture is -0.4061°.

[0041] Figure 3 FIG. 1 is a schematic diagram of a positioning point correction principle of a multi-aperture optical imaging system in an embodiment, wherein Figure 3 (a) Schematic diagram of extracting bias vector in basic aperture; Figure 3 (b) Schematic diagram of the new positioning point after correction in the aperture. Figure 3 In the image coordinate system, the image center is taken as the origin, the horizontal right direction is the X direction, and the vertical downward direction is the Y direction. Each aperture is in a closed loop state, and the center point of the image is the centroid of the linear image. Figure 3 In (a), the image coordinate system in the basic aperture is . Figure 3 (b) The image coordinate system in the aperture is . It is The rotation angle error of the aperture relative to the reference aperture, the offset vector step length from the initial point to the new positioning point is , the vector angle is .according to Figure 3 , coordinate system and There is a rotation angle between Therefore, the offset vector extracted from the reference aperture is used Determining new co-location points in other apertures will result in positioning errors. Figure 3 , the above positioning point can be corrected using the following formula:

[0042] ;

[0043] In the above formula, Indicates The coordinates of the new positioning points after correction in the aperture are: It is The rotation angle error of each aperture relative to the reference aperture, It is the initial positioning point of all aperture images in the multi-aperture optical imaging system. Coincident with the centroid of the target image and the center of the field of view.

[0044] In one embodiment, a multi-aperture imaging system collaborative positioning point calibration device is provided, comprising the following steps:

[0045] The far-field equivalent optical path is used to provide a far-field linear target for the multi-aperture optical imaging system to be calibrated. Each aperture of the multi-aperture optical imaging system images the far-field linear target. The far-field equivalent optical path is as follows: Figure 1 shown.

[0046] A rotation angle error calibration module is used to calibrate the rotation angle error between the image planes of each aperture by using the slope of the straight line image obtained by each aperture;

[0047] The positioning point correction module is used to establish a positioning point correction model according to the rotation angle error between the image planes of each aperture, so as to realize the positioning point correction in the image of each aperture.

[0048] Among them, the rotation angle error calibration module includes:

[0049] A straight line detection module is used to extract straight line image edges in the images obtained at each aperture;

[0050] The rotation angle error calibration module is used to calibrate the rotation angle error between the image planes of each aperture using the upper edge slope of the straight line image edge corresponding to each aperture.

[0051] The implementation methods of the above modules and the construction of the model can adopt the methods described in any of the above embodiments, which will not be repeated here.

[0052] Matters not covered by the present invention are known technologies.

[0053] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0054] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for collaborative positioning point calibration of a multi-aperture imaging system, characterized in that: include: Design a far-field equivalent optical path to provide a far-field straight line target for the multi-aperture optical imaging system to be calibrated, and each aperture of the multi-aperture optical imaging system images the far-field straight line target. The far-field equivalent optical path includes a DMD spatial light modulator, a laser, a collimating lens, a beam expanding lens group, and the multi-aperture optical imaging system to be calibrated; the DMD spatial light modulator is loaded with a straight line image; the laser is used to emit a beam of laser to the DMD spatial light modulator; a collimating lens is arranged on the reflection path of the DMD spatial light modulator, and the modulated image plane of the DMD spatial light modulator is located at the focal plane position of the collimating lens; the laser is modulated and reflected by the DMD spatial light modulator, and then collimated by the collimating lens, and then expanded by the beam expanding lens group and enters the multi-aperture imaging system; each aperture of the multi-aperture optical imaging system images the far-field straight line target; The slope of the straight line image obtained by each aperture is used to calibrate the rotation angle error between the image planes of each aperture; According to the rotation angle error between the image planes of each aperture, a positioning point correction model is established to realize the correction of the positioning points in the images of each aperture.

2. The method for collaborative positioning point calibration of a multi-aperture imaging system according to claim 1, characterized in that: Calibrate the rotation angle error of each aperture image plane, including: The straight line image edges in each aperture image are extracted by LSD line detection algorithm; The rotation angle error between the image planes of each aperture is calibrated by the upper slope of the straight image edge corresponding to each aperture.

3. The method for collaborative positioning point calibration of a multi-aperture imaging system according to claim 1 or 2, characterized in that: Correction of positioning points in each aperture image, including: Each aperture performs a closed loop on the centroid of the target image. In the closed loop state, the centroid of the target image in all apertures coincides with the center point of the imaging field of view. The target image is the straight line image observed by each aperture. The initial positioning point of each aperture is the centroid of the target image. Taking a certain aperture in the multi-aperture imaging system as the reference aperture, the coordinates of the new offset positioning point are determined in the reference aperture, and the offset vector is extracted. For the remaining apertures except the reference aperture, a positioning point correction model is established based on the initial positioning point of the aperture, the rotation angle error of the aperture relative to the reference aperture, and the offset vector; Based on the positioning point correction model, a new positioning point closed loop is switched for the target image in the aperture to achieve high-precision collaborative tracking.

4. The method for collaborative positioning point calibration of a multi-aperture imaging system according to claim 3, characterized in that: Establish the positioning point correction model as follows: ; In the above formula, the reference aperture is assumed to be the first aperture of the multi-aperture optical imaging system. Represents the first The coordinates of the new positioning points after correction in the aperture are: It is The rotation angle error of each aperture relative to the reference aperture, It is The coordinates of the initial positioning point in the aperture; represents the length of the bias vector, is the vector angle of the offset vector.

5. A multi-aperture imaging system collaborative positioning point calibration device, characterized in that: include: Far-field equivalent optical path, including DMD spatial light modulator, laser, collimating lens, beam expansion lens group and multi-aperture optical imaging system to be calibrated; A straight line image is loaded on the DMD spatial light modulator; The laser is used to emit a laser beam to the DMD spatial light modulator; a collimating lens is arranged on the reflection path of the DMD spatial light modulator, and the modulation image plane of the DMD spatial light modulator is located at the focal plane position of the collimating lens; the laser is modulated and reflected by the DMD spatial light modulator, and then collimated by the collimating lens, and then expanded by the beam expansion lens group and enters the multi-aperture imaging system; each aperture of the multi-aperture optical imaging system images a far-field linear target; A rotation angle error calibration module is used to calibrate the rotation angle error between the image planes of each aperture by using the slope of the straight line image obtained by each aperture; The positioning point correction module is used to establish a positioning point correction model according to the rotation angle error between the image planes of each aperture, so as to realize the positioning point correction in the image of each aperture.

6. The multi-aperture imaging system collaborative positioning point calibration device according to claim 5, characterized in that: Rotation angle error calibration module, including: A straight line detection module is used to extract straight line image edges in the images obtained at each aperture; The rotation angle error calibration module is used to calibrate the rotation angle error between the image planes of each aperture using the upper edge slope of the straight line image edge corresponding to each aperture.

7. The multi-aperture imaging system collaborative positioning point calibration device according to claim 6, characterized in that: The line detection module extracts the straight line image edges in the image obtained by each aperture through the LSD line detection algorithm.

8. The multi-aperture imaging system collaborative positioning point calibration device according to claim 5, 6 or 7, characterized in that: The positioning point correction module realizes the correction of positioning points in each aperture image, including: Each aperture performs a closed loop on the centroid of the target image. In the closed loop state, the centroid of the target image in all apertures coincides with the center point of the imaging field of view. The target image is the straight line image observed by each aperture. The initial positioning point of each aperture is the centroid of the target image. Taking a certain aperture in the multi-aperture imaging system as the reference aperture, the coordinates of the new offset positioning point are determined in the reference aperture, and the offset vector is extracted. For the remaining apertures except the reference aperture, a positioning point correction model is established based on the initial positioning point of the aperture, the rotation angle error of the aperture relative to the reference aperture, and the offset vector; Based on the positioning point correction model, a new positioning point closed loop is switched for the target image in the aperture to achieve high-precision collaborative tracking.

9. The multi-aperture imaging system collaborative positioning point calibration device according to claim 8, characterized in that: The positioning point correction model established in the positioning point correction module is as follows: ; In the above formula, the reference aperture is assumed to be the first aperture of the multi-aperture optical imaging system. Represents the first The coordinates of the new positioning points after correction in the aperture are: It is The rotation angle error of each aperture relative to the reference aperture, It is The coordinates of the initial positioning point in the aperture; represents the length of the bias vector, is the vector angle of the offset vector.

Images