Infrared spatial positioning system and method based on multi-view vision

Abstract

The application discloses a kind of infrared space positioning systems based on multi-vision, including multi-vision infrared camera and processing unit;Multi-vision infrared camera includes N camera, N≥3, at least the imaging point of one camera is not on the same straight line with the imaging point of each other camera.Positioning method includes: each camera is combined to form pair combination two by two;Each camera synchronously acquires image, and is sent to processing unit to process, obtains the morphological center coordinate position of infrared marker;Limit constraint principle is used to respectively match the morphological center coordinate of infrared marker photographed by each camera in pair combination;If the same matching relationship is satisfied in each pair combination, namely, it is determined that successful matching is matched.The application establishes multiple matching relationships by using multi-vision camera, and only when all matching relationships between any two cameras are satisfied, the matching of circle center coordinate is determined to be successful, and the problem of false matching can be completely avoided.

Description

Technical Field

[0001] This invention relates to the fields of biomedical applications and computers, specifically to an infrared spatial positioning system and method based on multi-view vision. Background Technology

[0002] Binocular infrared positioning devices are the most common three-dimensional spatial positioning systems, frequently used in surgical navigation and surgical robots, and rehabilitation medicine. Specific methods for achieving three-dimensional spatial positioning with binocular infrared positioning devices include:

[0003] 1) Perform camera calibration on the binocular camera to obtain the internal parameters of the left and right cameras and the external parameters between the two cameras, and establish the basic matrix;

[0004] 2) Add an infrared filter to the camera lens of the binocular camera so that it can only pass infrared light of a specified wavelength, such as 850nm or 940nm.

[0005] 3) Add an infrared filler around the camera lens of the binocular camera;

[0006] 4) By placing infrared reflective devices (such as infrared reflective balls) in the space, under the illumination of the infrared supplementary light source and after filtering through the infrared filter film, the area where the infrared reflective ball is located will appear bright (white) in the image captured by the binocular camera, while other background areas will appear dark (black). Figure 1 As shown;

[0007] 5) Using threshold segmentation and circle fitting techniques, determine the image coordinates of the center of the infrared reflective sphere region in the first image (left side) captured by the first camera and the second image (right side) captured by the second camera;

[0008] 6) The center coordinates of the circles calculated from the left and right images need to be matched one by one to ensure accuracy. Figure 1 For example, the pixel coordinates of the five center points are calculated on the left and the right. After matching the five pixel coordinates on the left and right, the accurate three-dimensional spatial position of the center point can be calculated by using the pixel position of the center point in the first and second images and the internal and external parameters of the first and second cameras calculated in step 1).

[0009] As described above, the key step is to match the center pixel coordinates of the left and right images one by one. If the matching relationship of the center coordinates is wrong, the calculated three-dimensional spatial position of the center of the infrared reflective sphere will have a large error.

[0010] In step 1), the fundamental matrix between the stereo cameras can be obtained through stereo calibration. Based on the epipolar geometry principle of the stereo cameras (e.g., ... Figure 2 As shown), X is the three-dimensional spatial coordinate of the infrared reflector, C1 is the imaging spatial coordinate of the left camera, C2 is the imaging spatial coordinate of the right camera, X1 is the center pixel coordinate of the infrared reflector on the left camera [x1, y1, 1], X2 is the center pixel coordinate of the infrared reflector on the right camera [x2, y2, 1]. By connecting X, C1, and C2, a planar triangle is obtained. The line that intersects the left camera's imaging plane I1 is l1, and the line that intersects the right camera's imaging plane I2 is l2. l1 and l2 are the so-called epipolar lines. All points on the left epipolar line and all points on the right epipolar line satisfy the following formula (1):

[0011]

[0012] However, in practice, there is a certain error in obtaining the center coordinate position of the circle through circle fitting. Therefore, the following method is often used to match the center coordinates of the left and right images: any center coordinate in the left image and all center coordinates in the right image are calculated according to formula (1). The theoretical value obtained is 0. The value obtained by calculating formula (1) from a certain center coordinate on the right and the specified center coordinate on the left is the smallest. That is, it is determined that the center on the right is matched with the specified center on the left. This matching method is called epipolar constraint.

[0013] like Figure 2 As shown, the above method is commonly used in existing technologies for matching the left and right center points, such as the published domestic patent 201510873609.4. However, the above operation has certain problems. It can be considered a one-to-one match as long as the center coordinates of the infrared reflective sphere in the left and right images satisfy the epipolar constraint condition. However, through… Figure 3 It can be seen that the projection center of the three infrared reflective spheres on the left image is x1, while there are three projection centers on the right image. The three centers on the right and the center on the left all satisfy the epipolar constraint condition. Therefore, the epipolar constraint may cause mismatch in certain specific cases. At the same time, due to the error in the calculation of the center, this mismatch problem is very common, resulting in the calculation of incorrect three-dimensional space pseudopoints.

[0014] Domestic patent 201710646652.6 and others proposed a method for eliminating false matching points based on the three-dimensional geometric size characteristics of the reflective sphere markers. Although this method solves the false point problem to some extent, it still has the following problems: 1) The calculation is complex and time-consuming, which reduces the real-time performance of capturing the spatial position of the infrared reflective sphere; 2) In practical applications, users need to frequently change the infrared reflective devices, such as the size of the reflective sphere, which leads to the size of the reflective sphere often being inconsistent in different usage scenarios. In this case, the method will fail. Specifically, this patent proposes a method for correcting binocular mismatches. If a binocular mismatch occurs (the pixel center of one infrared reflective marker in the left eye corresponds to the pixel centers of multiple infrared reflective markers in the right eye, or vice versa, or the pixel center of one infrared reflective marker in the right eye corresponds to the pixel centers of multiple infrared reflective markers in the left eye), the method calculates the 3D spatial position of all mismatches. Based on the known diameters of the infrared reflective markers, multiple virtual reflective spheres are constructed. Then, the tangent between the left-eye camera and the multiple virtual infrared reflective spheres is calculated, eliminating incorrect virtual reflective spheres and retaining the correct ones as the real reflective spheres. This method has several drawbacks: First, the process of calculating the virtual reflective spheres and the tangent is cumbersome, consuming significant computational resources and failing to guarantee real-time performance. Second, the calculation process requires knowing the diameter of the infrared reflective spheres to calculate the tangent. In practice, lens users often use infrared reflective spheres or infrared reflective disks of different sizes, which cannot be guaranteed to be consistent, causing the algorithm to fail.

[0015] Therefore, how to overcome the shortcomings of the existing technology is the subject of this invention. Summary of the Invention

[0016] The purpose of this invention is to provide an infrared spatial positioning system and method based on multi-view vision.

[0017] To achieve the above objectives, the technical solution adopted by the present invention at the system level is as follows:

[0018] An infrared spatial positioning system based on multi-view vision includes a multi-view infrared camera and a processing unit;

[0019] The multi-view infrared camera includes N cameras, N≥3, wherein the imaging point of at least one camera is not on the same straight line as the imaging points of the other cameras; and each camera is communicatively connected to the processing unit.

[0020] The processing unit synchronously acquires images captured by each camera in the multi-view infrared camera and calculates the three-dimensional spatial position of the infrared marker based on the images.

[0021] The relevant content in the above technical solution is explained as follows:

[0022] 1. In the above scheme, "the imaging point of at least one of the cameras is not on the same straight line as the imaging points of the other cameras", that is, the lines connecting the imaging points of the cameras are not on the same straight line.

[0023] 2. In the above scheme, "the three-dimensional spatial position of the infrared marker is calculated" to realize a real-time three-dimensional spatial positioning system.

[0024] 3. In the above scheme, the processing unit includes an image processing module and a three-dimensional reconstruction calculation module;

[0025] The input of the image processing module is communicatively connected to the signal output of the camera, and the output of the image processing module is communicatively connected to the signal input of the 3D reconstruction calculation module; the 3D reconstruction calculation module can be communicatively connected to the host computer via a transmission module (USB or Ethernet).

[0026] The number of image processing modules can be N or one.

[0027] Furthermore, the image processing module, the 3D reconstruction calculation module, and the transmission module can be integrated into a processing unit, which can be a chip.

[0028] 4. In the above scheme, the image processing module includes processors such as FPGA processors that have real-time image calculation functions and parallel processing capabilities, which can realize fast image calculation functions; the three-dimensional reconstruction calculation module includes processors such as ARM processors and Intel processors that have complex calculation functions and serial processing capabilities, which can realize complex three-dimensional coordinate calculation functions in real time.

[0029] 5. The above solution also includes a temperature sensor, which is located in the multi-view infrared camera and is communicatively connected to the processing unit.

[0030] Furthermore, the temperature sensor may include N sensors, each corresponding to one of the cameras.

[0031] 6. In the above scheme, the infrared marker can be an infrared reflective ball, such as an active and passive infrared reflective ball, or an active and passive infrared reflective disk, etc.

[0032] 7. In the above solution, the camera may include an infrared filter film or infrared filter sheet, and an infrared filler.

[0033] To achieve the above objectives, the technical solution adopted by the present invention at the method level is as follows:

[0034] An infrared spatial positioning method based on multi-view vision is implemented through an infrared spatial positioning system, the system including a multi-view infrared camera and a processing unit;

[0035] The multi-view infrared camera includes N cameras, N≥3, wherein the imaging point of at least one camera is not on the same straight line as the imaging points of the other cameras; and each camera is communicatively connected to the processing unit.

[0036] The processing unit synchronously acquires images captured by each camera in the multi-view infrared camera, and calculates the three-dimensional spatial position of the infrared marker based on these images;

[0037] The infrared spatial positioning method includes:

[0038] Step 1: Combine N cameras in pairs to form For combinations;

[0039] Each camera simultaneously acquires images and sends them to the processing unit for processing, thereby obtaining the morphological center coordinates of the infrared markers in each image;

[0040] Step 2: Apply the principle of limit constraints to... The morphological center coordinates of the infrared markers captured by each camera in the camera array are matched one by one.

[0041] If the morphological center coordinates of a certain infrared marker are in If the same matching relationship is satisfied in all combinations, that is, the morphological coordinate center of the infrared marker is successfully matched in all images;

[0042] If the morphological center coordinates of an infrared marker do not satisfy the same matching relationship in any pair of combinations, then the infrared marker is determined to be a mismatched marker.

[0043] The relevant content in the above technical solution is explained as follows:

[0044] 1. In the above scheme, "the imaging point of at least one of the cameras is not on the same straight line as the imaging points of the other cameras", that is, the lines connecting the imaging points of the cameras are not on the same straight line.

[0045] 2. In the above scheme, "the three-dimensional spatial position of the infrared marker is calculated" to realize a real-time three-dimensional spatial positioning system.

[0046] 3. In the above scheme, in step one, N cameras are combined in pairs to form... For combinations, since the imaging points of each camera are not on the same straight line, even if two infrared markers are located on the same pole line in any pair of camera combinations, they will not be on the same pole line in the other two pairs of camera combinations.

[0047] 4. In the above scheme, if there are three cameras, the cameras will be paired to form three pairs. If there are more than three cameras, taking four cameras as an example, the cameras will be paired to form six pairs.

[0048] 5. In the above scheme, the meaning of "one-to-one matching" in step two is as follows: taking three cameras as an example, that is, the morphological center coordinates captured by the first camera and the second camera are matched one-to-one, the morphological center coordinates captured by the second camera and the third camera are matched one-to-one, and the morphological center coordinates captured by the first camera and the third camera are matched one-to-one.

[0049] 6. In the above scheme, in step two, the meaning of "satisfying the same matching relationship" is as follows: taking three cameras as an example, that is, the morphological center coordinates calculated from the images captured by the first and second cameras are successfully matched, the morphological center coordinates calculated from the images captured by the second and third cameras are successfully matched, and the morphological center coordinates calculated from the images captured by the first and third cameras are successfully matched.

[0050] 7. The above scheme also includes a pre-preparation step, in which each of the cameras is calibrated by the processing unit, and the internal parameters of each camera and the external parameters and fundamental matrix between any two cameras are calculated.

[0051] The intrinsic parameters include the camera's intrinsic parameter matrix, focal length, principal point, radial distortion, and tangential distortion. The extrinsic parameters include the rotation matrix and translation matrix between the two cameras. The fundamental matrix is ​​an algebraic representation of the epipolar set, which describes the transformation relationship from a point on one image plane to the corresponding epipolar line on the other image plane in binocular vision.

[0052] 8. In the above scheme, the infrared marker can be an infrared reflective ball, such as an active and passive infrared reflective ball, or an active and passive infrared reflective disk, etc.

[0053] In step one, the processing unit synchronously obtains the images captured by each camera, performs threshold segmentation and circular fitting on the images, and then obtains the center coordinates of the infrared reflective sphere in each image.

[0054] 9. In the above scheme, step three involves using multiple pairs of cameras to calculate the three-dimensional spatial position of the same infrared marker based on the calibrated internal and external parameters of the cameras, obtaining multiple three-dimensional spatial coordinates. The number of these three-dimensional spatial coordinates corresponds to the number of cameras. Then, the average value of these three-dimensional spatial coordinates is calculated to obtain the average three-dimensional spatial position of the infrared marker.

[0055] 10. In the above scheme, during the pre-preparation step, a temperature sensor is installed in the multi-view infrared camera, and the temperature sensor is communicatively connected to the processing unit;

[0056] Then, multiple calibrations were performed at different temperatures, and the internal parameters of each camera, as well as the external parameters and fundamental matrix between any two cameras, were recorded at different temperatures.

[0057] Finally, when performing three-dimensional reconstruction of the morphological coordinate center of the matched infrared markers, the optimal parameters are selected for calculation based on the current temperature information obtained from the temperature sensor.

[0058] The working principle and advantages of this invention are as follows:

[0059] First, this invention avoids the problem of misregistration of the center coordinates of infrared markers in binocular imaging. By using multi-view cameras and establishing multiple matching relationships, the center coordinates are considered successfully matched only when all matching relationships between any two cameras are satisfied, thus completely avoiding mismatch problems. Therefore, when this invention's infrared multi-view positioning system is used as a positioning component in a surgical navigation system or motion analysis system, it ensures that during equipment operation, incorrect calculations of the infrared marker's three-dimensional spatial position due to mismatched center coordinates will not lead to surgical positioning failures or motion analysis errors, thereby ensuring surgical safety.

[0060] Second, after achieving infrared coordinate center matching, the multi-view infrared camera of this invention calculates the three-dimensional coordinate positions of any two cameras. Averaging these calculated three-dimensional coordinate positions yields a more accurate three-dimensional spatial position of the infrared marker's center. This improves the accuracy of surgical positioning and motion analysis when the infrared binocular positioning system of this invention is used as a positioning component in a surgical navigation system or motion analysis system. Consequently, it meets the requirements of high-precision surgery and improves surgical outcomes. Attached Figure Description

[0061] Appendix Figure 1 An image containing several infrared reflectors, captured by a binocular camera;

[0062] Appendix Figure 2 The geometric view principle of a binocular camera;

[0063] Appendix Figure 3The geometric view principle of a binocular camera (three infrared reflectors are located on the same straight line);

[0064] Appendix Figure 4 This is a diagram showing the frontal positional relationship of each camera in the trinocular camera of an embodiment of the present invention;

[0065] Appendix Figure 5 This describes the geometric view principle of the trinocular camera in an embodiment of the present invention.

[0066] Appendix Figure 6 This is a schematic diagram illustrating the principle of the positioning method according to an embodiment of the present invention;

[0067] Appendix Figure 7 The principle block diagram of the positioning system in the embodiment of the present invention Figure 1 ;

[0068] Appendix Figure 8 The principle block diagram of the positioning system in the embodiment of the present invention Figure 2 . Detailed Implementation

[0069] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0070] Example: The present invention will be clearly described below with illustrations and detailed description. Any person skilled in the art who understands the examples of the present invention can make changes and modifications based on the technology taught in the present invention without departing from the spirit and scope of the present invention.

[0071] The terms “include,” “including,” and “have” used in this article are all open-ended, meaning they include but are not limited to.

[0072] Unless otherwise specified, the terms used herein generally have their ordinary meaning in the context of the art, the subject matter, and the specific context. Certain terms used to describe this case will be discussed below or elsewhere in this specification to provide additional guidance to those skilled in the art in describing the case.

[0073] An infrared spatial positioning method based on multi-view vision is implemented through an infrared spatial positioning system, which includes a multi-view infrared camera and a processing unit.

[0074] The multi-view infrared camera includes at least three cameras, wherein the imaging point of at least one of the cameras is not on the same straight line as the imaging points of the other cameras (see...). Figure 4 This means that the imaging points of each camera are not connected on the same straight line; and each of the cameras is communicatively connected to the processing unit.

[0075] The processing unit synchronously acquires images captured by each camera in the multi-view infrared camera, and calculates the three-dimensional spatial position of the infrared marker based on these images, thereby realizing a real-time three-dimensional spatial positioning system.

[0076] The processing unit includes an image processing module and a three-dimensional reconstruction calculation module.

[0077] The input of the image processing module is communicatively connected to the signal output of the camera, and the output of the image processing module is communicatively connected to the signal input of the 3D reconstruction calculation module; the 3D reconstruction calculation module can be communicatively connected to the host computer via a transmission module (USB or Ethernet).

[0078] The number of image processing modules can be N or one.

[0079] Furthermore, the image processing module, the 3D reconstruction calculation module, and the transmission module can be integrated into a processing unit, which can be a chip.

[0080] Preferably, the image processing module includes a processor such as an FPGA processor that has real-time image computing capabilities and parallel processing capabilities, enabling fast image computing; the 3D reconstruction computing module includes a processor such as an ARM processor or an Intel processor that has complex computing capabilities and serial processing capabilities, enabling real-time complex 3D coordinate calculation.

[0081] Preferably, it also includes a temperature sensor, which is located in the multi-view infrared camera and is communicatively connected to the processing unit.

[0082] Furthermore, the temperature sensor may include N sensors, each corresponding to one of the cameras.

[0083] The infrared marker can be an infrared reflective sphere, such as an active or passive infrared reflective sphere, or an active or passive infrared reflective disk. The camera may include an infrared filter film or infrared filter sheet, and an infrared filler.

[0084] like Figure 6 As shown, taking a scenario with three cameras, the infrared spatial positioning method includes:

[0085] Step 1: Combine the three cameras in pairs to form three pairs. The first pair is the first camera and the second camera, the second pair is the first camera and the third camera, and the third pair is the second camera and the third camera.

[0086] Each camera simultaneously captures images and sends them to the processing unit;

[0087] The processing unit synchronously acquires images captured by each camera, processes the images, and then obtains the morphological center coordinates of the infrared markers in each image.

[0088] Step 2: Using the principle of limit constraints, match the morphological center coordinates of the infrared markers captured by each camera in the three pairs of camera combinations one by one.

[0089] If the morphological center coordinates of an infrared marker satisfy the same matching relationship in all three pairs of combinations, it is determined that the morphological coordinate center of the infrared marker has been successfully matched in all images.

[0090] If the morphological center coordinates of an infrared marker do not satisfy the same matching relationship in any pair of combinations, then the infrared marker is determined to be a mismatched marker.

[0091] Preferably, the method further includes a pre-preparation step, in which each of the cameras is calibrated by the processing unit, and the internal parameters of each camera and the external parameters and fundamental matrix between any two cameras are calculated.

[0092] Preferably, the infrared marker is an infrared reflective sphere. In step one, the processing unit simultaneously acquires images captured by each camera, performs threshold segmentation and circular fitting on the images, and then obtains the center coordinates of the infrared reflective sphere in each image.

[0093] Preferably, the method further includes: Step 3, using multiple pairs of cameras to calculate the three-dimensional spatial position of the same infrared marker based on the calibrated internal and external parameters of the cameras, obtaining multiple three-dimensional spatial coordinates, the number of which corresponds to the number of cameras; then averaging these three-dimensional spatial coordinates to obtain the precise three-dimensional spatial position of the infrared marker.

[0094] Preferably, in the pre-preparation step, a temperature sensor is provided in the multi-view infrared camera, and the temperature sensor is communicatively connected to the processing unit;

[0095] Then, multiple calibrations were performed at different temperatures, and the internal parameters of each camera, as well as the external parameters and fundamental matrix between any two cameras, were recorded at different temperatures.

[0096] Finally, when performing three-dimensional reconstruction of the morphological coordinate center of the matched infrared markers, the optimal parameters are selected for calculation based on the current temperature information obtained from the temperature sensor.

[0097] This design can further improve the accuracy of reconstructing the three-dimensional spatial position of infrared markers. This is because the connection structure between cameras, sensors, and lenses will all undergo slight deformation with temperature changes. By setting a temperature sensor, the accuracy reduction caused by errors introduced by these deformations can be avoided.

[0098] like Figure 7 As shown, a multi-camera system and its respective image processing module serve as image acquisition and analysis components. The raw images acquired by the cameras are input into the image processing module, which performs image processing operations on the raw images (image thresholding, circular fitting, calculation of the morphological coordinate center of the infrared marker, etc.). Each image processing module inputs the calculated morphological coordinate center into the 3D reconstruction calculation module. The 3D reconstruction calculation module synchronously acquires data recorded by the temperature sensor and reads the internal parameters of each camera at different temperatures, as well as the external parameters and fundamental matrix between any two cameras. It then performs 3D reconstruction of the infrared marker. The reconstruction result is transmitted to the host computer via USB or Ethernet through the transmission module.

[0099] like Figure 8 As shown, with the improvement of hardware technology and algorithms, in order to reduce the hardware cost and equipment size of multi-view infrared spatial positioning systems, multiple image processing modules can be merged into one, or even the image processing module, 3D reconstruction calculation module, and transmission module can be merged into the same processing unit chip.

[0100] like Figure 5As shown, X represents the three-dimensional spatial coordinates of the infrared reflective sphere, C1 represents the imaging spatial coordinates of Camera 1, C2 represents the imaging spatial coordinates of Camera 2, C3 represents the imaging spatial coordinates of Camera 3, X1 represents the center pixel coordinates of the infrared reflective marker in Camera 1 [x1, y1, 1], X2 represents the center pixel coordinates of the infrared marker in Camera 2 [x2, y2, 1], and X3 represents the center pixel coordinates of the infrared marker in Camera 3 [x3, y3, 1]. Connecting lines X, C1, and C3, we obtain a planar triangle. The line intersecting this planar triangle with the camera plane of Camera1 is l13, and the line intersecting this planar triangle with the camera plane of Camera3 is l31. L13 and l31 are the epipolar lines of Camera1 and Camera3, respectively. Connecting lines X, C1, and C2, we obtain a planar triangle. The line intersecting this planar triangle with the camera plane of Camera1 is l12, and the line intersecting this planar triangle with the camera plane of Camera2 is l21. L12 and l21 are the epipolar lines of Camera1 and Camera2, respectively. Connecting lines X, C2, and C3, we obtain a planar triangle. The line intersecting this planar triangle with the camera plane of Camera2 is l23, and the line intersecting this planar triangle with the camera plane of Camera3 is l32. L23 and l32 are the epipolar lines of Camera2 and Camera3, respectively. It can be observed that in the camera plane of Camera 1, l12 and l13 are not the same straight line; in the camera plane of Camera 2, l21 and l23 are not the same straight line; and in the camera plane of Camera 3, l31 and l32 are not the same straight line. Therefore, the center coordinates of point X in the three cameras are not on the same polar line. By applying the limit constraint principle to match the center coordinates of the circles captured by the first and second cameras one by one, matching the center coordinates of the circles captured by the second and third cameras one by one, and matching the center coordinates of the circles captured by the first and third cameras one by one, the center position of a specific infrared marker satisfies the same matching relationship in all three combinations, thus completing the one-to-one matching of the three camera planes.

[0101] In the process of reconstructing the three-dimensional coordinates of the center of the infrared reflective marker, the three-dimensional coordinates X1[x1, y1, z1] are calculated based on the intrinsic and extrinsic parameters of Camera1 and Camera2, the three-dimensional coordinates X2[x2, y2, z2] are calculated based on the intrinsic and extrinsic parameters of Camera2 and Camera3, and the three-dimensional coordinates X3[x3, y3, z3] are calculated based on the intrinsic and extrinsic parameters of Camera1 and Camera3. The final three-dimensional spatial coordinates X[(x1 + x2 + x3) / 3, (y1 + y2 + y3) / 3, (z1 + z2 + z3) / 3] are obtained by calculating the average value. The above method can greatly reduce the error. Compared to a stereo lens (infrared wavelength 850nm, camera resolution 2048*1024, camera spacing 0.3m) with an infrared marker capture error of 0.2-0.25mm at 2 meters, a tri-lens lens (infrared wavelength 850nm, camera resolution 2048*1024, camera spacing 0.3m, 0.155m, 0.155m respectively) with an infrared marker capture error of 0.1-0.15mm at 2 meters.

[0102] To demonstrate the advantage of this invention in terms of low matching error compared to traditional binocular methods, the following experiment was designed: different numbers of infrared reflective spheres were collected, and images were continuously acquired at a speed of 60Hz for more than 5 seconds. The infrared reflective spheres were held by a person and moved randomly. The images acquired by the tri-lens camera and the binocular camera were saved. For each saved image, the morphological center coordinates of the infrared reflective spheres were matched. If a mismatch occurred, it was determined to be an erroneous frame. The proportion of erroneous frames to the total number of frames was calculated, and the superiority of this method was determined accordingly (see Table 1).

[0103]

[0104] Table 1

[0105] To demonstrate the accuracy advantage of this invention compared to the traditional binocular solution, the following experiment was designed. A support was designed with infrared reflective spheres fixed at both ends, with a spacing of 0.3m between the spheres. High-precision machining ensured that the spacing error was less than 0.01mm. The support was swung at a distance of 2.0m-2.5m from both the binocular and tri-lens cameras, continuously acquiring images at a frequency of 60Hz for 5s. The center position of the infrared reflective spheres and the spacing l (in mm) between them were calculated. The error was calculated as follows: e = abs(l-300mm). The error for each frame was calculated, and the mean and variance were calculated. The mean error for the tri-lens camera was 0.12mm, and the variance was 0.1mm. The mean error for the binocular camera was 0.18mm, and the variance was 0.3mm. This demonstrates that the tri-lens camera has a significant accuracy advantage over the binocular camera.

[0106] Note: The three-lens and two-lens cameras mentioned above use the same lens, CCD sensor and computing unit. The distance between the two farthest cameras in the three-lens camera is equal to the distance between the cameras in the two-lens camera.

[0107] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. An infrared spatial positioning system based on multi-view vision, characterized in that: Includes a multi-view infrared camera and a processing unit; The multi-view infrared camera includes N cameras, N≥3, wherein the imaging point of at least one camera is not on the same straight line as the imaging points of the other cameras; and each camera is communicatively connected to the processing unit. The processing unit synchronously acquires images captured by each camera in the multi-view infrared camera, and calculates the three-dimensional spatial position of the infrared marker based on the images; It also includes a temperature sensor, which is located in the multi-view infrared camera and is communicatively connected to the processing unit; Infrared spatial positioning methods include: Step 1: Combine N cameras in pairs to form For combinations; Each camera simultaneously acquires images and sends them to the processing unit for processing, thereby obtaining the morphological center coordinates of the infrared markers in each image; Step 2: Apply the principle of limit constraints to... The morphological center coordinates of the infrared markers captured by each camera in the camera array are matched one by one. If the morphological center coordinates of a certain infrared marker are in If the same matching relationship is satisfied in all combinations, that is, the morphological coordinate center of the infrared marker is successfully matched in all images; If the morphological center coordinates of an infrared marker do not satisfy the same matching relationship in any pair of combinations, then the infrared marker is determined to be a mismatched marker. It also includes a pre-preparation step, in which each of the cameras is calibrated by the processing unit, and the internal parameters of each camera and the external parameters and fundamental matrix between any two cameras are calculated. The infrared marker is an infrared reflective sphere; In step one, the processing unit synchronously obtains the images captured by each camera, performs threshold segmentation and circular fitting on the images, and then obtains the center coordinates of the infrared reflective sphere in each image; Also includes: Step 3: Using multiple pairs of cameras, calculate the three-dimensional spatial position of the same infrared marker based on the calibrated internal and external parameters of the cameras, and obtain multiple three-dimensional spatial coordinates. The number of these three-dimensional spatial coordinates corresponds to the number of camera pairs. Then, the average value of these three-dimensional spatial coordinates is calculated to obtain the average three-dimensional spatial position of the infrared marker. In the pre-preparation step, a temperature sensor is installed in the multi-view infrared camera, and the temperature sensor is communicatively connected to the processing unit. Then, multiple calibrations were performed at different temperatures, and the internal parameters of each camera, as well as the external parameters and fundamental matrix between any two cameras, were recorded at different temperatures. Finally, when performing three-dimensional reconstruction of the morphological coordinate center of the matched infrared markers, the optimal parameters are selected for calculation based on the current temperature information obtained from the temperature sensor.

2. The infrared spatial positioning system according to claim 1, characterized in that: The processing unit includes an image processing module and a three-dimensional reconstruction calculation module; The input of the image processing module is communicatively connected to the signal output of the camera, and the output of the image processing module is communicatively connected to the signal input of the three-dimensional reconstruction calculation module. The number of image processing modules can be N or one.

3. The infrared spatial positioning system according to claim 2, characterized in that: The image processing module includes an FPGA processor, and the 3D reconstruction calculation module includes an ARM processor.

Images