Optical monitoring system based on multi-viewing for space station

Abstract

The optical monitoring system based on multi-eye vision applied to the space station belongs to the field of space monitoring. It solves the problem that the monitoring robot in the existing space station needs to adjust to different postures when monitoring different directions, and the posture adjustment consumes energy, resulting in energy waste and reduced monitoring efficiency. The multi-eye vision unit of the present invention is used to collect multiple images under different viewing angles, and synchronously send the collected multiple images to the image processor to pre-process the multiple images, and splice the pre-processed images to obtain a panoramic image of the space station cabin; the multi-eye vision unit includes multiple surround-view cameras and two front and rear view cameras, and the two front and rear view cameras are respectively arranged on the upper and lower bottom surfaces of the structural matrix, and the multiple surround-view cameras are located in the same plane and are evenly arranged around the circumference of the structural matrix. The present invention mainly monitors the panoramic image of the cabin of the space station.

Description

Technical Field

[0001] The invention belongs to the field of space monitoring. Background Art

[0002] With its unique experimental environment and conditions, the space station has become an ideal area for various scientific experimental activities, allowing a large number of scientific researchers to realize their dreams of exploring outer space and discover a variety of special materials and new products. The space station also serves as an outer space battlefield command and reconnaissance center, realizing the launch and testing of weapons, command, coordination and communication of the military, and building a complete tactical system.

[0003] In order to ensure the safety and daily observation of the space station, the space station needs cameras to monitor in each cabin. Currently, most of the cameras use fixed positions to monitor the cabin environment of the space station, but there are blind spots for complex scenes, and it is impossible to observe specific areas at close range. Using robots to monitor the space station environment is an effective method. Due to the special gravity-free environment of space, the monitoring robot only needs to carry energy to provide its own power. However, the monitoring robot may be disturbed and change its posture during movement, or even cause yaw. Therefore, when monitoring the space cabin, the monitoring robot in the space station needs to adjust to different postures to monitor different directions. The posture adjustment consumes energy, causes unnecessary fuel consumption, and reduces the working efficiency of the monitor. The above problems need to be solved. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the monitoring robots in the existing space station need to adjust to different postures when monitoring different directions of the space capsule, and the posture adjustment consumes energy, causing energy waste and reducing the monitoring efficiency. The present invention provides an optical monitoring system based on multi-eye vision applied to the space station.

[0005] An optical monitoring system based on multi-camera vision applied to a space station includes a multi-camera vision unit, a structural matrix, a power module, and an image processor; the power module and the image processor are fixed in an internal cavity of the structural matrix;

[0006] A power module, used to supply power to the multi-eye vision unit and the image processor;

[0007] A multi-eye vision unit is used to collect multiple images from different viewing angles and send the collected multiple images to the image processor synchronously;

[0008] An image processor is used to pre-process multiple images and stitch the pre-processed images to obtain a panoramic image of the space station cabin;

[0009] The multi-view vision unit includes multiple surround-view cameras and two front- and rear-view cameras, and the two front- and rear-view cameras are respectively arranged on the upper and lower bottom surfaces of the structural matrix. The multiple surround-view cameras are located in the same plane and are evenly arranged in the circumference of the structural matrix.

[0010] Preferably, the optical axes of all surround view cameras are in the same plane and intersect at one point, and the angle between the optical axes of any two adjacent surround view cameras is α; the horizontal field of view angle of each surround view camera is β, and the vertical field of view angle is β′;

[0011] The lines connecting all surround-view cameras form a circle with a center of O and a radius of r. The vertical distance between the center of the circle and the line connecting any two adjacent surround-view cameras is

[0012] The position of the first surround-view camera of any two adjacent surround-view cameras is recorded as point A, and the position of the second surround-view camera is recorded as point B. The distance between the two adjacent surround-view cameras is

[0013] When the overlap of the fields of view of any two adjacent surround-view cameras is u, the minimum working distance of each surround-view camera is R=d1+d2. A circle is drawn with O as the center and R as the radius. The intersection points of the two field of view lines of the overlapped part of the field of view of any two adjacent surround-view cameras are C and D respectively; the vertical distance from point D to the line connecting the two adjacent surround-view cameras is

[0014] The vertical field of view of each front- and rear-view camera is β q , and the offset of the front-view and rear-view cameras relative to the intersection of the optical axes of all surround-view cameras is γ′, and

[0015] Preferably, the image processor is used to preprocess multiple images and stitch the preprocessed images to obtain panoramic information inside the space station cabin, including the following steps:

[0016] S1, the image processor preprocesses each image, and the process of preprocessing each image is: performing image noise elimination, geometric correction and histogram equalization on each image in turn to complete the preprocessing;

[0017] S2, using an image stitching algorithm to stitch the preprocessed images to obtain an initial stitching image;

[0018] S3. Eliminate the black edges of the initial stitched image to obtain a panoramic image of the space station cabin.

[0019] The beneficial effects brought by the present invention are:

[0020] The optical monitoring system based on multi-eye vision applied to the space station described in the present invention can solve the problem that the existing monitors need to adjust their own posture and consume additional fuel during use. The present invention can achieve full coverage monitoring of the surrounding space without posture adjustment.

[0021] The optical monitoring system based on multi-eye vision applied to the space station described in the present invention has a compact structure and a large reach, avoiding blind spots in monitoring. It uses a multi-eye camera to obtain panoramic image information inside the space station cabin, and combined with its own posture information, it can output a panoramic monitoring image of the interior of the space cabin in any posture. The monitoring system can be spherical, cylindrical, etc., and is flexible to move without adjusting its own posture, and does not require additional fuel consumption for posture adjustment.

[0022] The use of a multi-camera surround arrangement can avoid excessive distortion caused by the use of fisheye cameras; the use of multi-camera vision units to collect image information of the interior environment of the space station, the fields of view of each camera overlap, and a panoramic image of the interior of the space station can be obtained with a wide field of view. Reduce image distortion and use multiple images for stitching. Since the overlapping area of ​​photos taken by adjacent cameras is large, the images on the left and right edges of each photo can be removed during stitching, and only the middle part with less distortion can be retained for stitching, which can reduce the distortion problem caused by the camera arrangement. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the structure of the optical monitoring system based on multi-eye vision applied to the space station according to the present invention; Figure 1 a is the front view of the optical monitoring system; Figure 1 b is a schematic diagram of the structure of the internal cavity of the structural matrix 2 of the optical monitoring system; Figure 1 c is a schematic diagram of the three-dimensional structure of the optical monitoring system;

[0024] Figure 2 is a schematic diagram of the horizontal field of view of the surround camera 1-1 in the downward direction of the optical monitoring system;

[0025] Figure 3 It is a schematic diagram of the vertical field of view of the surround camera 1-1 in the main viewing direction of the optical monitoring system; wherein,

[0026] K is the intersection of the vertical field line of the front and rear view camera 1-2 and the vertical field line of the surround view camera 1-1, J is the location of the front and rear view camera 1-2, N is the intersection of the perpendicular line of point K and the extension line of the straight line OA; M is the intersection of the perpendicular line of point J and the straight line KN;

[0027] Figure 4 Schematic diagram of blind areas and overlapping areas of the fields of view of multiple surround view cameras 1-1. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0030] Embodiment 1:

[0031] See Figure 1 and Figure 3 The present embodiment is described. The optical monitoring system based on multi-eye vision applied to a space station described in the present embodiment includes a multi-eye vision unit 1, a structural base 2, a power module 3, and an image processor 4;

[0032] A power module 3 and an image processor 4 are fixed in the internal cavity of the structural base 2;

[0033] A power module 3, used to supply power to the multi-eye vision unit 1 and the image processor 4;

[0034] The multi-eye vision unit 1 is used to collect multiple images at different viewing angles and synchronously send the collected multiple images to the image processor 4;

[0035] An image processor 4 is used to pre-process a plurality of images and stitch the pre-processed images to obtain a panoramic image of the space station cabin;

[0036] The multi-eye vision unit 1 includes multiple surround view cameras 1-1 and two front and rear view cameras 1-2, and the two front and rear view cameras 1-2 are respectively arranged on the upper and lower bottom surfaces of the structural base 2. The multiple surround view cameras 1-1 are located in the same plane and are evenly arranged around the circumference of the structural base 2.

[0037] In this embodiment, the optical monitoring system based on multi-eye vision applied to the space station has a compact structure and a large reach, avoiding blind spots in monitoring, and can realize autonomous movement in the cabin to cover all information of the entire space station and realize the status monitoring of the space station. There is no need to adjust the posture of the monitoring robot, which reduces fuel consumption and improves work efficiency. At the same time, it is more flexible and faster than manual inspection; the use of a multi-camera surround arrangement mode can avoid excessive distortion caused by the use of fisheye cameras; the multi-eye vision unit 1 collects image information of the cabin environment of the space station, and the fields of view of each camera are intersecting, which can obtain a panoramic image of the cabin of the space station with a wide field of view.

[0038] For further information, see Figure 2 and Figure 4 , the optical axes of all surround-view cameras 1-1 are in the same plane and intersect at one point, and the angle between the optical axes of any two adjacent surround-view cameras 1-1 is α; the field of view angle of each surround-view camera 1-1 in the horizontal direction is β, and the field of view angle in the vertical direction is β′; the line connecting all surround-view cameras 1-1 forms a circle, and the center of the circle is O and the radius is r. The vertical distance between the center of the circle and the line connecting any two adjacent surround-view cameras 1-1 is

[0039] The position of the first surround view camera 1-1 of any two adjacent surround view cameras 1-1 is recorded as point A, and the position of the second surround view camera 1-1 is recorded as point B. The distance between the two adjacent surround view cameras 1-1 is When the overlap of the fields of view of any two adjacent surround-view cameras 1-1 is u, the minimum working distance of each surround-view camera 1-1 is R=d1+d2. A circle is drawn with O as the center and R as the radius. The intersection points of the circle and the two field of view lines of the overlapping part of the fields of view of any two adjacent surround-view cameras 1-1 are C and D respectively; the vertical distance from point D to the line connecting the two adjacent surround-view cameras 1-1 is

[0040] In this embodiment, the field of view overlap u of any two adjacent surround view cameras 1-1 is set. Under the condition that the field of view overlap u is satisfied, the minimum working distance of each surround view camera 1-1 is set to R, and R=d1+d2. The process of verifying R=d1+d2 is as follows, and see Figure 2 , Figure 2 The intersection point of the straight line DF and the straight line AB is point F, and DF=d2.

[0041] Figure 2 The CD segment is the area observed by two adjacent cameras at the minimum working distance. u is the overlap of the fields of view of adjacent cameras, which means the ratio of the overlap angle of the two cameras to the field of view angle of the camera in the minimum working range. Points G and D are the intersections of the left and right lines of the field of view of the surround view camera 1-1 at point A and the circle with a radius of R, respectively. Points C and H are the intersections of the left and right lines of the field of view of the surround view camera 1-1 at point B and the circle with a radius of R, respectively. Then, there exists:

[0042] ∠CAD=∠CBD=uβ Formula 1;

[0043]

[0044]

[0045] ∠ADB=(1-u)β-α Formula 4;

[0046] From the law of sine we can conclude that:

[0047]

[0048] Substituting AB, ∠DBA, ∠ADB into calculation formula 5, we can get:

[0049]

[0050] Substituting AD and ∠DAB into the formula DF=ADsin(∠DAB), we can get

[0051]

[0052] From the trigonometric relationship, we know that:

[0053] AF=ADcos(∠DAB) Formula 8;

[0054] Substituting the expressions of AD and ∠DAB into formula 8, we can obtain:

[0055]

[0056] and

[0057] Through many experiments and calculations, it is found that when the minimum working distance of the surround camera 1-1 is R≈2r, the ratio of EF to OE+DF is usually small. For example, when u=0.2, In this case, when the minimum working distance limit is met, When calculating the minimum working distance R = OD, if OE + DF is used instead, the relative error value generated is

[0058]

[0059] This error value ε is within the acceptable range and can be ignored, so the minimum working distance of the surround camera 1-1 is approximately R=OE+DF=d1+d2;

[0060] Furthermore, to ensure the accuracy and success rate of image stitching, R ≥ 2r, u ≥ 0.2, ensuring full coverage monitoring information in the space capsule while minimizing image distortion.

[0061] Furthermore, the value range of β is

[0062] Since the field of view overlap rate u≥0.2, that is, when u≥0.2, the accuracy and success rate of image stitching can be guaranteed, u=0.2, R=2r, and substitute into the calculation formula R=OE+DF=d1+d2, we can get

[0063]

[0064] Since n×a=360°, n is the number of surround view cameras 1-1, and the selection factors of surround view cameras include two points: whether the content contained is sufficient and whether the distortion is too large, therefore, formula 12 is reflected as follows: If β is lower than If the field of view is too small, a large number of cameras are required. The distortion is too large and difficult to handle, so the field of view angle β is generally selected to be arrive between.

[0065] Furthermore, the number of surround cameras 1-1 In order for n to satisfy the above conditions, its value is generally 4, 5 or 6;

[0066] For further information, see Figure 3 , the vertical field of view of each front-to-back camera 1-2 is β q , and the offset of the front-rear view camera 1-2 relative to the intersection of the optical axes of all surround view cameras 1-1 is γ′, and

[0067] right The verification process is obtained as follows:

[0068] Depend on Figure 3 It can be seen that: MN = γ;

[0069]

[0070] Substituting KM, AN and MN into formula 13, we can obtain;

[0071]

[0072] Simplifying formula 14, we get:

[0073]

[0074] In order to make the front and rear view cameras 1-2 work properly, it is hoped that JK≤R, so Among them, γ′ is the design size, which is determined according to the shape of the structure, and β can be determined by γ′, β', and R q .

[0075] Furthermore, the image processor 4 is used to pre-process the multiple images and stitch the pre-processed images to obtain the panoramic information inside the space station cabin, including the following steps:

[0076] S1, the image processor 4 preprocesses each image, and the process of preprocessing each image is: performing image noise elimination, geometric correction and histogram equalization on each image in turn to complete the preprocessing;

[0077] S2, using an image stitching algorithm to stitch the preprocessed images to obtain an initial stitching image;

[0078] S3. Eliminate the black edges of the initial stitched image to obtain a panoramic image of the space station cabin.

[0079] In this preferred embodiment, the acquired image is post-processed to remove the black edges, which can avoid the influence of the black edges of the image on the observation of dark space station scenery; the attitude information and panoramic image information are combined, and there is no need to adjust the attitude of the monitoring system, which reduces fuel consumption and improves work efficiency.

[0080] Furthermore, in S2, the image stitching algorithm is used to stitch the preprocessed images to obtain the initial stitched image in the following manner:

[0081] S21, cropping both sides of each preprocessed image; when applied, cropping one-fifth of each side of the left and right images, retaining three-fifths of the middle part with small distortion to ensure subsequent stitching accuracy;

[0082] S22, performing image registration on the cropped multiple images and extracting feature points;

[0083] S23, measuring the similarity of feature points at corresponding positions of the cropped multiple images and matching the feature points;

[0084] S24, after the similarity measurement and feature point matching in step S23 are performed, coordinate transformation and image alignment are performed on the image, and then image fusion is performed to obtain an initial spliced ​​image.

[0085] Furthermore, in S3, the implementation method of eliminating the black edges of the initial spliced ​​image is as follows:

[0086] S31, copying the initial stitched image to obtain a copy stitched image, scanning each pixel of the copy stitched image, thereby obtaining a pixel value of each pixel in the copy stitched image;

[0087] S32, setting a threshold value according to the pixel value, taking the preset threshold value as the dividing point, performing black-and-white binarization processing on the image copy to obtain a binarized image;

[0088] S33, taking the edge areas around the binarized image as the pre-cropped area, and determining the black area appearing in the pre-cropped area, which is the black border;

[0089] S34, removing the corresponding position of the initial spliced ​​image according to the position of the black area, thereby completing the elimination of the black edge.

[0090] In this preferred embodiment, image distortion is reduced by stitching multiple images. Since the overlapping area of ​​the photos taken by adjacent cameras is relatively large, the images on the left and right edges of each photo can be removed during stitching, and only the middle part with less distortion is retained for stitching, which can reduce the distortion problem caused by the camera arrangement.

[0091] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. An optical monitoring system based on multi-eye vision for use in a space station, comprising a multi-eye vision unit (1), a structural base (2), a power module (3), and an image processor (4); characterized in that: A power module (3) and an image processor (4) are fixed in the internal cavity of the structural base (2); A power module (3) is used to supply power to the multi-eye vision unit (1) and the image processor (4); A multi-eye vision unit (1) is used to collect multiple images at different viewing angles and synchronously send the collected multiple images to an image processor (4); An image processor (4) is used to pre-process the multiple images and stitch the pre-processed images to obtain a panoramic image of the space station cabin; The multi-eye vision unit (1) comprises a plurality of surround view cameras (1-1) and two front and rear view cameras (1-2), wherein the two front and rear view cameras (1-2) are respectively arranged on the upper and lower bottom surfaces of a structural base (2), and the plurality of surround view cameras (1-1) are located in the same plane and are evenly arranged in the circumference of the structural base (2); The optical axes of all surround-view cameras (1-1) are in the same plane and intersect at one point, and the angle between the optical axes of any two adjacent surround-view cameras (1-1) is α; the horizontal field of view angle of each surround-view camera (1-1) is β, and the vertical field of view angle is β′; The lines connecting all surround-view cameras (1-1) form a circle with a center of O and a radius of r. The vertical distance between the center of the circle and the line connecting any two adjacent surround-view cameras (1-1) is The position of the first surround-view camera (1-1) of any two adjacent surround-view cameras (1-1) is recorded as point A, and the position of the second surround-view camera (1-1) is recorded as point B. The distance between the two adjacent surround-view cameras (1-1) is When the overlap of the fields of view of any two adjacent surround-view cameras (1-1) is u, the minimum working distance of each surround-view camera (1-1) is R=d1+d2. A circle is drawn with O as the center and R as the radius. The intersection points of the two field of view lines of the overlapped part of the field of view of any two adjacent surround-view cameras (1-1) are C and D respectively; the vertical distance from point D to the line connecting the two adjacent surround-view cameras (1-1) is The vertical field of view of each front-to-back camera (1-2) is β q , and the offset of the front-back camera (1-2) relative to the intersection of the optical axes of all surround cameras (1-1) is γ′, and 2. The optical monitoring system based on multi-eye vision applied to a space station according to claim 1, characterized in that: R≥2r,u≥0.

2.

3. The optical monitoring system based on multi-eye vision applied to a space station according to claim 1, characterized in that: The value range of β is 4. The optical monitoring system based on multi-eye vision applied to a space station according to claim 2, characterized in that: Number of surround view cameras (1-1) 5. The optical monitoring system based on multi-eye vision applied to a space station according to claim 1, characterized in that: The image processor (4) is used to pre-process the multiple images and stitch the pre-processed images. The implementation method of obtaining the panoramic information inside the space station cabin includes the following steps: S1, the image processor (4) pre-processes each image, and the process of pre-processing each image is: performing image noise elimination, geometric correction and histogram equalization on each image in turn to complete the pre-processing; S2, stitching the preprocessed images using an image stitching algorithm to obtain an initial stitched image; S3. Eliminate the black edges of the initial stitched image to obtain a panoramic image of the space station cabin.

6. The optical monitoring system based on multi-eye vision applied to a space station according to claim 5, characterized in that: In S2, the image stitching algorithm is used to stitch the preprocessed images to obtain the initial stitched image as follows: S21, cropping both sides of each preprocessed image; S22, performing image registration on the cropped multiple images and extracting feature points; S23, measuring the similarity of feature points at corresponding positions of the cropped multiple images and matching the feature points; S24, after the similarity measurement and feature point matching in step S23 are performed, coordinate transformation and image alignment are performed on the image, and then image fusion is performed to obtain an initial spliced ​​image.

7. The optical monitoring system based on multi-eye vision applied to a space station according to claim 5, characterized in that: In S3, the implementation method of eliminating the black edges of the initial spliced ​​image is: S31, copying the initial stitched image to obtain a copy stitched image, scanning each pixel of the copy stitched image, thereby obtaining a pixel value of each pixel in the copy stitched image; S32, setting a threshold value according to the pixel value, taking the preset threshold value as the dividing point, performing black-and-white binarization processing on the image copy to obtain a binarized image; S33, taking the edge areas around the binarized image as the pre-cropped area, and determining the black area appearing in the pre-cropped area, which is the black border; S34, removing the corresponding position of the initial spliced ​​image according to the position of the black area, thereby completing the elimination of the black edge.

Images