A method for autonomously detecting and identifying the terrain of a small celestial body surface sampling point

Abstract

The application discloses a kind of small celestial body surface sampling point terrain visual autonomous detection identification method of small celestial body surface exploration technical field, and is configured binocular stereo vision camera by mechanical arm system, to realize the visual imaging of sampling area, and the identification and measurement of sampling point terrain, first by pre-processing algorithm to filter out the noise in image;Then the image is rectified, to reduce the imaging error caused by camera optical system;Then after binocular epipolar rectification, the left eye image and right eye image are matched, and a dense disparity map is obtained;The three-dimensional coordinates of two-dimensional disparity map are calculated point by point in the left eye camera coordinate system, so that a three-dimensional point cloud map is obtained;The overall terrain situation is perceived using point cloud data;Finally, the ground is in the plane area Grid division is carried out, the area information in sliding window is calculated, and the terrain is judged, so that the operable position of sampling device on mechanical arm can be screened out.

Description

Technical Field

[0001] This invention belongs to the field of small celestial body surface detection technology, and in particular relates to a method for autonomous visual detection and recognition of topographic sampling points on the surface of small celestial bodies. Background Technology

[0002] By collecting rock samples from the surface of small celestial bodies and analyzing their composition, we can explore clues about the formation of the solar system and even the origin of life. Autonomous sampling and return from small celestial bodies using robotic arms is an important method. Due to limitations in energy and propulsion, the time a probe spends on a small celestial body is relatively short. Furthermore, the great distance between these bodies and Earth results in prolonged communication delays, making real-time tracking and control impossible. Therefore, the robotic arm must rely on fully autonomous on-orbit visual image processing to detect and identify the terrain at the sampling point during the sampling process, thus enabling the robotic arm to perform the sampling operation.

[0003] The detection and identification of non-cooperative targets is one of the difficult problems in the field of space vision. The surface rocks of small celestial bodies, which are the sampling objects, are even more difficult to process due to their weak texture and high similarity. The images of sample rocks have very small differences in pixel values, making it very difficult to extract features. Therefore, it is very challenging to identify and measure such unstructured, highly similar rock targets. The literature (Wang Yalin et al., Analysis and simulation verification of the surface terrain of asteroids with rubble pile structure, Journal of Deep Space Exploration, 2019, 6(5)) studied the surface terrain characteristics of asteroids with rubble pile structure characteristics and proposed a method to generate a simulation model of asteroid surface terrain. It experimentally simulated the power law distribution law of rocks that affect the terrain structure, but did not give a method for identifying and detecting the sampling points of the robotic arm. Invention patent CN111721302A discloses a method for identifying and perceiving the complex terrain features of irregular asteroid surfaces. Using optical images taken by deep space probes, it can detect and distinguish craters and rock features based on the geometric characteristics of asteroid surface terrain features. This method is mainly used for navigation and obstacle avoidance of deep space probes in orbit, but it is not suitable for detailed terrain identification based on sampling points of close-range imaging after landing.

[0004] This invention focuses on the selection of sampling points for robotic arms in small celestial body surface exploration missions, and proposes an autonomous detection and recognition method for sampling point terrain based on binocular stereo vision, which has broad application prospects in small celestial body surface exploration missions. Summary of the Invention

[0005] The technical problem solved by this invention is: to address the weak texture and high similarity characteristics of rocks on the surface of small celestial bodies that are the sampling objects, a method for autonomous detection and recognition of the terrain of sampling points on the surface of small celestial bodies based on binocular stereo vision is provided, which solves the problem of autonomous detection and recognition of the terrain of sampling points on the surface by a sampling robotic arm in a certain type of small celestial body surface exploration mission.

[0006] The objective of this invention is achieved through the following technical solutions:

[0007] The robotic arm system is equipped with binocular stereo vision cameras. The common field of view of the two cameras covers the sample collection area, enabling visual imaging of the sampling area and identification and measurement of the terrain at sampling points. This robotic arm system is already widely used and is existing technology, so it will not be elaborated upon here. The binocular cameras are equipped with active illumination sources to ensure good imaging results. The two cameras simultaneously image the sampling area. First, a preprocessing algorithm filters out noise in the image; then, distortion correction is performed to reduce imaging errors caused by the camera's optical system. Next, after binocular epipolar correction, the left and right images are matched to obtain a dense disparity map. The three-dimensional coordinates of each point in the two-dimensional disparity map within the coordinate system of the left camera are calculated, thus obtaining a three-dimensional point cloud map. The point cloud data is used to perceive the overall terrain. Finally, the planar area containing the ground is divided into grids, and the information within the sliding window is calculated and the terrain is assessed, thereby selecting the operable positions of the sampling device on the robotic arm.

[0008] A method for autonomous visual detection and recognition of sampling points on the surface of small celestial bodies mainly includes the following steps:

[0009] (1) Use a binocular stereo vision camera to image the sampling area; ensure that the common field of view of the left and right cameras can cover the sampling area;

[0010] (2) Preprocess the original images captured by the left and right cameras and reduce the impact of noise by using appropriate filtering algorithms;

[0011] (3) Perform distortion correction on the denoised image after filtering obtained in step (2) to reduce the error caused by the distortion of the camera optical system;

[0012] (4) Perform binocular epipolar correction on the distortion-corrected image obtained in step (3) to generate a left eye epipolar correction image and a right eye epipolar correction image.

[0013] (5) Search for matching points in the left image and calculate the disparity to obtain a dense disparity map;

[0014] (6) Calculate the three-dimensional coordinates of the two-dimensional disparity map obtained in step (5) in the coordinate system of the left eye camera point by point to obtain a three-dimensional point cloud map.

[0015] (7) Use point cloud data to perform ground detection in the visible area to determine the overall terrain situation in the sampling area;

[0016] (8) Divide the ground into grids within the plane area and calculate the regional information within the sliding window to determine its terrain.

[0017] Furthermore, the preprocessing in step (2) involves median filtering of the left and right eye images, with a filtering window size of m×m. The calculation method includes the following steps:

[0018] A1) The width and height of the image are W and H respectively. Expand the image boundary, and the width and height of the image become W+2×[m / 2] and H+2×[m / 2]. Set the pixels of the expanded image to 0.

[0019] A2) For a point (u) on the original image ori ,v ori Its grayscale value is I(u) ori ,v ori The median filter calculation formula is as follows:

[0020]

[0021] Where G(u) f ,v f ) represents the filtered pixel grayscale value, W is an m×m filtering template, and i and j represent the coordinates of the pixel points on the template W.

[0022] Furthermore, the distortion correction in step (3) includes the following calculation steps:

[0023] The homogeneous 2D coordinates of the k-th pixel in the image before distortion correction:

[0024] Two-dimensional physical homogeneous coordinates of the k-th pixel in the image before distortion correction:

[0025] Where matrix A is the intrinsic parameter matrix of the camera, A -1 Denotes the inverse of matrix A.

[0026] Lens distortion horizontal component values:

[0027]

[0028] Lens distortion vertical component value:

[0029]

[0030] in k1, k2, and k3 represent first, second, and third order radial distortions, respectively, while p1 and p2 represent first and second order tangential distortions, respectively.

[0031] The two-dimensional physical homogeneous coordinates of the k-th pixel after distortion correction are:

[0032]

[0033] Convert to 2D pixel homogeneous coordinates:

[0034]

[0035] Furthermore, in step (4), the correction formula for the external polarity correction is as follows:

[0036]

[0037] Among them, [u c v c 1] T Let [uv 1] be the homogeneous pixel coordinates of the spatial point in the image before epipolar correction. T Let M be the homogeneous pixel coordinates of a spatial point in the epipolar-corrected image, M be the camera intrinsic parameter matrix, R be the camera coordinate system rotation matrix, and M′ be the corrected camera intrinsic parameter matrix. rec The rotation matrix for the corrected camera coordinate system is λ≠0, which is a constant.

[0038] Furthermore, in step (5), the disparity is calculated by searching for matching points of the left eye image in the right eye image using the block matching method, which includes the following steps:

[0039] B1) Calculate the difference between two pixels, i.e., measure grayscale similarity under different parallaxes:

[0040] e(u,v,d)=|G L (u,v)-G R (ud,v)| ⑦

[0041] Where G(u,v) is the gray value of the pixel at coordinates (u,v) in the pixel coordinate system, and d is the disparity.

[0042] B2) Select a window surrounding the matching point as the similarity measurement region, with the corresponding pixel as the center point of the window. Within the selected window, sum the matching costs of the corresponding pixels, and the result is used as the matching similarity measurement value for that point.

[0043]

[0044] Where S is the similarity measurement region, which is generally an n×n rectangular region.

[0045] B3) Select the point corresponding to the minimum sum of matching costs within the search range as the final matching point.

[0046] B4) Perform pixel-by-pixel disparity calculation on the entire image to obtain a dense disparity map.

[0047] Furthermore, in step (6), for any point (u,v) in the two-dimensional disparity map obtained by stereo matching calculation, the formula for calculating its three-dimensional coordinates (X,Y,Z) in the coordinate system of the left eye camera is as follows:

[0048]

[0049] Among them, f x f y Let u0 and v0 be the equivalent focal length of the camera, u0 and v0 be the principal pixel coordinates, and B be the baseline length.

[0050] Furthermore, in step (7), the visible area ground detection includes the following steps:

[0051] C1) Randomly select K points to fit a plane. Let the equation of the plane be Z = AX + BY + C, then we have

[0052]

[0053] Among them, (X) l ,Y l Z l Let be the three-dimensional spatial coordinates of the l-th point. C2) Verify the plane fitting effect using the remaining points, calculating the distance from each point to the plane:

[0054]

[0055] Set a flatness threshold T, and count the number N points N that are less than the threshold T from the plane under a set of plane parameters. c To obtain N, repeat the process multiple times. c The maximum value is determined, and the corresponding plane is taken as the visible ground area.

[0056] Furthermore, in step (8), the process of judging the terrain by region includes the following steps:

[0057] S1) Determine the grid area by referring to the operating area of ​​the sampling device, and divide the point cloud set into p×q grid regions.

[0058] S2) Retrieve the distance D from a point within each grid region to the ground. i Calculate the roughness of the current window. Represent it using a histogram descriptor, where the histogram H ranges from 0 to the maximum distance D. max The distances are then divided into several equal parts and the distances are statistically analyzed.

[0059] S3) Set thresholds and judgment rules, and determine whether the terrain of the area is suitable for the task based on the roughness histogram.

[0060] S4) Set the sliding window and sliding step size, perform the roughness calculation for the sliding window as described above, and determine whether the sliding window can perform the task.

[0061] S5) Mark the grid area that meets the task requirements, record the terrain parameters inside the area, and form a list of operable area information.

[0062] Compared with the prior art, the beneficial effects of the present invention are:

[0063] I. This invention addresses the problem of terrain detection and recognition at sampling points on the surface of small celestial bodies by proposing an autonomous detection and recognition method based on binocular stereo vision, which effectively solves the problem of autonomous detection and recognition of rocks on the surface of small celestial bodies with weak texture and high similarity.

[0064] II. The terrain detection and identification method for sampling points on the surface of small celestial bodies provided by this invention can realize fully autonomous on-orbit data processing, effectively solving the problem of communication delays and the inability of ground systems to perform real-time processing in small celestial body exploration missions.

[0065] Third, the visual detection and recognition method used in this invention is simple, efficient and reliable, and is suitable for spatial environments with relatively scarce computing resources. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the configuration structure of the binocular stereo camera of the present invention;

[0067] Figure 2 This is a flowchart of the autonomous detection and recognition method for sampling point terrain based on binocular stereo vision according to the present invention;

[0068] Figure 3 This is a schematic diagram of the three-dimensional planar grid region division of the present invention. Detailed Implementation

[0069] To make the objectives and advantages of this invention clearer, the invention will be specifically described below with reference to embodiments. It should be understood that the following text is merely used to describe one or more specific embodiments of the invention and does not strictly limit the scope of protection specifically claimed by the invention.

[0070] Example

[0071] like Figure 1 As shown, a binocular stereo camera is positioned on the Earth observation surface of the small celestial body detector to image the sampled area, enabling the detection and identification of terrain within that area. The binocular stereo camera's position and orientation within the entire instrument are designed to ensure optimal observation range. It is equipped with an active light source to ensure good imaging of objects within its common field of view. Synchronous acquisition by the binocular stereo camera ensures that the left and right cameras image simultaneously, enabling stereoscopic vision calculations.

[0072] like Figure 2 As shown, the method for autonomous detection and recognition of terrain sampling points based on binocular stereo vision mainly includes the following steps:

[0073] (1) Images from the binocular camera were preprocessed to reduce noise. Considering the imaging characteristics of the sample rock, median filtering was applied to both the left and right eye images to smooth the data and preserve subtle, sharp details. The filtering window size was 3×3, and the specific calculation process is as follows:

[0074] A1) The width and height of the image are W and H respectively. Expand the image boundary, and the width and height of the image become W+2 and H+2 respectively. Set the pixels of the expanded image to 0.

[0075] A2) For a point (u) on the original image ori ,v ori Its grayscale value is I(u) ori ,v ori The formula for calculating the 3×3 median filter is as follows:

[0076]

[0077] Where G(u) f ,v f ) represents the filtered pixel grayscale value, and W is a 3×3 filtering template. Therefore, i and j are both integers in the interval [-1, 1].

[0078] (2) Distortion correction is performed on the filtered images of the left and right eyes. The calculation process is as follows (taking the image from one of the cameras as an example):

[0079] The homogeneous 2D coordinates of the k-th pixel in the image before distortion correction:

[0080] Two-dimensional physical homogeneous coordinates of the k-th pixel in the image before distortion correction:

[0081] Where matrix A is the intrinsic parameter matrix of the camera, A -1 This represents the inverse of matrix A.

[0082] Lens distortion horizontal component values:

[0083]

[0084] Lens distortion vertical component value:

[0085]

[0086] in k1, k2, and k3 are first, second, and third order radial distortions, respectively, while p1 and p2 are first and second order tangential distortions, respectively.

[0087] The two-dimensional physical homogeneous coordinates of the k-th pixel after distortion correction are:

[0088]

[0089] Convert to 2D pixel homogeneous coordinates:

[0090]

[0091] (3) Perform epipolar correction on the distortion-corrected image. The calculation process is as follows (taking the image from one of the cameras as an example):

[0092]

[0093] Among them, [u c v c 1] T Let [uv 1] be the homogeneous pixel coordinates of the spatial point in the image before epipolar correction. T Let M be the homogeneous pixel coordinates of a spatial point in the epipolar-corrected image, M be the camera intrinsic parameter matrix, R be the camera coordinate system rotation matrix, and M′ be the corrected camera intrinsic parameter matrix. rec The rotation matrix for the corrected camera coordinate system is λ≠0, which is a constant.

[0094] (4) Search and match corresponding points in the left and right eye images. The calculation process is as follows:

[0095] B1) First, calculate the difference between two pixels based on the pixel points:

[0096] e(u,v,d)=|G L (u,v)-G R (ud,v)| ⑦

[0097] Where G(u,v) is the gray value of the pixel with coordinates (u,v) in the pixel coordinate system.

[0098] B2) Select a window surrounding the matching point as the similarity measurement region, with the corresponding pixel as the center point of the window. Within the selected window, sum the matching costs of the corresponding pixels, and the result is used as the matching similarity measurement value for that point.

[0099]

[0100] Where S is the similarity measurement area, which is generally an n×n rectangular area. In this example, n = 21 is determined based on the resolution of the test image.

[0101] B3) Select the point corresponding to the minimum sum of matching costs within the search range as the final matching point;

[0102] B4) Perform pixel-by-pixel disparity calculation on the entire image to obtain a dense disparity map.

[0103] (5) For any point (u,v) in the 2D disparity map obtained from stereo matching, calculate its 3D coordinates (X,Y,Z) in the left eye camera coordinate system. The calculation formula is as follows:

[0104]

[0105] Among them, f x f y Let u0 and v0 be the equivalent focal length of the camera, u0 and v0 be the principal pixel coordinates, and B be the baseline length.

[0106] (6) Ground detection is performed on the visible area using point cloud data. The calculation process is as follows:

[0107] C1) Randomly select K points to fit a plane. Let the equation of the plane be Z = AX + BY + C, then we have

[0108]

[0109] Among them, (X) l ,Y l Z l ) represents the three-dimensional spatial coordinates of the l-th point.

[0110] C2) Verify the plane fitting effect using the remaining points by calculating the distance from each point to the plane:

[0111]

[0112] Set a flatness threshold T, and count the number N points N that are less than the threshold T from the plane under a set of plane parameters. c To obtain N, repeat the process multiple times. c The maximum value is determined, and the corresponding plane is taken as the visible ground area.

[0113] (7) Divide the area into grids within the plane containing the ground and determine its terrain. The calculation process is as follows:

[0114] S1) Divide the detected plane into regions and preset the sliding window, such as... Figure 3 As shown. The grid size can be determined based on the single-point operating area of ​​the sampling device. For example, if the single-point operating area of ​​the sampling device is 100mm×100mm and the common field of view of the stereo camera is 1.2m×1.1m, 12×11 grid areas can be set, with each grid size set to 100mm×100mm.

[0115] S2) Retrieve the distance D from a point within each grid region to the ground. i Calculate the roughness of the current window and represent it using a histogram descriptor, where the histogram H ranges from 0 to the maximum distance D. max The distances were divided into 15 equal parts and the distances were statistically analyzed.

[0116] S3) Set the histogram interval threshold and corresponding judgment rules, and determine whether a certain type of task can be performed in the region based on the roughness histogram.

[0117] S4) Set the sliding window and sliding step size. The reference mesh size can be set to 100mm × 100mm for the sliding window and 50mm for the sliding step size. Perform the roughness calculations described above for the sliding window and determine if the sliding window is suitable for the task.

[0118] S5) Mark the grid area that meets the task requirements, record the terrain parameters inside the area, and form a list of operable area information.

[0119] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "setting" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection, a direct connection, or an indirect connection through an intermediate medium. Those skilled in the art will understand the specific meaning of the above terms in this invention according to the specific circumstances. In the description of this specification, references to terms such as "an embodiment," "example," and "specific example" indicate that the specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples.

[0120] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.

Claims

1. A method for autonomous visual detection and recognition of topographic features at sampling points on the surface of small celestial bodies, comprising the following steps: (1) Use a binocular stereo vision camera to image the sampling area; ensure that the common field of view of the left and right cameras can cover the sampling area; (2) Preprocess the original images captured by the left and right cameras and reduce the influence of noise by using filtering algorithms; (3) Perform distortion correction on the filtered and denoised image obtained in step (2) to reduce the error caused by the distortion of the camera optical system; (4) Perform binocular epipolar correction on the distortion-corrected image obtained in step (3) to generate a left eye epipolar correction image and a right eye epipolar correction image; (5) Search for matching points in the left eye image and calculate the disparity to obtain a dense disparity map; (6) Calculate the three-dimensional coordinates of the two-dimensional disparity map obtained in step (5) in the coordinate system of the left eye camera point by point to obtain the three-dimensional point cloud map; (7) Use point cloud data to perform ground detection on the visible area to determine the overall terrain situation within the sampling area; (8) Divide the area into grids within the plane of the ground, calculate the area information within the sliding window, and thus determine its terrain; In step (8), the process of judging the terrain by region includes the following steps: S1) Determine the grid area based on the operating area of ​​the reference sampling device, and divide the point cloud set into... p × q Each grid area; S2) Retrieve the distance from points within each grid region to the ground. Calculate the roughness of the current window and represent it using a histogram descriptor, where the histogram H ranges from 0 to the maximum distance. Divide the sample into several equal parts and count the distances. S3) Set thresholds and judgment rules, and determine whether the terrain of the area is suitable for the task based on the roughness histogram; S4) Set the sliding window and sliding step size, perform the roughness calculation for the sliding window as described above, and determine whether the sliding window can perform the task. S5) Mark the grid area that meets the task requirements, record the terrain parameters inside the area, and form a list of operable area information.

2. The method for autonomous visual detection and recognition of terrain sampling points on the surface of small celestial bodies according to claim 1, characterized in that: The preprocessing in step (2) involves performing median filtering on the left and right eye images, with a filter window size of [size missing]. m × m The calculation method includes the following steps: A1) The width and height of the image are W and H respectively. Expanding the image boundary changes the image width and height to W + 2 × [ m / 2]、H+2×[ m / 2], the expanded image pixel value is set to 0; A2) For a point on the original image Its grayscale value The formula for median filtering is as follows: ① in Let w be the filtered pixel grayscale value. m × m The filter template, i , j This represents the coordinates of a pixel on template w.

3. The method for autonomous visual detection and recognition of topographic features at sampling points on the surface of small celestial bodies according to claim 2, characterized in that: The distortion correction in step (3) includes the following calculation steps: Image before distortion correction k Two-dimensional homogeneous coordinates of a pixel: ; Image before distortion correction k Two-dimensional physical homogeneous coordinates of each pixel: ; Where the matrix Let be the intrinsic parameter matrix of the camera. Representation matrix The reverse, Lens distortion horizontal component values: ② Lens distortion vertical component value: ③ in , The first-order radial distortion coefficient, It is the second-order radial distortion coefficient. The third-order radial distortion coefficient, The first-order tangential distortion coefficient, Let be the second-order tangential distortion coefficient, then the distortion-corrected first... k Two-dimensional physical homogeneous coordinates of each pixel: ④ Convert to 2D pixel homogeneous coordinates: ⑤。 4. The method for autonomous visual detection and recognition of terrain sampling points on the surface of small celestial bodies according to claim 3, characterized in that: In step (4), the correction formula for the external polarity correction is as follows: ⑥ in, Here are the homogeneous pixel coordinates of the spatial point in the image before epipolar correction. Here are the homogeneous pixel coordinates of the spatial point in the epipolar-corrected image. Let be the intrinsic parameter matrix of the camera. Let be the rotation matrix of the camera coordinate system. For the corrected camera intrinsic parameter matrix, The rotation matrix for the corrected camera coordinate system. It is a non-zero constant.

5. The method for autonomous visual detection and recognition of terrain sampling points on the surface of small celestial bodies according to claim 4, characterized in that: In step (5), the disparity is calculated by searching for matching points between the left and right images using the block matching method, which includes the following steps: B1) Calculate the difference between two pixels, i.e., measure grayscale similarity under different disparities: ⑦ in The coordinates in the pixel coordinate system are The pixel grayscale value, d For parallax; B2) Select a window surrounding the matching point as the similarity measurement region, where the corresponding pixel is the center point of the window. Within the selected window, the matching costs of the corresponding pixels are summed, and the result is used as the matching similarity measurement value for that point. ⑧ in S For the similarity measurement region, take n × n Rectangular area; B3) Select the point corresponding to the minimum sum of matching costs within the search range as the final matching point; B4) Perform pixel-by-pixel disparity calculation on the entire image to obtain a dense disparity map.

6. The method for autonomous visual detection and recognition of terrain sampling points on the surface of small celestial bodies according to claim 5, characterized in that: In step (6), for any point in the two-dimensional disparity map obtained by stereo matching calculation... Its three-dimensional coordinates in the left eye camera coordinate system The calculation formula is: ⑨ in, f x , f y This is the camera's equivalent focal length. u 0、 v 0 is the primary pixel coordinate. B This represents the baseline length.

7. The method for autonomous visual detection and recognition of terrain sampling points on the surface of small celestial bodies according to claim 6, characterized in that: In step (7), the visible area ground detection includes the following steps: C1) Random selection K Fit a plane to points, and let the equation of the plane be... Then there is ⑩ in, For the first l The three-dimensional spatial coordinates of each point; C2) Verify the plane fitting effect using the remaining points by calculating the distance from each point to the plane: Set flatness threshold T Statistics show that, under a set of planar parameters, the distance to the plane is less than a threshold. T Number of points N c Obtained through multiple iterations N c The maximum value is determined, and the corresponding plane is taken as the visible ground area.

Images