A small celestial body three-dimensional modeling method based on SFS

Abstract

This invention discloses a method for 3D modeling of small celestial bodies based on SFS (Self-Focused Spectroscopic Method), belonging to the field of intelligent 3D modeling and image reconstruction technology for small celestial bodies. To improve the accuracy of 3D modeling of small celestial bodies, this invention performs modeling under the condition that the camera's extrinsic parameters are determined. This includes setting the positions of the camera and the small celestial body in a simulation environment and setting the camera's motion trajectory to obtain multiple frames of small celestial body observation images; extracting the contours from the small celestial body observation images; transforming the 3D modeling problem of small celestial bodies into a pseudo-Boolean function minimization problem; applying local minimization and hierarchical minimization algorithms to solve for the minimum value of the pseudo-Boolean function to obtain a 3D voxel model of the small celestial body. For modeling under the condition that the camera's extrinsic parameters are uncertain, this includes converting the 3D voxel model of the small celestial body into a polygon model; selecting 3D model points; matching the 3D model points with 2D image points; and applying the LM algorithm to optimize the matching error between the 3D model points and 2D image points. This patented method is applied to the process of small celestial body detection.

Description

Technical Field

[0001] This invention belongs to the field of intelligent 3D modeling and image reconstruction technology for small celestial bodies, specifically involving a 3D modeling method for small celestial bodies based on SFS. Background Technology

[0002] Asteroid exploration is one of the most complex deep space exploration missions and a current hot topic in international space activities. Accurately obtaining a three-dimensional model of an asteroid is a prerequisite for navigation and guidance during the landing process and for a safe landing. It also serves as the basis for precisely determining the mass, volume, moment of inertia, principal axis of inertia, and gravitational field model of the asteroid. Therefore, obtaining a three-dimensional model of an asteroid is a crucial objective of all asteroid exploration missions.

[0003] Currently, widely used methods for 3D modeling of small celestial bodies include light curve inversion, Structure from Motion (SFM), and Shape from Photometric Stereo (SPC). Light curve inversion yields relatively coarse models that cannot capture detailed surface features of the small celestial body, making it suitable only for preliminary exploration when the detector is far from the body. Both SFM and SPC can produce high-precision 3D models, but SFM is based on surface features, while SPC is based on surface brightness. When the detector is tens of kilometers away, it is impossible to obtain clear surface features and brightness information; only the outline can be acquired, making these two methods unsuitable. However, obtaining a high-precision 3D model of the small celestial body at such distances is essential, as it provides a basis for subsequent SPC 3D modeling and SPC mapping planning.

[0004] In summary, during asteroid exploration, when the probe is tens of kilometers away from the asteroid, a three-dimensional modeling method based on the asteroid's outline information is needed. Summary of the Invention

[0005] The problem this invention aims to solve is to improve the accuracy of 3D modeling of small celestial bodies, and proposes a 3D modeling method for small celestial bodies based on SFS (Shape from Shading).

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for 3D modeling of small celestial bodies based on SFS includes the following steps:

[0008] S1. Determine the status of the camera's external parameters. If the camera's external parameters are determined, proceed to step S2. If the camera's external parameters are determined to be uncertain, proceed to step S5.

[0009] S2. In the simulation environment, acquire multiple frames of small celestial body observation images from different camera perspectives, and extract the contour information of the small celestial body observation images;

[0010] S3. Based on the contour information of the small celestial body observation image obtained in step S2, construct the small celestial body initial voxel model, and then construct the pseudo-Boolean function based on the small celestial body initial voxel model;

[0011] S4. For the pseudo-Boolean function constructed in step S3, apply the local minimization search algorithm in the hierarchical minimization search algorithm to solve for the minimum value of the pseudo-Boolean function, complete the three-dimensional modeling of the small celestial body based on SFS, and obtain the three-dimensional voxel model of the small celestial body with determined camera extrinsic parameters.

[0012] S5. Initialize the camera extrinsic parameters using the existing camera parameters, and obtain the initial three-dimensional voxel model of the small celestial body with initialized camera extrinsic parameters according to the method of steps S2-S4, and convert the initial three-dimensional voxel model of the small celestial body with initialized camera extrinsic parameters into a polygon model.

[0013] S6. Use the initialized camera extrinsic parameters to back-project the polygonal model obtained in step S5 to obtain the reconstructed image, and calculate the difference Loss1 between the reconstructed image and the observed image.

[0014] S7. Select the coordinates of the points in the 3D model;

[0015] S8. Match the coordinates of the three-dimensional model points selected in step S7 with the two-dimensional image points of the observed image, and then apply the LM algorithm to optimize the matching error between the coordinates of the three-dimensional model points and the two-dimensional image points to obtain the optimized camera extrinsic parameters.

[0016] S9. Using the optimized camera extrinsic parameters obtained in step S8, perform step S6 to obtain the difference Loss2 between the updated reconstructed image and the observed image. If Loss2 is less than Loss1, accept the optimized camera extrinsic parameters to obtain the final three-dimensional voxel model of the small celestial body. If Loss2 is greater than Loss1, discard the worst 10% of the three-dimensional and two-dimensional points in step S8 and continue with steps S5 to S9 until the final three-dimensional voxel model of the small celestial body is obtained.

[0017] Furthermore, the specific implementation method of step S3 includes the following steps:

[0018] S3.1. Construct an initial voxel model of the small celestial body based on the contour information of the small celestial body observation image obtained in step S2;

[0019] S3.2. Construct a pseudo-Boolean function;

[0020] S3.2.1. Set a set of voxels {x0, x2, ..., x} in the initial voxel model of the small celestial body. n-1 If a voxel is occupied, its label is 0; if it is empty, its label is 1.

[0021] S3.2.2. Let p i q represents a pixel in the observed image. i For in p i The location of the reconstructed image pixels, p i 0 represents p i In the observed image, it is the target, q i 0 represents q i The target is in the reconstructed image;

[0022] The pixel values ​​of the reconstructed image are calculated based on a set of voxels in a given small celestial body initialization voxel model as follows:

[0023] By the camera's optical center and p i The subset obtained by the intersection of the formed ray and voxel set is represented as Then reconstruct the corresponding q in the image i for:

[0024]

[0025] Where, x a Initialize the elements of a set of voxels in the voxel model for the small celestial body;

[0026] The difference between the observed image and the reconstructed image is represented by a pseudo-Boolean function, with the expression:

[0027] Error(p i ,q i )=|p i -q i |

[0028] Error(p,q) is the sum of the differences between all observed and reconstructed images, expressed as:

[0029] Error(p,q)=∑|p i -q i |

[0030] Furthermore, the specific implementation method of step S4 includes the following steps:

[0031] S4.1. Set the set of labeled voxels for the initial voxel model of the small celestial body: V = {v1, v2, ..., v...} j ,...v n}, where vj For the j-th voxel block, the set of labeled voxels for the initialized small celestial body voxel model is V. (0) Initialize the voxel side length to s0;

[0032] S4.2. Split each voxel block into 8 sub-voxel blocks. The voxel side length s of the i-th sub-voxel block after splitting is... i =s i-1 / 2, the label of each daughter voxel block is consistent with the voxel block label before the daughter voxel block splits;

[0033] S4.3. Based on the camera's external parameters, project the 8 sub-voxels of each voxel block onto the image plane. If the projection is not on the image plane, there is no back projection information. Set the voxel label of the sub-voxel block without back projection information to empty. If the projection is on the image plane, there is back projection information. Perform a local minimization search on the sub-voxel block with back projection information.

[0034] S4.4. Iterate through steps S4.2 and S4.3 until the desired voxel resolution of the small celestial body initialization voxel model is achieved. Divide the voxel blocks through the iterative process and determine the edges of the small celestial body initialization voxel model.

[0035] Furthermore, the specific implementation method for performing local minimization search in step S4.3 includes the following steps:

[0036] S4.3.1. Initialize V (0) ;

[0037] S4.3.2. Determine if the current voxel set is a local minimum. If not, iterate through each voxel block and determine if each voxel block satisfies the condition for a local minimum. If not, invert the voxel block label value. The specific method is as follows:

[0038] For the initial voxel model of a small celestial body, the set of labeled voxels V = {v1, v2, ... v} i-1 ,v i ,...v n}, has a neighborhood N labeled Y, and Y and V are identical except for the label of one element;

[0039] If for any Y∈N(V), V is a value that makes f attain a local minimum, then the expression is:

[0040]

[0041] in, Indicates in v i The first partial derivative of f at point v, where f represents a pseudo-Boolean function, v i The elements of the labeled voxel set representing the initial voxel model of a small celestial body;

[0042]

[0043] To obtain V, from any V... (0) Starting with the initial value, continuously invert the voxel block label values ​​until all voxels satisfy the expression that makes f achieve a local minimum;

[0044] S4.3.3. Iteratively execute step S4.3.2 until the entire set of voxels is a local minimum.

[0045] Furthermore, the expression for calculating the difference Loss1 between the reconstructed image and the observed image in step S6 is as follows:

[0046]

[0047] Where pi represents a pixel, and I represents the observed image. S This indicates a reconstructed image.

[0048] Furthermore, the specific implementation conditions for step S7, which selects the coordinates of points in the 3D model, are as follows:

[0049] The X coordinate of a point on the 3D model should be located on the surface of the 3D model.

[0050] The pixel coordinates x obtained after backprojecting the coordinates X of the 3D model point lie on the contour of the small celestial body region in the reconstructed image. The expression for obtaining the pixel coordinates x by backprojecting the coordinates X of the 3D model point is:

[0051] x = K[RT]X

[0052] Where K represents the camera intrinsic parameter matrix, R represents the camera rotation matrix, and T represents the camera translation matrix.

[0053] Furthermore, the specific implementation method of step S8 includes the following steps:

[0054] S8.1. Find the nearest pixel coordinate φ(X) on the outline of the small celestial body region in the observation image corresponding to the pixel coordinate x obtained by back projection of the 3D model point coordinate X. The distance between x and φ(X) is the matching error.

[0055] S8.2. The LM algorithm is applied to optimize the matching error between the coordinates of points in the 3D model and the points in the 2D image, resulting in optimized camera extrinsic parameters, expressed as:

[0056]

[0057] Where M(m) represents the camera parameter matrix, M(m)1 represents the first row of the camera parameter matrix, M(m)2 represents the second row of the camera parameter matrix, M(m)3 represents the third row of the camera parameter matrix, and φ(X) represents the third row of the camera parameter matrix. i )1 represents φ(X) i The coordinates of φ(X) in the x-direction i )2 represents φ(X) i The coordinates in the y-direction. The camera extrinsic parameter that minimizes the matching error is argmin, which is the parameter value that minimizes the function or expression.

[0058] The beneficial effects of this invention are:

[0059] This invention discloses a 3D modeling method for small celestial bodies based on SFS (Surface-Sensitive Functions). When the detector is far from the small celestial body (tens of kilometers) and cannot accurately obtain the surface texture features, a 3D model of the small celestial body can be established using only its contour information. By capturing multiple frames of images of the small celestial body, extracting its contour, and constructing pseudo-Boolean functions, the 3D modeling problem is transformed into a pseudo-Boolean function optimization problem. The minimum value of the pseudo-Boolean function is then obtained using a pseudo-Boolean function optimization algorithm, resulting in a 3D reconstructed model of the small celestial body. This method offers good real-time performance and high modeling accuracy. Attached Figure Description

[0060] Figure 1 This is a flowchart of a small celestial body 3D modeling method based on SFS as described in this invention;

[0061] Figure 2 This is a flowchart illustrating the process of determining the camera's external parameters according to the present invention.

[0062] Figure 3 This is a flowchart illustrating the workflow of the present invention when the camera's external parameters are uncertain.

[0063] Figure 4 These are images of small bodies observed by Bennu, as presented in this invention. (a) is an image of the small body taken at 0° rotation around its rotation axis, (b) is an image of the small body taken at 40° rotation around its rotation axis, (c) is an image of the small body taken at 80° rotation around its rotation axis, and (d) is an image of the small body taken at 120° rotation around its rotation axis.

[0064] Figure 5The images shown are binarized results of Bennu small celestial body observations in this invention. Among them, (a) is the binarized result of the small celestial body observation image obtained by rotating the small celestial body 0° around its rotation axis, (b) is the binarized result of the small celestial body observation image obtained by rotating the small celestial body 40° around its rotation axis, (c) is the binarized result of the small celestial body observation image obtained by rotating the small celestial body 80° around its rotation axis, and (d) is the binarized result of the small celestial body observation image obtained by rotating the small celestial body 120° around its rotation axis.

[0065] Figure 6 To initialize the voxel model;

[0066] Figure 7 The image shows a 3D reconstruction model of the Bennu small body with a voxel resolution of 4 meters. (a) is an image taken when the voxel model is rotated 0° around its rotation axis, (b) is an image taken when the voxel model is rotated 90° around its rotation axis, (c) is an image taken when the voxel model is rotated 180° around its rotation axis, and (d) is an image taken when the voxel model is rotated 270° around its rotation axis.

[0067] Figure 8 The image shows a 3D reconstruction model of the Bennu small body with a voxel resolution of 1 meter. (a) is an image taken when the voxel model is rotated 0° around its rotation axis, (b) is an image taken when the voxel model is rotated 90° around its rotation axis, (c) is an image taken when the voxel model is rotated 180° around its rotation axis, and (d) is an image taken when the voxel model is rotated 270° around its rotation axis.

[0068] Figure 9 The images show the three-dimensional reconstruction model of the Bennu small body with a voxel resolution of 0.5 meters. (a) is an image taken when the voxel model is rotated 0° around its rotation axis, (b) is an image taken when the voxel model is rotated 90° around its rotation axis, (c) is an image taken when the voxel model is rotated 180° around its rotation axis, and (d) is an image taken when the voxel model is rotated 270° around its rotation axis. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0070] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0071] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1 - Appendix Figure 9 Detailed explanation is as follows:

[0072] Example 1:

[0073] A method for 3D modeling of small celestial bodies based on SFS includes the following steps:

[0074] S1. Determine the status of the camera's external parameters. If the camera's external parameters are determined, proceed to step S2. If the camera's external parameters are determined to be uncertain, proceed to step S5.

[0075] S2. In the simulation environment, acquire multiple frames of small celestial body observation images from different camera perspectives, and extract the contour information of the small celestial body observation images;

[0076] S3. Based on the contour information of the small celestial body observation image obtained in step S2, construct the small celestial body initial voxel model, and then construct the pseudo-Boolean function based on the small celestial body initial voxel model;

[0077] Furthermore, the specific implementation method of step S3 includes the following steps:

[0078] S3.1. Construct an initial voxel model of the small celestial body based on the contour information of the small celestial body observation image obtained in step S2;

[0079] S3.2. Construct a pseudo-Boolean function;

[0080] S3.2.1. Set a set of voxels {x0, x2, ..., x} in the initial voxel model of the small celestial body. n-1 If a voxel is occupied, its label is 0; if it is empty, its label is 1.

[0081] S3.2.2. Let p i q represents a pixel in the observed image. i For in p i The location of the reconstructed image pixels, p i 0 represents p i In the observed image, it is the target, q i 0 represents q i The target is in the reconstructed image;

[0082] The pixel values ​​of the reconstructed image are calculated based on a set of voxels in a given small celestial body initialization voxel model as follows:

[0083] By the camera's optical center and p i The subset obtained by the intersection of the formed ray and voxel set is represented as Then reconstruct the corresponding q in the image i for:

[0084]

[0085] Where, x a Initialize a set of voxels {x0, x2, ..., x} in the voxel model for a small celestial body. n-1}element;

[0086] The difference between the observed image and the reconstructed image is represented by a pseudo-Boolean function, with the expression:

[0087] Error(p i ,q i )=|p i -q i |

[0088] Error(p,q) is the sum of the differences between all observed and reconstructed images, expressed as:

[0089] Error(p,q)=∑|p i -q i |

[0090] S4. For the pseudo-Boolean function constructed in step S3, apply the local minimization search algorithm in the hierarchical minimization search algorithm to solve for the minimum value of the pseudo-Boolean function, complete the three-dimensional modeling of the small celestial body based on SFS, and obtain the three-dimensional voxel model of the small celestial body with determined camera extrinsic parameters.

[0091] Furthermore, the specific implementation method of step S4 includes the following steps:

[0092] S4.1. Set the set of labeled voxels for the initial voxel model of the small celestial body: V = {v1, v2, ..., v...} j ,...v n}, where v j For the j-th voxel block, the set of labeled voxels for the initialized small celestial body voxel model is V. (0) Initialize the voxel side length to s0;

[0093] S4.2. Split each voxel block into 8 sub-voxel blocks. The voxel side length s of the i-th sub-voxel block after splitting is... i=s i-1 / 2, the label of each daughter voxel block is consistent with the voxel block label before the daughter voxel block splits;

[0094] S4.3. Based on the camera's external parameters, project the 8 sub-voxels of each voxel block onto the image plane. If the projection is not on the image plane, there is no back projection information. Set the voxel label of the sub-voxel block without back projection information to empty. If the projection is on the image plane, there is back projection information. Perform a local minimization search on the sub-voxel block with back projection information.

[0095] Furthermore, the specific implementation method for performing local minimization search in step S4.3 includes the following steps:

[0096] S4.3.1. Initialize V (0) ;

[0097] S4.3.2. Determine if the current voxel set is a local minimum. If not, iterate through each voxel block and determine if each voxel block satisfies the condition for a local minimum. If not, invert the voxel block label value. The specific method is as follows:

[0098] For the initial voxel model of a small celestial body, the set of labeled voxels V = {v1, v2, ... v} i-1 ,v i ,...v n}, has a neighborhood N labeled Y, and Y and V are identical except for the label of one element;

[0099] If for any Y∈N(V), V is a value that makes f attain a local minimum, then the expression is:

[0100]

[0101] in, Indicates in v i The first partial derivative of f at point v, where f represents a pseudo-Boolean function, v i The elements of the labeled voxel set V represent the initial voxel model of the small celestial body;

[0102]

[0103] To obtain V, from any V... (0) Starting with the initial value, continuously invert the voxel block label values ​​until all voxels satisfy the expression that makes f achieve a local minimum;

[0104] S4.3.3. Iteratively execute step S4.3.2 until the entire set of voxels is a local minimum.

[0105] S4.4. Iterate through steps S4.2 and S4.3 until the desired voxel resolution of the small celestial body initial voxel model is achieved. Divide the voxel blocks through the iterative process and determine the edges of the small celestial body initial voxel model.

[0106] S5. Initialize the camera extrinsic parameters using the existing camera parameters, and obtain the initial three-dimensional voxel model of the small celestial body with initialized camera extrinsic parameters according to the method of steps S2-S4, and convert the initial three-dimensional voxel model of the small celestial body with initialized camera extrinsic parameters into a polygon model.

[0107] S6. Use the initialized camera extrinsic parameters to back-project the polygonal model obtained in step S5 to obtain the reconstructed image, and calculate the difference Loss1 between the reconstructed image and the observed image.

[0108] Furthermore, the expression for calculating the difference Loss1 between the reconstructed image and the observed image in step S6 is as follows:

[0109]

[0110] Where pi represents a pixel, and I represents the observed image. S This indicates a reconstructed image.

[0111] S7. Select the coordinates of the points in the 3D model;

[0112] Furthermore, the specific implementation conditions for step S7, which selects the coordinates of points in the 3D model, are as follows:

[0113] The X coordinate of a point on the 3D model should be located on the surface of the 3D model.

[0114] The pixel coordinates x obtained after backprojecting the coordinates X of the 3D model point lie on the contour of the small celestial body region in the reconstructed image. The expression for obtaining the pixel coordinates x by backprojecting the coordinates X of the 3D model point is:

[0115] x = K[RT]X

[0116] Where K represents the camera intrinsic parameter matrix, R represents the camera rotation matrix, and T represents the camera translation matrix.

[0117] S8. Match the coordinates of the three-dimensional model points selected in step S7 with the two-dimensional image points of the observed image, and then apply the LM algorithm to optimize the matching error between the coordinates of the three-dimensional model points and the two-dimensional image points to obtain the optimized camera extrinsic parameters.

[0118] Furthermore, the specific implementation method of step S8 includes the following steps:

[0119] S8.1. Find the nearest pixel coordinate φ(X) on the outline of the small celestial body region in the observation image corresponding to the pixel coordinate x obtained by back projection of the 3D model point coordinate X. The distance between x and φ(X) is the matching error.

[0120] S8.2. The LM algorithm is applied to optimize the matching error between the coordinates of points in the 3D model and the points in the 2D image, resulting in optimized camera extrinsic parameters, expressed as:

[0121]

[0122] Where M(m) represents the camera parameter matrix, M(m)1 represents the first row of the camera parameter matrix, M(m)2 represents the second row of the camera parameter matrix, M(m)3 represents the third row of the camera parameter matrix, and φ(X) represents the third row of the camera parameter matrix. i )1 represents φ(X) i The coordinates of φ(X) in the x-direction i )2 represents φ(X) i The coordinates in the y-direction. The camera extrinsic parameter that minimizes the matching error is argmin, which is the parameter value that minimizes the function or expression.

[0123] S9. Using the optimized camera extrinsic parameters obtained in step S8, perform step S6 to obtain the difference Loss2 between the updated reconstructed image and the observed image. If Loss2 is less than Loss1, accept the optimized camera extrinsic parameters to obtain the final three-dimensional voxel model of the small celestial body. If Loss2 is greater than Loss1, discard the worst 10% of the three-dimensional and two-dimensional points in step S8 and continue with steps S5 to S9 until the final three-dimensional voxel model of the small celestial body is obtained.

[0124] Example 2:

[0125] A method for 3D modeling of small celestial bodies based on SFS includes the following steps:

[0126] S1. Determine the status of the camera's external parameters. If the camera's external parameters are determined, proceed to step S2. If the camera's external parameters are determined to be uncertain, proceed to step S5.

[0127] S2. In the simulation environment, acquire multiple frames of small celestial body observation images from different camera perspectives, and extract the contour information of the small celestial body observation images;

[0128] S3. Based on the contour information of the small celestial body observation image obtained in step S2, construct the small celestial body initial voxel model, and then construct the pseudo-Boolean function based on the small celestial body initial voxel model;

[0129] Furthermore, the specific implementation method of step S3 includes the following steps:

[0130] S3.1. Construct an initial voxel model of the small celestial body based on the contour information of the small celestial body observation image obtained in step S2;

[0131] S3.2. Construct a pseudo-Boolean function;

[0132] S3.2.1. Set a set of voxels {x0, x2, ..., x} in the initial voxel model of the small celestial body. n-1 If a voxel is occupied, its label is 0; if it is empty, its label is 1.

[0133] S3.2.2. Let p i q represents a pixel in the observed image. i For in p i The location of the reconstructed image pixels, p i 0 represents p i In the observed image, it is the target, q i 0 represents q i The target is in the reconstructed image;

[0134] The pixel values ​​of the reconstructed image are calculated based on a set of voxels in a given small celestial body initialization voxel model as follows:

[0135] By the camera's optical center and p i The subset obtained by the intersection of the formed ray and voxel set is represented as Then reconstruct the corresponding q in the image i for:

[0136]

[0137] Where, x a Initialize the elements of a set of voxels in the voxel model for the small celestial body;

[0138] The difference between the observed image and the reconstructed image is represented by a pseudo-Boolean function, with the expression:

[0139] Error(p i ,q i )=|p i -q i |

[0140] Error(p,q) is the sum of the differences between all observed and reconstructed images, expressed as:

[0141] Error(p,q)=∑|p i -q i |

[0142] S4. For the pseudo-Boolean function constructed in step S3, apply the local minimization search algorithm in the hierarchical minimization search algorithm to solve for the minimum value of the pseudo-Boolean function, complete the three-dimensional modeling of the small celestial body based on SFS, and obtain the three-dimensional voxel model of the small celestial body with determined camera extrinsic parameters.

[0143] Furthermore, the specific implementation method of step S4 includes the following steps:

[0144] S4.1. Set the set of labeled voxels for the initial voxel model of the small celestial body: V = {v1, v2, ..., v...} j ,...v n}, where v j For the j-th voxel block, the set of labeled voxels for the initialized small celestial body voxel model is V. (0) Initialize the voxel side length to s0;

[0145] S4.2. Split each voxel block into 8 sub-voxel blocks. The voxel side length s of the i-th sub-voxel block after splitting is... i =s i-1 / 2, the label of each daughter voxel block is consistent with the voxel block label before the daughter voxel block splits;

[0146] S4.3. Based on the camera's external parameters, project the 8 sub-voxels of each voxel block onto the image plane. If the projection is not on the image plane, there is no back projection information. Set the voxel label of the sub-voxel block without back projection information to empty. If the projection is on the image plane, there is back projection information. Perform a local minimization search on the sub-voxel block with back projection information.

[0147] Furthermore, the specific implementation method for performing local minimization search in step S4.3 includes the following steps:

[0148] S4.3.1. Initialize V (0) ;

[0149] S4.3.2. Determine if the current voxel set is a local minimum. If not, iterate through each voxel block and determine if each voxel block satisfies the condition for a local minimum. If not, invert the voxel block label value. The specific method is as follows:

[0150] For the initial voxel model of a small celestial body, the set of labeled voxels V = {v1, v2, ... v} i-1 ,v i ,...v n}, has a neighborhood N labeled Y, and Y and V are identical except for the label of one element;

[0151] If for any Y∈N(V), V is a value that makes f attain a local minimum, then the expression is:

[0152]

[0153] in, Indicates in v i The first partial derivative of f at point v, where f represents a pseudo-Boolean function, v i The elements of the labeled voxel set V represent the initial voxel model of the small celestial body;

[0154]

[0155] To obtain V, from any V... (0) Starting with the initial value, continuously invert the voxel block label values ​​until all voxels satisfy the expression that makes f achieve a local minimum.

[0156] S4.3.3. Iteratively execute step S4.3.2 until the entire set of voxels is a local minimum.

[0157] S4.4. Iterate through steps S4.2 and S4.3 until the desired voxel resolution of the small celestial body initialization voxel model is achieved. Divide the voxel blocks through the iterative process and determine the edges of the small celestial body initialization voxel model.

[0158] This embodiment provides a detailed description of the process of performing three-dimensional modeling of the Bennu asteroid in a simulation environment according to the present invention.

[0159] When the camera's extrinsic parameters are determined, the 3D modeling process for the Bennu small body is as follows:

[0160] (I) Image Acquisition and Contour Extraction from Small Celestial Bodies

[0161] On the CINEMA4D simulation platform, the small celestial body was positioned at the origin of the world coordinate system. A detector was positioned in the XOZ plane of the world coordinate system, rotating around the Y-axis to orbit and photograph the Bennu celestial body, taking one image every 10 degrees of rotation. The detector was set 20 kilometers away from the small celestial body, with an image resolution of 1024*1024, and 36 images of the small celestial body were captured. Some image data is shown below. Figure 4 As shown.

[0162] A threshold-based image binarization method was employed. Since the small celestial body region and the background region in the acquired observation image were clearly distinguishable, the threshold was set to 0. Some image binarization results are shown below. Figure 5 As shown, by comparing the observed image and the binarized image, it can be seen that using the threshold-based binarization method for segmenting the observed image of the small celestial body model can achieve better segmentation results.

[0163] (II) Model Initialization

[0164] Initialize a cube with a side length of 140 meters. Set the lengths of the X, Y, and Z axes of the Bennu small body model to approximately 112 meters, 107 meters, and 99 meters, respectively. This initialization model can completely enclose the Bennu small body model. The initialization model is as follows: Figure 6 As shown.

[0165] (III) Construction and Optimization of Pseudo-Boolean Functions

[0166] Based on steps S3 and S4, a pseudo-Boolean function can be constructed and optimized by searching for its local minimum. The voxel model is initialized with a resolution of 8 meters. After four splits, a small celestial body 3D model with a voxel resolution of 0.5 meters is finally obtained, as shown below. Figures 7-9 As shown.

[0167] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0168] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for three-dimensional modeling of small celestial bodies based on SFS, characterized in that, Includes the following steps: S1. Determine the status of the camera's external parameters. If the camera's external parameters are determined, proceed to step S2. If the camera's external parameters are determined to be uncertain, proceed to step S5. S2. In the simulation environment, acquire multiple frames of small celestial body observation images from different camera perspectives, and extract the contour information of the small celestial body observation images; S3. Based on the contour information of the small celestial body observation image obtained in step S2, construct the initial voxel model of the small celestial body, and then construct the pseudo-Boolean function based on the initial voxel model of the small celestial body; S4. For the pseudo-Boolean function constructed in step S3, apply the local minimization search algorithm in the hierarchical minimization search algorithm to solve for the minimum value of the pseudo-Boolean function, complete the three-dimensional modeling of the small celestial body based on SFS, and obtain the three-dimensional voxel model of the small celestial body with determined camera extrinsic parameters. S5. Initialize the camera extrinsic parameters using the existing camera parameters, and obtain the initial three-dimensional voxel model of the small celestial body with initialized camera extrinsic parameters according to the method of steps S2-S4, and convert the initial three-dimensional voxel model of the small celestial body with initialized camera extrinsic parameters into a polygon model. S6. Use the initialized camera extrinsic parameters to back-project the polygonal model obtained in step S5 to obtain the reconstructed image, and calculate the difference Loss1 between the reconstructed image and the observed image. S7. Select the coordinates of the points in the 3D model; S8. Match the coordinates of the three-dimensional model points selected in step S7 with the two-dimensional image points of the observed image, and then apply the LM algorithm to optimize the matching error between the coordinates of the three-dimensional model points and the two-dimensional image points to obtain the optimized camera extrinsic parameters. S9. Using the optimized camera extrinsic parameters obtained in step S8, perform step S6 to obtain the difference Loss2 between the updated reconstructed image and the observed image. If Loss2 is less than Loss1, accept the optimized camera extrinsic parameters to obtain the final three-dimensional voxel model of the small celestial body. If Loss2 is greater than Loss1, discard the worst 10% of the three-dimensional and two-dimensional points in step S8 and continue with steps S5 to S9 until the final three-dimensional voxel model of the small celestial body is obtained.

2. The method for three-dimensional modeling of small celestial bodies based on SFS according to claim 1, characterized in that, The specific implementation method of step S3 includes the following steps: S3.

1. Construct an initial voxel model of the small celestial body based on the contour information of the small celestial body observation image obtained in step S2; S3.

2. Construct a pseudo-Boolean function; S3.2.

1. Set a set of voxels {x0, x2, ..., x} in the initial voxel model of the small celestial body. n-1 If a voxel is occupied, its label is 0; if it is empty, its label is 1. S3.2.

2. Let p i q represents a pixel in the observed image. i For in p i The location of the reconstructed image pixels, p i 0 represents p i In the observed image, it is the target, q i 0 represents q i The target is in the reconstructed image; The pixel values ​​of the reconstructed image are calculated based on a set of voxels in a given small celestial body initialization voxel model as follows: By the camera's optical center and p i The subset obtained by the intersection of the formed ray and the voxel set is denoted as T. pi Then the corresponding q in the reconstructed image i for: Where, x a Initialize the elements of a set of voxels in the voxel model for the small celestial body; The difference between the observed image and the reconstructed image is represented by a pseudo-Boolean function, with the expression: Error(p i ,q i )=|p i -q i | Error(p,q) is the sum of the differences between all observed and reconstructed images, expressed as: Error(p,q)=∑|p i -q i |。 3. The method for three-dimensional modeling of small celestial bodies based on SFS according to claim 2, characterized in that, The specific implementation method of step S4 includes the following steps: S4.

1. Set the set of labeled voxels for the initial voxel model of the small celestial body: V = {v1, v2, ..., v...} j ,...v n }, where v j For the j-th voxel block, the set of labeled voxels for the initialized small celestial body voxel model is V. (0) Initialize the voxel side length to s0; S4.

2. Split each voxel block into 8 sub-voxel blocks. The voxel side length s of the i-th sub-voxel block after splitting is... i =s i-1 / 2, the label of each daughter voxel block is consistent with the voxel block label before the daughter voxel block splits; S4.

3. Based on the camera's external parameters, project the 8 sub-voxels of each voxel block onto the image plane. If the projection is not on the image plane, there is no back projection information. Set the voxel label of the sub-voxel block without back projection information to empty. If the projection is on the image plane, there is back projection information. Perform a local minimization search on the sub-voxel block with back projection information. S4.

4. Iterate through steps S4.2 and S4.3 until the desired voxel resolution of the small celestial body initialization voxel model is achieved. Divide the voxel blocks through the iterative process and determine the edges of the small celestial body initialization voxel model.

4. The method for three-dimensional modeling of small celestial bodies based on SFS according to claim 3, characterized in that, The specific implementation method for performing local minimization search in step S4.3 includes the following steps: S4.3.

1. Initialize V (0) ; S4.3.

2. Determine if the current voxel set is a local minimum. If not, iterate through each voxel block and determine if each voxel block satisfies the condition for a local minimum. If not, invert the voxel block label value. The specific method is as follows: For the initial voxel model of a small celestial body, the set of labeled voxels V = {v1, v2, ... v} i-1 ,v i ,...v n }, has a neighborhood N labeled Y, and Y and V are identical except for the label of one element; If for any Y∈N(V), V is a value that makes f attain a local minimum, then the expression is: in, Indicates in v i The first partial derivative of f at point v, where f represents a pseudo-Boolean function, v i The elements of the labeled voxel set V represent the initial voxel model of the small celestial body; To obtain V, from any V... (0) Starting with the initial value, continuously invert the voxel block label values ​​until all voxels satisfy the expression that makes f achieve a local minimum; S4.3.

3. Iteratively execute step S4.3.2 until the entire set of voxels is a local minimum.

5. The method for three-dimensional modeling of small celestial bodies based on SFS according to claim 4, characterized in that, The expression for Loss1, the difference between the reconstructed image and the observed image, in step S6 is: Where pi represents a pixel, and I represents the observed image. S This indicates a reconstructed image.

6. The method for three-dimensional modeling of small celestial bodies based on SFS according to claim 5, characterized in that, The specific implementation conditions for step S7, selecting the coordinates of points in the 3D model, are as follows: The X coordinate of a point on the 3D model should be located on the surface of the 3D model. The pixel coordinates x obtained after backprojecting the coordinates X of the 3D model point lie on the contour of the small celestial body region in the reconstructed image. The expression for obtaining the pixel coordinates x by backprojecting the coordinates X of the 3D model point is: x = K[RT]X Where K represents the camera intrinsic parameter matrix, R represents the camera rotation matrix, and T represents the camera translation matrix.

7. The method for three-dimensional modeling of small celestial bodies based on SFS according to claim 6, characterized in that, The specific implementation method of step S8 includes the following steps: S8.

1. Find the nearest pixel coordinate φ(X) on the outline of the small celestial body region in the observation image corresponding to the pixel coordinate x obtained by back projection of the 3D model point coordinate X. The distance between x and φ(X) is the matching error. S8.

2. The LM algorithm is applied to optimize the matching error between the coordinates of points in the 3D model and the points in the 2D image, resulting in optimized camera extrinsic parameters, expressed as: Where M(m) represents the camera parameter matrix, M(m)1 represents the first row of the camera parameter matrix, M(m)2 represents the second row of the camera parameter matrix, M(m)3 represents the third row of the camera parameter matrix, and φ(X) represents the third row of the camera parameter matrix. i )1 represents φ(X) i The coordinates of φ(X) in the x-direction. i )2 represents φ(X) i The coordinates in the y-direction. The camera extrinsic parameter that minimizes the matching error is argmin, which is the parameter value that minimizes the function or expression.

Images