A data dynamic acquisition method for high-precision 3D reconstruction

Through the dynamic data acquisition method for high-precision three-dimensional reconstruction, the shooting mode is optimized to improve the image data quality, and the problem of difficulty in accurately reconstructing the three-dimensional model under interference in the prior art is solved, achieving efficient and accurate three-dimensional reconstruction effect.

CN119963746BActive Publication Date: 2025-06-13NORTHWESTERN POLYTECHNICAL UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510436381.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-04-09
Publication Date
2025-06-13
Estimated Expiration
2045-04-09

AI Technical Summary

Technical Problem

The prior art is difficult to accurately reconstruct a three-dimensional model under many interference situations, especially when the image data viewing angle is insufficient, the overlap between images is insufficient, or the exposure is extreme, and a complete and accurate three-dimensional model cannot be obtained.

Method used

A dynamic data acquisition method for high-precision three-dimensional reconstruction is adopted. By selecting the initial shooting mode from the preset shooting mode pool of the imaging system, the image sequence is acquired and the three-dimensional reconstruction is carried out. The reconstruction point cloud model is evaluated based on the real point cloud model, and the shooting mode is optimized using the prediction network to achieve high-quality dynamic acquisition of image data.

Benefits of technology

It realizes that the three-dimensional structure of the scene can be accurately reconstructed under many interference situations, improves the quality of image data and dynamic acquisition efficiency, and ensures the integrity and accuracy of the three-dimensional reconstruction results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119963746B_ABST
    Figure CN119963746B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for dynamically acquiring data for high-precision three-dimensional reconstruction, including: selecting an initial shooting mode from a preset shooting mode pool of an imaging system, acquiring an image sequence for a target in a simulated environment in the initial shooting mode and performing three-dimensional reconstruction to obtain a reconstructed point cloud model; performing quality evaluation on the reconstructed point cloud model of the target in the simulated environment based on the true point cloud model of the target to obtain a quality evaluation score; inputting the reconstructed point cloud model of the target in the simulated environment, the quality evaluation score, and environmental prior information into a prediction network, constructing a policy gradient loss function with the quality evaluation score as a reward to perform iterative optimization of the prediction network, and determining a final optimal shooting mode in the shooting mode pool for the imaging system to acquire an image sequence of the target and perform three-dimensional reconstruction in an actual environment. The present invention can still accurately reconstruct the three-dimensional structure of a scene under various interferences and achieve high-quality dynamic acquisition of image data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and particularly to a method for dynamically acquiring data for high-precision three-dimensional reconstruction. Background Art

[0002] With the rapid development of aerospace technology, the number and types of space debris (such as satellites, space stations, etc.) are increasing day by day. Detecting, identifying, and tracking space debris have become particularly important. Three-dimensional reconstruction technology can recover the three-dimensional geometric structure of the target from two-dimensional optical image data, providing a more intuitive and comprehensive perspective for understanding and analyzing space debris, and playing an important role in space situation awareness, spacecraft design and manufacturing, deep space exploration, etc.

[0003] Three-dimensional reconstruction is the process of estimating the camera pose and reconstructing the three-dimensional structure based on a series of optical images taken from different perspectives. The quality of the three-dimensional reconstruction result depends on the quality of the input images, and the quality of the input images depends on the shooting mode, that is, the shooting parameters during shooting. For space debris targets, the controllable parameters mainly include turntable angle, exposure time, exposure times, etc. However, the current technology cannot obtain good three-dimensional results for image data under any conditions. When the perspective coverage of the image data is insufficient, the overlap between images is not enough, or the exposure is extreme, a complete and accurate three-dimensional model usually cannot be obtained. Although in practical applications, the optimal shooting mode can be obtained by testing all combinations of simulation data, this not only occupies a large amount of storage resources but is also very inefficient, and the shooting mode set by experience is unreliable. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for dynamically acquiring data for high-precision three-dimensional reconstruction, so as to accurately reconstruct the three-dimensional structure of the scene under various interferences and achieve high-quality dynamic acquisition of image data.

[0005] To achieve the above task, the present invention adopts the following technical solutions:

[0006] A method for dynamically acquiring data for high-precision three-dimensional reconstruction, comprising:

[0007] Select an initial shooting mode from the preset shooting mode pool of the imaging system, and acquire an image sequence for the target in the simulation environment in the initial shooting mode and perform three-dimensional reconstruction to obtain a reconstructed point cloud model;

[0008] Based on the real point cloud model of the target, evaluate the quality of the reconstructed point cloud model of the target in the simulation environment to obtain a quality evaluation score;

[0009] Input the reconstructed point cloud model of the target, the quality assessment score, and the environmental prior information in the simulation environment into the prediction network. Construct a policy gradient loss function with the quality assessment score as the reward to iteratively optimize the prediction network, and determine the final optimal shooting mode in the shooting mode pool for the imaging system to acquire the image sequence of the target and perform 3D reconstruction in the actual environment.

[0010] Further, a variety of shooting modes are stored in the shooting mode pool, and each shooting mode is used for the imaging system to shoot the target to form a corresponding image sequence; the shooting mode is represented as a triple , where represents the i-th shooting mode, represents the angle by which the turntable carrying the imaging system rotates each time in the i-th shooting mode, respectively represent the exposure time and the number of exposures; in each shooting mode, the turntable starts from the initial position and rotates by each time within its rotation range and shoots the target according to to form the corresponding image sequence in this shooting mode.

[0011] Further, the process of performing 3D reconstruction is as follows:

[0012] (1) Set up a simulation environment to simulate the real state of the target; construct the models of the target and the imaging system in the simulation environment and simulate the motion trajectory of the target; the imaging system images the target according to the selected initial shooting mode to obtain the image sequence; during the shooting process of the imaging system, obtain the environmental prior information of the target during shooting.

[0013] (2) For each image in the image sequence, extract the feature points of the image.

[0014] (3) Brutally match the images in the image sequence pairwise to obtain the corresponding relationship of the feature points between the two images, and form a scene graph with the images as nodes and the corresponding relationship of the feature points as edges.

[0015] (4) Select a pair of images with the most matching feature points for initial reconstruction, solve the pose of the camera in the imaging system, perform triangulation on the matching feature points to obtain the corresponding 3D points, and finally use bundle adjustment to optimize the pose of the camera and generate a 3D point cloud model.

[0016] (5) Add other images in the image sequence for image registration; during image registration, the camera pose corresponding to the newly added image is solved using the PnP algorithm, and the feature points in the newly added image are triangulated to obtain 3D points and added to the 3D point cloud model to continuously improve the 3D point cloud model. At the same time, local bundle adjustment and outlier filtering are performed; repeat this step until all other images in the image sequence are completed for image registration, and the reconstructed point cloud model of the target is obtained, denoted as the reconstructed point cloud model.

[0017] Further, the environmental prior information includes the corresponding solar illumination angle, the distance between the target and the imaging system, the movement speed of the target, and the angular velocity.

[0018] Further, based on the true point cloud model of the target, the quality of the reconstructed point cloud model of the target in the simulated environment is evaluated to obtain a quality evaluation score, including:

[0019] Construct a true point cloud model of the target; calculate the distance from each 3D point in the true point cloud model to the reconstructed point cloud model, and the distance from each 3D point in the reconstructed point cloud model to the true point cloud model.

[0020] Calculate the quality evaluation score, including accuracy and completeness; where accuracy is defined as the average distance from each 3D point in the reconstructed point cloud model to the corresponding 3D point in the true point cloud model, and completeness is defined as the average distance from each 3D point in the true point cloud model to the corresponding 3D point in the reconstructed point cloud model.

[0021] Further, the prediction network includes a point cloud segmentation network, a first multi-layer perceptron, a second multi-layer perceptron, and a shooting mode confidence prediction unit; randomly initialize the weights of the point cloud segmentation network, the first multi-layer perceptron, and the second multi-layer perceptron.

[0022] Further, the point cloud segmentation network is used to extract global features from the reconstructed point cloud model of the target.

[0023] The first multi-layer perceptron extracts features from the environmental prior information corresponding to the image sequence used to construct the reconstructed point cloud model to obtain environmental features.

[0024] The second multi-layer perceptron performs feature mapping on the quality evaluation score of the reconstructed point cloud model to obtain high-dimensional features.

[0025] Combine the global features extracted by the point cloud segmentation network, the environmental features extracted by the first multi-layer perceptron, and the high-dimensional features mapped by the second multi-layer perceptron into a joint feature vector and input it into the shooting mode confidence prediction unit, and use the shooting mode confidence prediction unit to obtain the original score.

[0026] Normalize the original scores and convert them into a probability distribution to obtain the confidence of each shooting mode; determine the current optimal shooting mode based on the confidence and perform training iterations of the prediction network.

[0027] Further, the training iteration of the prediction network includes:

[0028] Using the current optimal shooting mode as the reselected initial shooting mode, obtain the target acquisition image sequence and perform three-dimensional reconstruction, then iteratively calculate the current optimal shooting mode and train the prediction network until the accuracy and integrity in the quality evaluation score both meet the corresponding preset thresholds, or stop after reaching the preset maximum number of iterations, and output the current optimal shooting mode as the final optimal shooting mode.

[0029] Further, construct a policy gradient loss function with the quality evaluation score as the reward, expressed as:

[0030] ;

[0031] Among them, and respectively represent the accuracy and integrity in the quality evaluation score, is the confidence of the current optimal shooting mode a; update the weights of the prediction network through the gradient descent algorithm during training.

[0032] An imaging system that uses the data dynamic acquisition method for high-precision three-dimensional reconstruction to determine the final optimal shooting mode, thereby obtaining the image sequence of the target and performing three-dimensional reconstruction.

[0033] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, it implements the data dynamic acquisition method for high-precision three-dimensional reconstruction.

[0034] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, it implements the data dynamic acquisition method for high-precision three-dimensional reconstruction.

[0035] Compared with the prior art, the present invention has the following technical features:

[0036] 1. Active perception optimization mechanism: For the first time, combine deep learning with three-dimensional reconstruction evaluation metrics to form a closed-loop system of "shooting - reconstruction - evaluation - optimization".

[0037] 2. Multimodal feature fusion: Combine point cloud geometric features, environmental priors, and historical scores to enhance the scene adaptability of mode prediction.

[0038] 3. End-to-end optimization: The MLP directly learns the mapping from features to shooting modes without manually designing rules.

[0039] 4. Dynamic adaptability: The confidence vector reflects the applicability of each shooting mode in the current scene, and the system can dynamically select the optimal mode.

[0040] 5. Simulation-to-reality transfer framework: Reduce the data acquisition cost through virtual simulation and use domain adaptation technology to solve the problem of real-world gaps. Brief Description of the Drawings

[0041] Figure 1 It is a schematic flowchart of the method of the present invention;

[0042] Figure 2 It is an image sequence obtained by shooting a target in an initial shooting mode in an embodiment of the present invention;

[0043] Figure 3 It is a reconstructed point cloud model of a target using the image sequence obtained in an initial shooting mode in an embodiment of the present invention;

[0044] Figure 4 It is a visualization effect diagram of the ground truth model of a target in an embodiment of the present invention;

[0045] Figure 5 It is a three-dimensional reconstruction effect diagram of a target in the final optimal shooting mode in an embodiment of the present invention;

[0046] Figure 6 It is a three-dimensional reconstruction flowchart in an embodiment of the present invention;

[0047] Figure 7 It is a process architecture diagram of shooting mode optimization in the present invention;

[0048] Figure 8 It is an architecture diagram of the point cloud segmentation network PointNet. Detailed Embodiment

[0049] The present invention provides a data dynamic acquisition method for high-precision three-dimensional reconstruction, and the method includes:

[0050] Step 1, select an initial shooting mode from the preset shooting mode pool of the imaging system, and obtain an image sequence and perform three-dimensional reconstruction on the target in the simulated environment in the initial shooting mode to obtain a reconstructed point cloud model.

[0051] Step 1.1, construction of the shooting mode pool.

[0052] Among them, a variety of shooting modes are stored in the shooting mode pool, and each shooting mode is used for the imaging system to shoot a target to form a corresponding image sequence; the shooting mode is represented as a triple , where represents the i-th shooting mode, represents the angle by which the turntable carrying the imaging system rotates each time in the i-th shooting mode, respectively represent the corresponding exposure time and exposure times.

[0053] In each shooting mode, the turntable starts from the initial position in its rotation range and rotates each time and shoots the target according to to form the corresponding image sequence in this shooting mode.

[0054] When constructing the shooting mode pool, discrete sampling can be performed from the rotation range of the turntable, the exposure time range of the imaging system, and the exposure times range, so as to combine and generate multiple groups of shooting modes.

[0055] Step 1.2, selection of the initial shooting mode.

[0056] Select any shooting mode from the shooting mode pool as the initial shooting mode, or select a shooting mode with better imaging effect according to experience as the initial shooting mode.

[0057] Step 1.3, 3D reconstruction of the target.

[0058] The process of 3D reconstruction using the image sequence is as follows:

[0059] (1) Build a simulation environment to simulate the real state of the target; the simulation environment can be built by software such as Blender, etc. Build the model of the target and the imaging system in the simulation environment and simulate the motion trajectory of the target; the imaging system images the target according to the selected initial shooting mode to obtain the image sequence; during the shooting process of the imaging system, obtain the environmental prior information of the target during shooting, including the corresponding solar illumination angle, the distance between the target and the imaging system, the motion speed and angular velocity of the target.

[0060] (2) For each image in the image sequence, extract the SIFT feature points of the image.

[0061] (3) Brute-force match the images in the image sequence pairwise to obtain the corresponding relationship of feature points between two images, and form a scene graph with images as nodes and the corresponding relationship of feature points as edges.

[0062] (4) Select a pair of images with the most feature point matches for initial reconstruction, solve the pose of the camera in the imaging system, perform triangulation on the matched feature points to obtain the corresponding 3D point cloud, and finally optimize the camera pose and generate a 3D point cloud model using bundle adjustment.

[0063] (5) Incorporate other images in the image sequence (other images except for the pair of matched images) for image registration; during image registration, the camera pose corresponding to the newly added image is solved using the PnP algorithm, perform triangulation on the feature points in the newly added image to obtain 3D points and add them to the 3D point cloud model to continuously improve the 3D point cloud model, while performing local bundle adjustment and outlier filtering; repeat this step until all other images in the image sequence have completed image registration, obtaining the reconstructed point cloud model of the target, denoted as the reconstructed point cloud model R.

[0064] Step 2: Evaluate the quality of the reconstructed point cloud model of the target in the simulated environment based on the true point cloud model of the target to obtain a quality evaluation score.

[0065] Step 2.1: Construct a true point cloud model of the target, denoted as D, and calculate the distance from each 3D point d in the true point cloud model D to the reconstructed point cloud model R. It is:

[0066] ;

[0067] where is the position of the 3D point d, is the position of the 3D point r, represents calculating the distance between two positions.

[0068] The distance from each 3D point r in the reconstructed point cloud model R to the true point cloud model D is:

[0069] .

[0070] Step 2.2: Calculate the quality evaluation score, including accuracy and completeness.

[0071] Accuracy is defined as the average distance from each 3D point in the reconstructed point cloud model R to the corresponding 3D point in the true point cloud model D , that is:

[0072] ;

[0073] where represents the number of 3D points in the reconstructed point cloud model R.

[0074] Completeness is defined as the average distance from each 3D point in the real point cloud model D to the corresponding 3D point in the reconstructed point cloud model R, that is: , namely:

[0075] ;

[0076] where represents the number of 3D points in the real point cloud model D.

[0077] Therefore, the lower the quality assessment score, the better the quality of 3D reconstruction.

[0078] Step 3: Input the reconstructed point cloud model of the target, the quality assessment score, and the environmental prior information in the simulation environment into the prediction network. Construct a policy gradient loss function with the quality assessment score as the reward for iterative optimization of the prediction network, and determine the final optimal shooting mode in the shooting mode pool for the imaging system in the actual environment to obtain the image sequence of the target and 3D reconstruction.

[0079] Step 3.1: Construct a prediction network; the prediction network includes a point cloud segmentation network PointNet, a first multi-layer perceptron (MLP), a second multi-layer perceptron (MLP), and a shooting mode confidence prediction unit; randomly initialize the weights of the point cloud segmentation network PointNet, the first multi-layer perceptron, and the second multi-layer perceptron.

[0080] Step 3.2: Use the point cloud segmentation network PointNet to extract global features (256 dimensions) from the reconstructed point cloud model of the target.

[0081] Step 3.3: Use the first multi-layer perceptron to extract features from the environmental prior information corresponding to the image sequence used to construct the reconstructed point cloud model to obtain environmental features.

[0082] The design purpose of this step is to automatically learn the non-linear relationship between environmental parameters (such as the combined influence of "sunlight angle, distance between the target and the imaging system" on light attenuation) and make the output dimension consistent with the point cloud feature dimension extracted by the point cloud segmentation network PointNet for subsequent fusion.

[0083] Among them, the structure of the first multi-layer perceptron is as follows:

[0084] The input layer is a vector composed of parameters in the environmental prior information. In this embodiment, the dimension of the vector is 4.

[0085] The hidden layer has two layers. The first layer is a fully connected layer (4 dimensions → 64 dimensions), with the activation function ReLU to introduce non-linear expression ability; the first layer is also a fully connected layer (64 dimensions → 128 dimensions), with the activation function ReLU to extract high-order feature interactions.

[0086] The output layer is a fully connected layer (128 - dimensional → 256 - dimensional feature vector), and finally the extracted environmental feature vector is output.

[0087] In step 3.4, the second multi - layer perceptron performs feature mapping on the quality assessment score of the reconstructed point cloud model to obtain high - dimensional features.

[0088] The purpose of this step is to learn the non - linear relationship between integrity and accuracy (for example, high integrity but low accuracy may mean more noise) and to make the output dimension consistent with the dimensions of the global feature and the environmental feature vector for subsequent fusion.

[0089] Among them, the structure of the second multi - layer perceptron is as follows:

[0090] The input layer is a vector composed of accuracy and integrity in the quality assessment score. In this embodiment, the dimension of the vector is 2.

[0091] The hidden layer has two layers. The first layer is a fully connected layer (2 - dimensional → 64 - dimensional), with the activation function ReLU to introduce non - linear expression ability; the second layer is also a fully connected layer (64 - dimensional → 128 - dimensional), with the activation function ReLU to extract high - order feature interactions.

[0092] The output layer is a fully connected layer (128 - dimensional → 256 - dimensional feature vector), and finally the mapped high - dimensional features are output.

[0093] Through multi - layer non - linear transformation, the MLP can learn the complex relationships among the three types of features. For example, low integrity may mean that the shooting angle needs to be increased; low accuracy may mean that the exposure time or the rotation angle needs to be adjusted.

[0094] In step 3.5, the global feature extracted by the point cloud segmentation network, the environmental feature extracted by the first multi - layer perceptron, and the high - dimensional feature mapped by the second multi - layer perceptron are combined into a joint feature vector and input into the shooting mode confidence prediction unit (MPL), and the original score is obtained by using the shooting mode confidence prediction unit.

[0095] Among them, the structure of the shooting mode confidence prediction unit is:

[0096] The input layer is the joint feature vector = [global feature, environmental feature, high - dimensional feature]. In this embodiment, it is a 640 - dimensional feature.

[0097] The hidden layer has two layers. The first layer is a fully connected layer (640 - dimensional → 512 - dimensional), with the activation function ReLU to extract high - order feature interactions; the second layer is also a fully connected layer (512 - dimensional → 256 - dimensional), with the activation function ReLU to further abstract the features.

[0098] The output layer is a fully connected layer (256 - dimensional → - dimensional), is the number of shooting modes in the shooting mode pool, and finally a dimensional vector s is obtained as the original score.

[0099] Step 3.6, perform a normalization operation on the original score and convert it into a probability distribution:

[0100] ;

[0101] where is the confidence of the i-th shooting mode, ; represents the i-th and j-th features in the dimensional vector s.

[0102] Then the current optimal shooting mode a is expressed as:

[0103] .

[0104] Step 3.7, use the current optimal shooting mode a as the initial shooting mode for re-selection. After acquiring the target image sequence and performing 3D reconstruction, iterate Steps 2 and 3 and train the prediction network until both the accuracy and integrity in the quality evaluation score meet the corresponding preset thresholds, or stop after reaching the preset maximum number of iterations, and output the current optimal shooting mode as the final optimal shooting mode.

[0105] When training the prediction network, construct a policy gradient loss function L with the quality evaluation score, which is expressed as follows:

[0106] ;

[0107] where , respectively represent the accuracy and integrity in the quality evaluation score, is the confidence of the current optimal shooting mode a; during training, update the weights of the prediction network through the gradient descent algorithm.

[0108] Write the final optimal shooting mode into the shooting program of the imaging system, and then actual shooting can be performed. After acquiring the image sequence, 3D reconstruction is carried out. The imaging system described in the present invention can be carried on platforms such as roadbed, space-based or sea-based platforms.

[0109] In an embodiment of the present invention, a low-earth orbit satellite (orbital altitude 500 km) is used as the target, and the specific application process of the present invention is as follows:

[0110] Considering factors such as the inclination angle of low-earth orbit satellites, the latitude of ground stations, and horizon occlusion, the rotation range of the imaging system turntable in this embodiment is: azimuth angle of 120° - 180°, pitch angle of 80° - 100°, exposure time of 1 - 500 milliseconds, and exposure times of 0.1 - 10 times per second; based on these parameters, multiple shooting modes are discretely formed to construct a shooting mode pool.

[0111] The initial shooting mode in this embodiment is selected as: the angle of rotation of the turntable each time is 0.5°, the exposure time is 10 milliseconds, and the exposure times are 1 time per second; the image sequence obtained in this shooting mode is as Figure 2 shown, and the reconstructed point cloud model of the target is as Figure 3 shown. It can be seen that the reconstruction result is not accurate.

[0112] The visualization of the true point cloud model of the target is as Figure 4 shown; comparing the reconstructed point cloud model with the true point cloud model, the calculated completeness is 0.36 millimeters, and the accuracy is 1.77 millimeters.

[0113] As Figure 6 and Figure 7 shown, the reconstructed point cloud model of the target, the quality evaluation score, and the environmental prior information in the simulation environment are input into the prediction network. The environmental prior information is specifically set as: solar illumination angle of 30°, distance between the target and the imaging system of 500 kilometers, target movement speed of 7.5 km / s, and angular velocity of 0.1° / s. The current optimal shooting mode is output through the prediction network and enters the next round of iteration; until the final iteration ends, the final optimal shooting mode output is: the angle of rotation of the turntable each time is 1.4°, the exposure time is 12 milliseconds, and the exposure times are 1 time per second.

[0114] Three-dimensional reconstruction is performed using the image sequence obtained with the final optimal shooting mode, and the effect is as Figure 5 shown. At this time, the completeness is 0.23 millimeters, and the accuracy is 0.25 millimeters.

[0115] It can be found by comparison that the shooting mode obtained by the method of the present invention can achieve a better reconstruction effect on the acquired data than experience. Compared with the manual method, the process automation is realized.

[0116] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for dynamic data acquisition for high-precision three-dimensional reconstruction, characterized in that: include: An initial shooting mode is selected from a shooting mode pool preset by the imaging system, and an image sequence is acquired for a target in a simulated environment under the initial shooting mode and three-dimensional reconstruction is performed to obtain a reconstructed point cloud model; Performing a quality assessment on the reconstructed point cloud model of the target in the simulation environment based on the real point cloud model of the target to obtain a quality assessment score; The reconstructed point cloud model of the target in the simulated environment, the quality assessment score, and the prior information of the environment are input into the prediction network. The policy gradient loss function is constructed with the quality assessment score as the reward to iteratively optimize the prediction network. The final optimal shooting mode is determined in the shooting mode pool, which is used for the imaging system to obtain the image sequence and 3D reconstruction of the target in the actual environment. The prediction network includes a point cloud segmentation network, a first multi-layer perceptron, a second multi-layer perceptron, and a shooting mode confidence prediction unit; the weights of the point cloud segmentation network, the first multi-layer perceptron, and the second multi-layer perceptron are randomly initialized; The point cloud segmentation network is used to extract global features from the reconstructed point cloud model of the target; The first multi-layer perceptron extracts features of the environmental prior information corresponding to the image sequence used to construct the reconstructed point cloud model to obtain environmental features; The second multi-layer perceptron performs feature mapping on the quality assessment score of the reconstructed point cloud model to obtain high-dimensional features; The global features extracted by the point cloud segmentation network, the environmental features extracted by the first multi-layer perceptron, and the high-dimensional features mapped by the second multi-layer perceptron are combined into a joint feature vector and input into a shooting mode confidence prediction unit, and the shooting mode confidence prediction unit is used to obtain an original score; The original scores are normalized and converted into probability distributions to obtain the confidence of each shooting mode. The current optimal shooting mode is determined based on the confidence and the prediction network is trained iteratively.

2. The method for dynamic data acquisition for high-precision three-dimensional reconstruction according to claim 1, characterized in that: The shooting mode pool stores a plurality of shooting modes, each of which is used by the imaging system to shoot a target to form a corresponding image sequence; the shooting mode is represented by a triplet ,in Indicates i Shooting modes, Indicates i The angle of each rotation of the turntable carrying the imaging system in each shooting mode, Indicates exposure time and exposure times respectively; in each shooting mode, the turntable starts from the initial position within its rotation range and rotates each time And follow The target is photographed to form a corresponding image sequence under the photographing mode.

3. The method for dynamic data acquisition for high-precision three-dimensional reconstruction according to claim 1, characterized in that: The process of performing three-dimensional reconstruction is as follows: (1) building a simulation environment to simulate the real state of the target; constructing models of the target and the imaging system in the simulation environment and simulating the motion trajectory of the target; the imaging system images the target according to the selected initial shooting mode to obtain the image sequence; and obtaining environmental prior information of the target during shooting by the imaging system; (2) For each image in the image sequence, extract the feature points of the image; (3) Perform brute force matching on the images in the image sequence, obtain the correspondence between the feature points of the two images, and form a scene graph with the images as nodes and the correspondence between the feature points as edges; (4) Select a pair of images with the most matching feature points for initial reconstruction, solve the position and posture of the camera in the imaging system, perform triangulation on the matching feature points to obtain the corresponding 3D point cloud, and finally use bundle adjustment to optimize the camera position and posture and generate a 3D point cloud model; (5) Add other images in the image sequence for image registration. During image registration, the camera pose corresponding to the newly added image is solved using the PnP algorithm. The feature points in the newly added image are triangulated to obtain three-dimensional points and added to the three-dimensional point cloud model to continuously improve the three-dimensional point cloud model. At the same time, local bundle adjustment and outlier filtering are performed. Repeat this step until all other images in the image sequence have completed image registration, and obtain a reconstructed point cloud model of the target, which is recorded as the reconstructed point cloud model.

4. The method for dynamic data acquisition for high-precision three-dimensional reconstruction according to claim 1, characterized in that: The quality of the reconstructed point cloud model of the target in the simulation environment is evaluated based on the real point cloud model of the target, and the quality evaluation score is obtained, including: Construct a real point cloud model of the target; calculate the distance from each 3D point in the real point cloud model to the reconstructed point cloud model, and the distance from each 3D point in the reconstructed point cloud model to the real point cloud model; Calculate quality assessment scores, including accuracy and completeness; accuracy is defined as the average distance between each 3D point in the reconstructed point cloud model and the corresponding 3D point in the real point cloud model, and completeness is defined as the average distance between each 3D point in the real point cloud model and the corresponding 3D point in the reconstructed point cloud model.

5. The method for dynamic data acquisition for high-precision three-dimensional reconstruction according to claim 1, characterized in that: The training iteration of the prediction network includes: Taking the current optimal shooting mode as the reselected initial shooting mode, after acquiring an image sequence of the target and performing three-dimensional reconstruction, the current optimal shooting mode is iteratively calculated and the prediction network is trained until the accuracy and completeness of the quality assessment score meet the corresponding preset thresholds, or the preset maximum number of iterations is reached, and the current optimal shooting mode is output as the final optimal shooting mode.

6. The method for dynamic data acquisition for high-precision three-dimensional reconstruction according to claim 1, characterized in that: The policy gradient loss function is constructed with the quality evaluation score as the reward, expressed as: ; in, , Respectively represent the accuracy and completeness of the quality assessment score. The optimal shooting mode for the current time a The confidence level of the prediction network is updated by the gradient descent algorithm during training.

7. An imaging system, characterized in that: The imaging system uses the data dynamic acquisition method for high-precision three-dimensional reconstruction according to any one of claims 1 to 6 to determine the final optimal shooting mode, thereby acquiring image sequences and performing three-dimensional reconstruction of the target.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; characterized in that: When the processor executes the computer program, the method for dynamic data acquisition for high-precision three-dimensional reconstruction according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Method and device for camera parameter calibration and three-dimensional point cloud generating

    CN102750687A

  • Method for correcting oblique optical axis through three-dimensional point cloud reconstruction technology based on deep learning

    CN118229753A