Real-time iterative three-dimensional modeling method, system and readable medium based on aerial video stream
By integrating POS information and SLAM technology, and utilizing parallel dense matching and point cloud matching, the problem of real-time iterative 3D modeling of dense point cloud and texture mapping, which is difficult to achieve in existing technologies, is solved, and efficient dense point cloud model generation and texture mapping are realized.
Patent Information
- Application Number
- CN202010747745.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-28
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2040-07-28
AI Technical Summary
Existing SFM and SLAM technologies struggle to achieve real-time iterative 3D modeling with dense point clouds and texture mapping.
By integrating POS information and SLAM technology, and through parallel dense matching and point cloud matching, key frames are iteratively matched and optimized frame by frame in real time to generate a dense point cloud model and perform texture mapping.
It enables real-time generation of dense point cloud models with realistic textures, improving the accuracy and efficiency of 3D modeling.
Smart Images

Figure CN112085849B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of video three-dimensional modeling, and in particular to a real-time iterative three-dimensional modeling method, system and readable medium based on aerial video stream. Background Art
[0002] With the widespread application of drones, the use of drone real-time video data for rapid 3D modeling has become a prominent technical demand and important research topic in the application fields of surveying and mapping, security, emergency response, inspection and other industries. The existing image or video-based 3D modeling methods generally use SFM (structure from motion) technology, which has a large number of successful applications in the fields of surveying and mapping and computer vision. Representative open source projects include Bundler, COLMAP, openMVG, etc. However, the bottleneck of SFM technology is the large amount of computation. Since image matching and overall optimization are very time-consuming, offline processing is generally used, which makes it difficult to achieve real-time processing requirements. In recent years, SLAM (simultaneous localization and mapping) technology has been a hot research topic and has been widely used in the fields of unmanned driving and robot automatic navigation. According to different acquisition devices, SLAM technology can be divided into visual SLAM and laser SLAM, but the basic processing framework is similar, including front-end odometer, back-end graph optimization, loop detection and other modules. SLAM technology achieves real-time processing through fast matching, graph optimization and parallel processing. Visual SLAM technology can calculate the position and attitude information based on video data, but due to the low accuracy of position and attitude, current algorithms can generally only obtain sparse point clouds or semi-dense point clouds. Representative algorithms include ORB-SLAM and LSD-SLAM.
[0003] In view of the limitation that current SFM and SLAM technologies are difficult to achieve real-time iterative 3D modeling with dense point cloud and texture mapping, the present invention proposes a real-time iterative 3D modeling method, system and readable medium that integrate POS information and SLAM, parallel dense matching and point cloud matching technology. Summary of the invention
[0004] In view of the limitations of the prior art, the present invention discloses a real-time iterative 3D modeling method, system and readable medium based on aerial video stream, which are used to solve the bottleneck that the current SFM and SLAM technologies are difficult to achieve real-time iterative 3D modeling with dense point cloud and texture mapping.
[0005] The present invention is achieved through the following technical solutions:
[0006] In a first aspect, the present invention discloses a real-time iterative 3D modeling method based on an aerial video stream, comprising the following steps:
[0007] S1 selects the initial key frame according to the number of feature point extractions, and then continuously calculates the overlap rate to extract key frames according to the camera calibration parameters and POS data;
[0008] S2 matches the feature points of the initial key frame obtained in S1 with the second key frame, generates an initial feature point cloud model in combination with POS data, and uses graph optimization method to optimize the whole point cloud and pose accuracy;
[0009] S3 matches the feature points of the subsequent key frames obtained in S1 with the adjacent key frames frame by frame, adds and expands the initial feature point cloud model obtained in S2, and uses the beam method and graph optimization method to optimize the point cloud and pose accuracy as a whole, and obtains a real-time iterative feature point cloud model;
[0010] S4 uses a parallel semi-global dense matching method based on the pose parameters and feature point cloud models obtained in S2 and S3 to perform full pixel dense matching of the newly added key frames with the adjacent key frames frame by frame, and uses point cloud matching technology to fuse the entire point cloud to obtain a real-time iterative dense point cloud model.
[0011] S5 generates a triangulated network and performs texture mapping based on the dense point cloud model obtained by S4, and finally obtains a three-dimensional model with real texture that is iterated in real time.
[0012] Furthermore, in S1, starting from the first frame of video at the start of scanning the task area, fast feature point extraction is performed frame by frame, the first video frame whose number of feature points is not less than a set threshold is selected as the initial key frame, and subsequent key frames are extracted continuously and iteratively according to the overlap rate.
[0013] Furthermore, in S1, the overlap rate of the subsequent video frames and the ground area corresponding to the current key frame image is calculated according to the height information provided by the POS data and the focal length and video image parameters provided by the camera calibration parameters, wherein the relative height of the POS data is H, the focal length of the camera calibration is f, and the principle of similar triangles is used to calculate the four corner points p1, p2, p3, p4 of the video frame image and project them onto the ground to obtain the ground coordinate points P1, P2, P3, P4. The overlap rate of the subsequent video frames with the current key frame can be calculated according to the position and area of the ground quadrilateral.
[0014] Furthermore, in S2, a fast feature point extraction and matching method is used to obtain accurate same-name point pairs of the initial key frame and the second key frame, and then the initial three-dimensional coordinate values of the same-name point pairs are calculated using the posture information provided by the POS data to generate an initial feature point cloud model.
[0015] Furthermore, in S2, the relative position and orientation of the initial feature point cloud is calculated using the five-point method, and is used as a constraint to perform overall graph optimization together with the projection constraint to obtain a higher-precision initial feature point cloud model.
[0016] Furthermore, in S3, after a newly added key frame is subsequently extracted, its adjacent key frames are calculated according to the POS data corresponding to the key frame, and then feature points of the newly added key frame are matched with all adjacent key frames.
[0017] If the match is successful, the connection points are extracted according to the matching results. The connection points are the three-dimensional points corresponding to the matching points with the same name. The feature point cloud of the newly added key frame is added to expand the initial feature point cloud model. At the same time, the corresponding relationship between the connection points and the feature points is used to restore the pose parameters of the newly added key frame.
[0018] If the matching fails, the key frame is retained and its POS data is used as the pose parameters of the key frame. Feature point matching is continued with the subsequently extracted adjacent key frames. After the matching is successful, the feature point cloud of the key frame is added to expand the initial feature point cloud model, and the pose parameters of the key frame are restored and updated, thereby achieving a more complete three-dimensional model effect.
[0019] Furthermore, in S3, after obtaining the initial values of the pose parameters of the newly added key frame, the poses of all current key frames and the feature point cloud data are subjected to overall adjustment optimization by the bundle method; then the position and posture of the key frame are regarded as pose nodes, and the essential matrix decomposition in computer vision is used to obtain the relative position and posture between the pose nodes, and these are optimized together as constraints to obtain a higher-precision feature point cloud model and pose parameters that can be iterated in real time.
[0020] Furthermore, in S4, after obtaining a higher-precision feature point cloud model and posture parameters, a parallel semi-global dense matching method is used to densely match the newly added key frames with adjacent key frames frame by frame, and point cloud matching technology is used to fuse the entire point cloud to obtain a real-time iterative dense point cloud model.
[0021] In a second aspect, the present invention discloses a real-time iterative three-dimensional modeling system based on aerial video streams, wherein the system is used to execute the real-time iterative three-dimensional modeling method based on aerial video streams described in the first aspect, wherein the system includes a key frame calculation and extraction module, a feature point cloud matching optimization module, a dense point cloud matching fusion module and a triangulated network texture mapping module, wherein the key frame calculation and extraction module, the feature point cloud matching optimization module and the dense point cloud matching fusion module are respectively run with three processes, wherein the dense point cloud matching fusion module is run based on a GPU.
[0022] In a third aspect, the present invention discloses a readable medium comprising execution instructions. When a processor of a real-time iterative three-dimensional modeling system based on aerial video stream executes the execution instructions, the real-time iterative three-dimensional modeling system based on aerial video stream executes the real-time iterative three-dimensional modeling method based on aerial video stream described in the first aspect.
[0023] The beneficial effects of the present invention are:
[0024] The present invention integrates POS information and SLAM, parallel dense matching, and point cloud matching technology, uses POS data to continuously extract video key frames, and after the initial key frame matching optimizes the initial feature point cloud model, the subsequent key frames are matched and expanded in real-time iterative frame by frame, the feature point cloud model and the dense point cloud model are optimized as a whole, and a triangulated network and texture mapping are synchronously generated, finally realizing a real-time iterative three-dimensional model with real texture. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0026] Figure 1 is a system architecture diagram of an embodiment of the present invention;
[0027] Figure 2 is a key frame calculation and extraction flow chart of an embodiment of the present invention;
[0028] Figure 3 is a schematic diagram of video frame overlap calculation according to an embodiment of the present invention;
[0029] Figure 4 is a flow chart of initializing a feature point cloud model according to an embodiment of the present invention;
[0030] Figure 5 It is a flowchart of iterative growth optimization of a dense point cloud model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0032] Example 1
[0033] This embodiment discloses a real-time iterative 3D modeling method based on an aerial video stream, comprising the following steps:
[0034] S1 selects the initial key frame according to the number of feature point extractions, and then continuously calculates the overlap rate to extract key frames according to the camera calibration parameters and POS data;
[0035] S2 matches the feature points of the initial key frame obtained in S1 with the second key frame, generates an initial feature point cloud model in combination with POS data, and uses graph optimization method to optimize the whole point cloud and pose accuracy;
[0036] S3 matches the feature points of the subsequent key frames obtained in S1 with the adjacent key frames frame by frame, adds and expands the initial feature point cloud model obtained in S2, and uses the beam method and graph optimization method to optimize the point cloud and pose accuracy as a whole, and obtains a real-time iterative feature point cloud model;
[0037] S4 uses a parallel semi-global dense matching method based on the pose parameters and feature point cloud models obtained in S2 and S3 to perform full pixel dense matching of the newly added key frames with the adjacent key frames frame by frame, and uses point cloud matching technology to fuse the entire point cloud to obtain a real-time iterative dense point cloud model.
[0038] S5 generates a triangulated network and performs texture mapping based on the dense point cloud model obtained by S4, and finally obtains a three-dimensional model with real texture that is iterated in real time.
[0039] In S1, starting from the first frame of video at the beginning of the task area scanning, fast feature point extraction is performed frame by frame, the first video frame whose number of feature points is not less than the set threshold is selected as the initial key frame, and subsequent key frames are extracted continuously and iteratively according to the overlap rate.
[0040] According to the height information provided by the POS data, the focal length provided by the camera calibration parameters, and the video image parameters, the overlap rate of the subsequent video frames and the ground area corresponding to the current key frame image is calculated, where the relative height of the POS data is H, and the focal length of the camera calibration is f. The principle of similar triangles is used to calculate the four corner points p1, p2, p3, and p4 of the video frame image and project them onto the ground to obtain the ground coordinate points P1, P2, P3, and P4. The overlap rate of the subsequent video frames and the current key frame can be calculated based on the position and area of the ground quadrilateral.
[0041] In S2, the feature point fast extraction and matching method is used to obtain accurate same-name point pairs of the initial key frame and the second key frame, and then the posture information provided by the POS data is used to calculate the initial three-dimensional coordinate values of the same-name point pairs to generate an initial feature point cloud model.
[0042] The relative orientation of the initial feature point cloud is calculated using the five-point method, and used as a constraint to perform overall graph optimization solution together with the projection constraint to obtain a higher-precision initial feature point cloud model.
[0043] In S3, after a newly added key frame is subsequently extracted, its adjacent key frames are calculated according to the POS data corresponding to the key frame, and then feature points of the newly added key frame are matched with all adjacent key frames.
[0044] If the match is successful, the connection points are extracted according to the matching results. The connection points are the three-dimensional points corresponding to the matching points with the same name. The feature point cloud of the newly added key frame is added to expand the initial feature point cloud model. At the same time, the corresponding relationship between the connection points and the feature points is used to restore the pose parameters of the newly added key frame.
[0045] If the matching fails, the key frame is retained and its POS data is used as the pose parameters of the key frame. Feature point matching is continued with the subsequently extracted adjacent key frames. After the matching is successful, the feature point cloud of the key frame is added to expand the initial feature point cloud model, and the pose parameters of the key frame are restored and updated, thereby achieving a more complete three-dimensional model effect.
[0046] After obtaining the initial values of the pose parameters of the newly added keyframes, the overall adjustment optimization of all the current keyframe poses and feature point cloud data is performed using the bundle method. Then the position and posture of the keyframes are regarded as pose nodes, and the essential matrix decomposition in computer vision is used to obtain the relative position and posture between the pose nodes, which are then optimized together as constraints to obtain a higher-precision feature point cloud model and pose parameters that can be iterated in real time.
[0047] In S4, after obtaining a higher-precision feature point cloud model and pose parameters, a parallel semi-global dense matching method is used to densely match the newly added key frames with the adjacent key frames frame by frame, and the point cloud matching technology is used to fuse the overall point cloud to obtain a real-time iterative dense point cloud model.
[0048] Example 2
[0049] This embodiment discloses Figure 1 The real-time iterative 3D modeling technology system based on aerial video stream shown in the figure mainly includes four key modules: key frame calculation and extraction module, feature point cloud matching optimization module, dense point cloud matching fusion module and triangulation texture mapping module. In order to meet the requirements of real-time iterative processing, the key frame calculation and extraction module, the feature point cloud matching optimization module and the dense point cloud matching fusion module are respectively run in three processes, among which the dense point cloud matching fusion module is run based on GPU.
[0050] Example 3
[0051] This embodiment discloses Figure 2 The key frame calculation and extraction method shown is different from the traditional SLAM algorithm that uses feature point matching of continuous video frames to extract key frames, which results in large amount of calculation and unstable initialization. This embodiment uses camera calibration parameters and combines POS data to calculate video frame overlap and extract key frames.
[0052] The initial keyframe is the basis for initializing the point cloud model. The number of feature points in the initial keyframe cannot be less than the set threshold (the default is 100) to ensure the subsequent modeling process. In this embodiment, starting from the first frame of video scanned in the task area, the orb algorithm is used to quickly match and extract feature points frame by frame, and the first pair of keyframes with the number of feature points not less than the set threshold are selected as the initial keyframes, and the initial keyframes are used as the current keyframes to start extracting subsequent keyframes in a continuous iterative manner according to the overlap rate.
[0053] The camera calibration parameters mainly include focal length, principal point and lens distortion coefficient. The POS data is the camera position and posture data corresponding to each video frame. The default threshold for the overlap rate setting is 0.5.
[0054] According to the height information provided by the POS data, the focal length provided by the camera calibration parameters, and the video image parameters, the overlap rate of the ground area corresponding to the subsequent video frames and the current key frame image is calculated, such as Figure 3 As shown, the upper shadow polygon is the video frame image, the lower shadow area is its coverage area on the ground, the relative height of the POS data is H, the calibrated focal length of the camera is f, and the four corner points p1, p2, p3, and p4 of the video frame image are calculated using the principle of similar triangles and projected onto the ground to obtain the ground coordinate points P1, P2, P3, and P4. Each video frame image can calculate its corresponding ground quadrilateral area, and the overlap rate of subsequent video frames and the current key frame can be calculated based on the position and area of the quadrilateral.
[0055] Example 4
[0056] This embodiment discloses Figure 4 The feature point cloud initialization method shown is characterized in that the initial key frame is used in combination with POS to initialize the model as the basis of an incremental iterative accretive modeling method, providing ground control point information for subsequent newly added key frames.
[0057] This embodiment uses a fast feature point extraction and matching method to obtain accurate same-name point pairs of the initial key frame and the second key frame, and then uses the posture information provided by the POS data to calculate the initial three-dimensional coordinate values of the same-name point pairs to generate an initial feature point cloud model.
[0058] Since there are errors in the posture parameters of POS data, especially the error in posture has a great influence on the accuracy of the three-dimensional model, this embodiment introduces the five-point relative orientation method to calculate the relative posture of the initial feature point cloud, and uses it as a constraint to perform graph optimization together with the projection constraint to obtain an overall solution, so as to obtain a higher-precision initial feature point cloud model.
[0059] Example 5
[0060] This embodiment discloses an iterative proliferation optimization method for feature point clouds. After a newly added key frame is subsequently extracted, its adjacent key frames are calculated based on the POS data corresponding to the key frame, and then feature points of the newly added key frame are matched with all adjacent key frames.
[0061] If the match is successful, the connection points are extracted according to the matching results. The connection points are the three-dimensional points corresponding to the matching points with the same name. The feature point cloud of the newly added key frame is added to expand the initial feature point cloud model. At the same time, the corresponding relationship between the connection points and the feature points is used to restore the pose parameters of the newly added key frame.
[0062] If the matching fails, unlike the traditional SLAM algorithm which discards the key frame, this embodiment temporarily retains the key frame that failed to match, uses its POS data as the pose parameter of the key frame, continues to match the feature points with the subsequently extracted adjacent key frames, adds the feature point cloud of the key frame to expand the initial feature point cloud model, and restores and updates the pose parameters of the key frame, thereby achieving a more complete three-dimensional model effect.
[0063] After obtaining the initial values of the pose parameters of the newly added keyframes, the overall adjustment optimization of all the current keyframe poses and feature point cloud data is performed using the beam method; then, drawing on the idea of SLAM graph optimization, the position and pose of the keyframes are used as pose nodes, and the essential matrix decomposition in computer vision is used to obtain the relative position and pose between the pose nodes, which are then optimized together as constraints to obtain a higher-precision feature point cloud model and pose parameters that can be iterated in real time.
[0064] Example 6
[0065] This embodiment discloses Figure 5 The iterative proliferation optimization method of the dense point cloud model shown in the figure, after the key frame feature point matching optimization obtains the sparse point cloud model, continues to use dense matching and point cloud fusion technology to obtain the dense point cloud model.
[0066] This embodiment first uses parallel dense matching technology to match each pixel in the key frame image, and adopts the currently widely used semi-global matching method to achieve dense matching and three-dimensional reconstruction of stereo image pairs. Each newly added key frame can generate new point cloud data that is densely matched with adjacent key frames.
[0067] Since there are errors in the posture of the key frames, there will be misalignment between the newly added point cloud data and the fused point cloud data. This embodiment uses point cloud matching technology to perform overall point cloud fusion, specifically the Fast-GICP algorithm, to obtain a real-time iterative dense point cloud model.
[0068] The dense point cloud model can simultaneously generate a triangulated network and perform texture mapping, ultimately obtaining a real-time iterative 3D model with realistic textures.
[0069] Example 7
[0070] This embodiment discloses a readable medium, including execution instructions. When a processor of the real-time iterative three-dimensional modeling system based on aerial video stream executes the execution instructions, the real-time iterative three-dimensional modeling system based on aerial video stream executes the real-time iterative three-dimensional modeling method based on aerial video stream.
[0071] In summary, the key technologies of the present invention include: key frame calculation and extraction, feature point cloud model initialization, and real-time iterative matching optimization between feature point cloud and dense point cloud model.
[0072] The innovation of the present invention lies in: the present invention integrates POS information and SLAM, parallel dense matching, and point cloud matching technology, uses POS data to continuously extract video key frames, and after matching the selected initial key frames and generating an optimized initial feature point cloud model, the subsequent key frames are matched and expanded in real-time iteration frame by frame, the feature point cloud model and the dense point cloud model are optimized as a whole, and a triangulated network and texture mapping are synchronously generated, finally realizing a real-time iterative three-dimensional model with real texture.
[0073] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A real-time iterative 3D modeling method based on aerial video stream, It is characterized in that The method comprises the following steps: S1 selects the initial key frame according to the number of feature point extractions, and then continuously calculates the overlap rate to extract key frames according to the camera calibration parameters and POS data; S2 matches the feature points of the initial key frame obtained in S1 with the second key frame, generates an initial feature point cloud model in combination with POS data, and uses graph optimization method to optimize the whole point cloud and pose accuracy; S3 matches the feature points of the subsequent key frames obtained in S1 with the adjacent key frames frame by frame, adds and expands the initial feature point cloud model obtained in S2, and uses the beam method and graph optimization method to optimize the point cloud and pose accuracy as a whole, and obtains a real-time iterative feature point cloud model; S4 uses a parallel semi-global dense matching method based on the pose parameters and feature point cloud models obtained in S2 and S3 to perform full pixel dense matching of the newly added key frames with the adjacent key frames frame by frame, and uses point cloud matching technology to fuse the entire point cloud to obtain a real-time iterative dense point cloud model. S5 generates a triangulated network and performs texture mapping based on the dense point cloud model obtained by S4, and finally obtains a three-dimensional model with real texture that is iterated in real time; In the S3, after the newly added key frame is subsequently extracted, its adjacent key frames are calculated according to the POS data corresponding to the key frame, and then the newly added key frame is matched with all adjacent key frames for feature points. If the match is successful, the connection points are extracted according to the matching results, and the connection points are the three-dimensional points corresponding to the matching points with the same name. The feature point cloud of the newly added key frame is added to expand the initial feature point cloud model, and the corresponding relationship between the connection points and the feature points is used to restore the pose parameters of the newly added key frame. If the match fails, the key frame is retained, and its POS data is used as the pose parameters of the key frame. Feature point matching is continued with the adjacent key frames extracted subsequently. If the match is successful, the feature point cloud of the key frame is added to expand the initial feature point cloud model, and the pose parameters of the key frame are restored and updated, so as to achieve a more complete three-dimensional model effect. In S3, after obtaining the initial values of the pose parameters of the newly added key frame, the poses of all current key frames and the feature point cloud data are subjected to overall adjustment optimization by the bundle method; then the position and posture of the key frame are regarded as pose nodes, and the essential matrix decomposition in computer vision is used to obtain the relative position and posture between the pose nodes, and these are optimized together as constraints to obtain a higher-precision feature point cloud model and pose parameters that are iterated in real time.
2. The real-time iterative 3D modeling method based on aerial video stream according to claim 1, It is characterized in that In S1, starting from the first frame of video at the start of scanning the task area, fast feature point extraction is performed frame by frame, the first video frame whose number of feature points is not less than a set threshold is selected as the initial key frame, and subsequent key frames are extracted continuously and iteratively according to the overlap rate.
3. The real-time iterative 3D modeling method based on aerial video stream according to claim 1, It is characterized in that In S1, the overlap rate of the ground area corresponding to the subsequent video frames and the current key frame image is calculated according to the height information provided by the POS data and the focal length and video image parameters provided by the camera calibration parameters, wherein the relative height of the POS data is H, the focal length of the camera calibration is f, and the four corner points p1, p2, p3, and p4 of the video frame image are calculated by using the principle of similar triangles and projected onto the ground to obtain the ground coordinate points P1, P2, P3, and P4. The overlap rate of the subsequent video frames and the current key frame can be calculated according to the position and area of the ground quadrilateral.
4. The real-time iterative 3D modeling method based on aerial video stream according to claim 1, It is characterized in that In S2, a fast feature point extraction and matching method is used to obtain accurate same-name point pairs of the initial key frame and the second key frame, and then the initial three-dimensional coordinate values of the same-name point pairs are calculated using the posture information provided by the POS data to generate an initial feature point cloud model.
5. The real-time iterative 3D modeling method based on aerial video stream according to claim 1, It is characterized in that In S2, the relative position and orientation of the initial feature point cloud is calculated using the five-point method, and is used as a constraint to perform overall graph optimization together with the projection constraint to obtain a higher-precision initial feature point cloud model.
6. The real-time iterative 3D modeling method based on aerial video stream according to claim 1, It is characterized in that In S4, after obtaining a higher-precision feature point cloud model and posture parameters, a parallel semi-global dense matching method is used to perform full-pixel dense matching of the newly added keyframes with the adjacent keyframes, and point cloud matching technology is used to perform overall point cloud fusion to obtain a real-time iterative dense point cloud model.
7. A real-time iterative 3D modeling system based on aerial video stream, the system being used to execute the real-time iterative 3D modeling method based on aerial video stream according to any one of claims 1 to 6, It is characterized in that The system includes a key frame calculation and extraction module, a feature point cloud matching optimization module, a dense point cloud matching fusion module and a triangulated network texture mapping module. The key frame calculation and extraction module, the feature point cloud matching optimization module and the dense point cloud matching fusion module are respectively run with three processes, among which the dense point cloud matching fusion module is run based on a GPU.
8. A readable medium comprising execution instructions. When a processor of the real-time iterative three-dimensional modeling system based on aerial video stream executes the execution instructions, the real-time iterative three-dimensional modeling system based on aerial video stream executes the real-time iterative three-dimensional modeling method based on aerial video stream as described in any one of claims 1-6.
Citation Information
Patent Citations
Real human body three-dimensional modeling method specific to personalized virtual fitting
CN104992441A
Three-dimensional reconstruction method, apparatus and device and storage medium
CN108898630A
