Modeling method and system for three-dimensional real-time measurement reconstruction

Abstract

A modeling method for three-dimensional real-time measurement reconstruction, comprising: on the basis of a three-dimensional camera, a laser radar, and an infrared sensor device, generating a multi-source original data set; performing data preprocessing; performing feature extraction and data fusion; performing three-dimensional space construction; by means of a deep machine learning algorithm, performing real-time dynamic adjustment and optimization on a model; and, by adopting an advanced image processing algorithm, generating a high-precision three-dimensional model.

Description

A three-dimensional real-time measurement and reconstruction modeling method and system Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a three-dimensional real-time measurement and reconstruction modeling method and system. Background Art

[0002] Computer vision is a discipline that studies the ability of machines to interpret and understand images or videos. This field involves multiple aspects such as image processing, pattern recognition, and three-dimensional reconstruction. Its goal is to enable computers to simulate and understand the human visual system, thereby achieving advanced processing of visual information.

[0003] 3D real-time measurement and reconstruction modeling is a branch of computer vision. Its primary goal is to acquire and reconstruct a 3D model of an object or scene in real time by collecting images or point cloud data from the environment. This method uses sensors and algorithms to accurately measure and reconstruct 3D information. Its core goal is to achieve fast, accurate, and real-time 3D modeling of real-world scenes. Its goal is to obtain information about the geometric shape, structure, and surface features of an object or scene, achieving a deep understanding of the environment. This deep understanding has broad applications in fields such as autonomous navigation, virtual reality, and industrial manufacturing. By employing sensors and corresponding algorithms, 3D real-time measurement and reconstruction modeling can capture the 3D shape and structure of an object in real time. This method effectively provides high-quality, high-precision 3D models, enabling computers to more comprehensively understand the environment. By employing stereo vision and structured light technology, this method achieves accurate measurement and modeling of real-world scenes, providing important support for these applications.

[0004] Existing 3D modeling methods have limitations. For data acquisition, traditional methods rely on a single data source, using lidar or stereo cameras, which limits the diversity and comprehensiveness of the acquired data. For data processing, many existing methods fail to fully utilize advanced deep learning techniques to extract and fuse features, resulting in inefficient data utilization and inaccurate feature recognition. When it comes to 3D model construction, some traditional techniques are unable to accurately reproduce complex geometric structures, impacting the accuracy and realism of the models. Traditional methods lack sufficient flexibility and adaptability for real-time dynamic adjustment and optimization. For detailed processing, some existing methods fail to implement advanced image processing, limiting the quality and realism of the final models. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a three-dimensional real-time measurement and reconstruction modeling method and system.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: a three-dimensional real-time measurement and reconstruction modeling method, comprising the following steps:

[0007] S1: Based on stereo cameras, lidar, and infrared sensor equipment, a continuous sampling method is used to generate multi-source raw data sets;

[0008] S2: Based on the multi-source original data set, data cleaning and standardization algorithms are used to perform data preprocessing to generate a unified preprocessed data set;

[0009] S3: Based on the preprocessed unified data set, a deep learning algorithm is used to perform feature extraction and data fusion to generate a feature fusion data set;

[0010] S4: Based on the feature fusion dataset, structured light technology and variational autoencoder are used to construct a three-dimensional space and generate a preliminary three-dimensional model;

[0011] S5: Based on the preliminary three-dimensional model, the model is dynamically adjusted and optimized in real time through a machine deep learning algorithm to generate an optimized three-dimensional model;

[0012] S6: Based on the optimized three-dimensional model, an advanced image processing algorithm is used to generate a high-precision three-dimensional model;

[0013] The multi-source original data set is specifically a data set collected by stereo cameras, lidar, and infrared sensor equipment. The pre-processed unified data set includes denoising, format unification, and range standardization processing data. The feature fusion data set specifically refers to the data set processed by the deep learning algorithm. The preliminary three-dimensional model uses structured light technology and variational autoencoders to generate a basic three-dimensional structure. The optimized three-dimensional model is specifically an advanced model optimized and adjusted by a machine deep learning algorithm. The high-precision three-dimensional model is specifically a model that has been detail enhanced and refined by an advanced image processing algorithm.

[0014] As a further solution of the present invention, based on stereo cameras, laser radar, and infrared sensor equipment, a continuous sampling method is used to generate a multi-source raw data set in the following steps:

[0015] S101: Based on a stereo camera, it uses image sequence acquisition and stereo matching algorithms to acquire visual data, perform time synchronization processing, and generate a continuous visual data set;

[0016] S102: Based on the continuous visual data set, using point cloud acquisition and spatial positioning algorithms, performing spatial information acquisition and spatial calibration to generate a spatial point cloud data set;

[0017] S103: Based on the spatial point cloud dataset, thermal imaging capture and temperature difference analysis algorithm are used to acquire thermal imaging data and calibrate hot spots to generate an infrared thermal imaging dataset;

[0018] S104: Based on the infrared thermal imaging data set, a data fusion algorithm is used to merge and verify multi-source data to generate a multi-source original data set.

[0019] As a further solution of the present invention, based on the multi-source original data set, data cleaning and normalization algorithms are used to perform data preprocessing to generate a unified data set after preprocessing. Specifically, the steps are as follows:

[0020] S201: Based on the multi-source original data set, noise filtering and outlier detection algorithms are used to perform data cleaning and data elimination to generate a denoised data set;

[0021] S202: Based on the denoised data set, data interpolation and missing value filling algorithms are used to improve and supplement the data to generate a complete data set;

[0022] S203: Based on the completed data set, adopt a format unification and data conversion algorithm to perform data standardization and format adjustment to generate a unified format data set;

[0023] S204: Based on the unified data set in the format, a range normalization algorithm is used to perform data normalization and scale adjustment to generate a preprocessed unified data set.

[0024] As a further solution of the present invention, based on the preprocessed unified data set, a deep learning algorithm is used to perform feature extraction and data fusion to generate a feature fusion data set. Specifically, the steps are as follows:

[0025] S301: Based on the preprocessed unified data set, a convolutional neural network and feature extraction are used to perform preliminary feature analysis to generate a preliminary feature extraction data set;

[0026] S302: Based on the preliminary feature extraction data set, using recursive neural network and time series analysis, perform time feature analysis to generate a time series feature data set;

[0027] S303: Based on the time series feature dataset, a long short-term memory network and dynamic feature analysis are used to perform dynamic feature analysis to generate a dynamic feature dataset;

[0028] S304: Based on the dynamic feature dataset, a data fusion algorithm is used to integrate and summarize features to generate a feature fusion dataset.

[0029] As a further solution of the present invention, based on the feature fusion dataset, structured light technology and variational autoencoder are used to construct a three-dimensional space and generate a preliminary three-dimensional model. Specifically, the steps are as follows:

[0030] S401: Based on the feature fusion data set, structured light scanning is used to capture the three-dimensional geometric structure and generate an initial point cloud data set;

[0031] S402: Based on the initial point cloud dataset, performing point cloud processing, performing noise filtering and data smoothing, and generating an optimized point cloud dataset;

[0032] S403: Based on the optimized point cloud data set, a three-dimensional reconstruction algorithm is used to perform surface reconstruction to generate a preliminary three-dimensional mesh model;

[0033] S404: Based on the preliminary three-dimensional mesh model, a variational autoencoder is used to perform detail optimization and structural adjustment to generate a preliminary three-dimensional model.

[0034] As a further solution of the present invention, based on the preliminary three-dimensional model, the model is dynamically adjusted and optimized in real time by a machine deep learning algorithm to generate an optimized three-dimensional model. Specifically, the steps are as follows:

[0035] S501: Based on the preliminary three-dimensional model, geometric analysis technology is used to perform model structure evaluation and feature point marking to generate a model feature point set;

[0036] S502: Based on the model feature point set, a machine deep learning algorithm is used to dynamically adjust the feature points and iteratively optimize the strategy to generate an adjusted feature point set;

[0037] S503: Based on the adjusted feature point set, a dynamic model is constructed using real-time simulation technology, and a dynamic three-dimensional model is generated using a model reconstruction algorithm;

[0038] S504: Based on the dynamic three-dimensional model, a refinement adjustment algorithm is used to perform final structural fine-tuning to generate an optimized three-dimensional model.

[0039] As a further solution of the present invention, based on the optimized three-dimensional model, an advanced image processing algorithm is used to perform detail processing to generate a high-precision three-dimensional model. Specifically, the steps are as follows:

[0040] S601: Based on the optimized three-dimensional model, surface texture processing and coloring optimization are performed through texture mapping to generate a texture-optimized three-dimensional model;

[0041] S602: Performing a lighting simulation based on the texture-optimized three-dimensional model to generate a lighting-effect-optimized three-dimensional model;

[0042] S603: Based on the three-dimensional model after the lighting effect optimization, a rendering algorithm is used to generate a three-dimensional model after the rendering effect enhancement;

[0043] S604: Based on the three-dimensional model after the rendering effect is enhanced, final details are refined and image quality is improved through post-processing to generate a high-precision three-dimensional model.

[0044] A three-dimensional real-time measurement, reconstruction and modeling system is used to execute the above-mentioned three-dimensional real-time measurement, reconstruction and modeling method. The system includes a visual data acquisition module, a data preprocessing module, a feature analysis module, a three-dimensional structure capture module, a model optimization and adjustment module, a texture and lighting processing module, and a high-precision rendering module.

[0045] As a further solution of the present invention, the visual data acquisition module is based on a stereo camera and adopts image sequence acquisition and stereo matching algorithm to acquire visual data and perform time synchronization to generate a continuous visual data set;

[0046] The data preprocessing module uses noise filtering and outlier detection algorithms to clean the data based on the multi-source original data set to generate a denoised data set;

[0047] The feature analysis module applies a convolutional neural network to perform preliminary feature analysis based on the preprocessed unified data set to generate a preliminary feature extraction data set;

[0048] The three-dimensional structure capture module uses structured light scanning technology to capture the three-dimensional structure based on the feature fusion data set to generate an initial point cloud data set;

[0049] The model optimization and adjustment module applies geometric analysis technology to perform model evaluation and feature point marking based on the preliminary three-dimensional model to generate a model feature point set;

[0050] The texture and illumination processing module performs surface texture processing and coloring optimization through texture mapping based on the optimized three-dimensional model to generate a texture-optimized three-dimensional model;

[0051] The high-precision rendering module applies a rendering algorithm to perform detail rendering and effect optimization based on the three-dimensional model after illumination effect optimization, and generates a three-dimensional model after rendering effect enhancement.

[0052] As a further solution of the present invention, the visual data acquisition module includes an image sequence acquisition submodule, a point cloud acquisition submodule, a thermal imaging capture submodule, and a data fusion submodule;

[0053] The data preprocessing module includes a noise filtering submodule, a data improvement submodule, a format standardization submodule, and a data normalization submodule;

[0054] The feature analysis module includes a preliminary feature analysis submodule, a time feature analysis submodule, a dynamic feature analysis submodule, and a feature fusion submodule;

[0055] The three-dimensional structure capture module includes a structured light scanning submodule, a point cloud processing submodule, a surface reconstruction submodule, and a detail optimization submodule;

[0056] The model optimization and adjustment module includes a geometric analysis submodule, a feature point dynamic adjustment submodule, a dynamic model construction submodule, and a refinement adjustment submodule;

[0057] The texture and illumination processing module includes a texture mapping submodule and an illumination simulation submodule;

[0058] The high-precision rendering module includes a detail rendering submodule, a rendering effect optimization submodule, and a post-processing submodule.

[0059] Compared with the prior art, the advantages and positive effects of the present invention are:

[0060] In the present invention, by integrating stereo cameras, lidar and infrared sensor equipment, a continuous sampling method is adopted to generate multi-source raw data sets. Data can be acquired from angles and spectra, thereby providing more comprehensive and detailed environmental information. In the data preprocessing stage, the quality and consistency of the data are improved through data cleaning and standardization algorithms, laying a solid foundation for subsequent deep learning processing. The process of feature extraction and data fusion utilizes the powerful capabilities of deep learning, so that data source information can be more accurately identified and integrated. The application of structured light technology and variational autoencoders in the construction of three-dimensional space provides an accurate geometric framework for generating preliminary three-dimensional models. The real-time dynamic adjustment and optimization of the model by machine deep learning algorithms enhances the accuracy and adaptability of the model. The use of advanced image processing algorithms further improves the detail quality and visual effects of the three-dimensional model. Overall, this new solution has achieved a qualitative leap in multiple links of data acquisition, processing and model generation, providing a comprehensive solution for high-precision three-dimensional modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] FIG1 is a schematic diagram of the workflow of the present invention;

[0062] FIG2 is a flow chart of the refinement of S1 of the present invention;

[0063] FIG3 is a flow chart of the refinement of S2 of the present invention;

[0064] FIG4 is a flow chart of the refinement of S3 of the present invention;

[0065] FIG5 is a flow chart of the refinement of S4 of the present invention;

[0066] FIG6 is a flow chart of the refinement of S5 of the present invention;

[0067] FIG7 is a flow chart of the refinement of S6 of the present invention;

[0068] FIG8 is a system flow chart of the present invention;

[0069] FIG9 is a schematic diagram of the system framework of the present invention. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0071] In the description of the present invention, it should be understood that the terms "length," "width," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like, indicating positions or relationships, are based on the positions or relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, in the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

[0072] Example

[0073] Referring to FIG1 , the present invention provides a technical solution: a three-dimensional real-time measurement and reconstruction modeling method, comprising the following steps:

[0074] S1: Based on stereo cameras, lidar, and infrared sensor equipment, a continuous sampling method is used to generate multi-source raw data sets;

[0075] S2: Based on the multi-source original data sets, data cleaning and standardization algorithms are used to perform data preprocessing and generate a unified data set after preprocessing;

[0076] S3: Based on the preprocessed unified dataset, a deep learning algorithm is used to perform feature extraction and data fusion to generate a feature fusion dataset;

[0077] S4: Based on the feature fusion dataset, structured light technology and variational autoencoder are used to construct a three-dimensional space and generate a preliminary three-dimensional model;

[0078] S5: Based on the preliminary 3D model, the model is dynamically adjusted and optimized in real time through a machine deep learning algorithm to generate an optimized 3D model;

[0079] S6: Based on the optimized 3D model, advanced image processing algorithms are used to generate a high-precision 3D model;

[0080] The multi-source original data set is specifically a data set collected by stereo cameras, lidar, and infrared sensor equipment. The unified data set after preprocessing includes denoising, format unification, and range standardization. The feature fusion data set specifically refers to the data set processed by the deep learning algorithm. The preliminary three-dimensional model uses structured light technology and variational autoencoders to generate a basic three-dimensional structure. The optimized three-dimensional model is specifically an advanced model optimized and adjusted by a machine deep learning algorithm. The high-precision three-dimensional model is specifically a model that has been detail enhanced and refined by an advanced image processing algorithm.

[0081] First, in terms of data collection, we utilize stereo cameras, lidar, and infrared sensors to generate multi-source raw data sets, enabling multi-angle and multi-dimensional data collection. This provides a rich and diverse source of information for subsequent processing. This helps improve the comprehensiveness and accuracy of the data, laying a solid foundation for the entire process.

[0082] Secondly, during the data preprocessing phase, data cleaning and standardization algorithms are applied to the raw data to remove noise, standardize the format, and standardize the range. This preprocessing helps improve data quality, eliminate outliers and noise, and make the data more regular and reliable, providing a good data foundation for subsequent feature extraction and data fusion.

[0083] Subsequently, deep learning algorithms are used to extract and fuse features from the preprocessed data to generate a feature-fused dataset. This helps extract feature information from the dataset and integrate the strengths of the data sources, thereby improving the comprehensive cognition of objects or scenes and laying a solid foundation for subsequent 3D space construction.

[0084] During the 3D model construction phase, we combined structured light technology and variational autoencoders to generate a preliminary 3D model. This step not only quickly builds the basic 3D structure but also improves the model's stability and accuracy to a certain extent, laying the foundation for subsequent model optimization.

[0085] The initial 3D model is dynamically adjusted and optimized in real time using a machine learning algorithm to generate an optimized 3D model. This real-time optimization helps continuously improve the accuracy and stability of the model during the construction process, making it more realistic and laying a good foundation for subsequent refined processing.

[0086] Finally, advanced image processing algorithms are applied to enhance and refine the model's details, generating a highly accurate 3D model. This step helps improve the model's accuracy and clarity, making it more realistic and closer to the actual scene, providing higher-quality data support for subsequent applications. The gradual optimization and refinement of the entire process provides multi-level and multi-faceted optimization for the production of 3D models, making them more realistic and accurate, and providing a reliable data foundation for practical applications.

[0087] Refer to Figure 2. Based on stereo cameras, lidar, and infrared sensor equipment, the steps for generating a multi-source raw data set using a continuous sampling method are as follows:

[0088] S101: Based on a stereo camera, it uses image sequence acquisition and stereo matching algorithms to acquire visual data, perform time synchronization processing, and generate a continuous visual data set;

[0089] S102: Based on the continuous visual data set, point cloud acquisition and spatial positioning algorithms are used to obtain spatial information and perform spatial calibration to generate a spatial point cloud data set;

[0090] S103: Based on the spatial point cloud dataset, thermal imaging capture and temperature difference analysis algorithms are used to acquire thermal image data and calibrate hotspots to generate an infrared thermal imaging dataset.

[0091] S104: Based on the infrared thermal imaging data set, a data fusion algorithm is used to merge and verify multi-source data to generate a multi-source original data set.

[0092] First, a stereo camera is used to capture image sequences, and a stereo matching algorithm is used to acquire and time-synchronize visual data. Image sequences are captured at specific time points, and the stereo matching algorithm is used to determine corresponding points within the images, thereby acquiring visual data. This step involves calibrating camera parameters, capturing image sequences, and applying the image matching algorithm to ensure accurate stereo visual data.

[0093] Next, based on the continuous visual data set, point cloud acquisition and spatial positioning algorithms are used to acquire spatial information and achieve spatial calibration. This involves converting the continuous visual data into point cloud data and using spatial positioning algorithms to achieve point cloud data positioning and calibration. Specific operations include generating point cloud data, locating spatial coordinates, and calibrating the data to ensure an accurate spatial point cloud data set.

[0094] Then, based on the spatial point cloud dataset, thermal imaging capture and temperature difference analysis algorithms are used to acquire thermal imaging data and calibrate hotspots. This involves acquiring thermal imaging data using a thermal imaging device and calibrating hotspot information using a temperature difference analysis algorithm. The specific operations include capturing thermal imaging data, analyzing temperature information, and calibrating hotspots to ensure an accurate infrared thermal imaging dataset.

[0095] Finally, based on the infrared thermal imaging dataset, a data fusion algorithm is used to merge and verify multi-source data to generate a multi-source raw dataset. This involves fusing visual, spatial point cloud, and thermal imaging data, and ensuring source consistency through data verification. The specific operations include applying the data fusion algorithm, merging multi-source data, and verifying the data to ensure the generation of a high-quality multi-source raw dataset.

[0096] Overall, this process ensures data accuracy and consistency through meticulous steps and sensor-specific features. Each step has specific operational guidelines to ensure the desired results are achieved during implementation.

[0097] Please refer to Figure 3. Based on the multi-source original data sets, the data cleaning and standardization algorithms are used to perform data preprocessing to generate the unified data set after preprocessing. The specific steps are as follows:

[0098] S201: Based on the original data sets from multiple sources, noise filtering and outlier detection algorithms are used to clean and eliminate data to generate a denoised data set;

[0099] S202: Based on the denoised dataset, data interpolation and missing value filling algorithms are used to improve and supplement the data to generate a complete dataset;

[0100] S203: Based on the completed data set, a format unification and data conversion algorithm is used to perform data standardization and format adjustment to generate a unified format data set;

[0101] S204: Based on the unified format dataset, a range normalization algorithm is used to perform data normalization and scale adjustment to generate a preprocessed unified dataset.

[0102] First, based on the multi-source raw data sets, noise filtering and outlier detection algorithms are used to clean and eliminate the data. This involves identifying and filtering out noise points and outliers in the data to ensure data quality and accuracy. In practice, filtering techniques and statistical methods are used to filter out noise, while outlier detection algorithms are used to eliminate irregularities and generate a de-noised data set.

[0103] Next, based on the denoised dataset, data interpolation and missing value imputation algorithms are used to complete and supplement the data. This involves interpolating missing data and using algorithms to fill in missing values ​​in the dataset to ensure its integrity. The specific operations include selecting an interpolation method, identifying and imputing missing values, and generating a completed dataset.

[0104] Then, based on the completed dataset, we use format unification and data conversion algorithms to standardize and adjust the data format. This involves unifying the source data format to ensure dataset consistency. Specific operations include defining format standards, converting data types, and generating a uniformly formatted dataset.

[0105] Finally, based on the uniformly formatted dataset, a range normalization algorithm is used to normalize and rescale the data. This involves standardizing the data's range of values ​​to ensure a consistent scale, which facilitates subsequent algorithm processing. This involves selecting a normalization method, adjusting the range, and generating a unified, preprocessed dataset.

[0106] Overall, the process uses a series of meticulous steps to clean, complete, standardize, and adjust the range of multi-source raw data sets, ensuring data quality and consistency. Each step has specific operational guidelines to ensure that the desired results can be achieved during the actual implementation of the solution. The detailed operational steps above are based on general descriptions, and the specific implementation needs to be adjusted according to the characteristics and specific circumstances of the data set. In actual operations, it may be necessary to select appropriate noise filtering, outlier detection, interpolation, and missing value filling algorithms, as well as specific methods for data conversion and range standardization to meet actual needs. Therefore, during implementation, it is recommended to select the most appropriate algorithms and parameters according to the specific situation to ensure the effectiveness of data preprocessing.

[0107] Refer to Figure 4. Based on the preprocessed unified data set, the deep learning algorithm is used to perform feature extraction and data fusion. The specific steps for generating a feature fusion data set are as follows:

[0108] S301: Based on the preprocessed unified data set, a convolutional neural network and feature extraction are used to perform preliminary feature analysis to generate a preliminary feature extraction data set;

[0109] S302: Based on the preliminary feature extraction data set, recursive neural network and time series analysis are used to perform time feature analysis to generate a time series feature data set;

[0110] S303: Based on the time series feature dataset, a long short-term memory network and dynamic feature analysis are used to perform dynamic feature analysis to generate a dynamic feature dataset;

[0111] S304: Based on the dynamic feature dataset, a data fusion algorithm is used to integrate and summarize the features to generate a feature fusion dataset.

[0112] First, we perform preliminary processing on the preprocessed unified dataset, using a convolutional neural network to extract spatial features and generate a preliminary feature extraction dataset. Next, we perform time series analysis on the preliminary feature extraction dataset using a recurrent neural network to generate a time series feature dataset. We also perform dynamic feature analysis using a long short-term memory network to generate a dynamic feature dataset.

[0113] Finally, a data fusion algorithm combines the initial feature extraction, time series features, and dynamic features into a fused feature dataset. This dataset integrates spatial, temporal, and dynamic information, providing a more comprehensive feature representation for the deep learning model. In practice, the parameters of each step and the data fusion algorithm are adjusted, and the model is trained and optimized based on the specific problem and data characteristics to achieve optimal feature extraction and data fusion results. This entire process ensures that the deep learning model fully considers multiple aspects of the data, providing a more representative and comprehensive fused feature dataset for subsequent tasks.

[0114] Refer to Figure 5. Based on the feature fusion dataset, structured light technology and variational autoencoders are used to construct a three-dimensional space and generate a preliminary three-dimensional model. The specific steps are as follows:

[0115] S401: Based on the feature fusion dataset, structured light scanning is used to capture the 3D geometric structure and generate an initial point cloud dataset;

[0116] S402: Based on the initial point cloud dataset, point cloud processing is performed to remove noise and smooth the data to generate an optimized point cloud dataset;

[0117] S403: Based on the optimized point cloud data set, a 3D reconstruction algorithm is used to perform surface reconstruction and generate a preliminary 3D mesh model;

[0118] S404: Based on the preliminary 3D mesh model, a variational autoencoder is used to perform detail optimization and structural adjustment to generate a preliminary 3D model.

[0119] First, the feature fusion dataset is scanned using structured light scanning technology to generate an initial point cloud dataset, which captures the basic information of the three-dimensional geometric structure.

[0120] Next, the initial point cloud dataset is subjected to noise filtering and data smoothing to optimize the point cloud data quality and generate a processed optimized point cloud dataset.

[0121] Subsequently, a 3D reconstruction algorithm is applied to reconstruct the surface of the optimized point cloud dataset to generate a preliminary 3D mesh model representing the overall shape of the object.

[0122] Finally, a variational autoencoder optimizes the details and adjusts the structure of the initial 3D mesh model. This model, combined with high-level features from the feature fusion dataset, generates a final preliminary 3D model. This model takes into account multiple aspects of the original data and features richer details and structural features.

[0123] In practice, the parameters and algorithm selection for each step need to be adjusted according to the specific situation to ensure the quality and accuracy of the final 3D model. The entire process aims to fully utilize the feature fusion dataset, combining structured light technology and variational autoencoders to achieve accurate construction and detailed optimization of the 3D space.

[0124] Referring to Figure 6, based on the preliminary 3D model, the model is dynamically adjusted and optimized in real time using a machine deep learning algorithm. The specific steps for generating the optimized 3D model are as follows:

[0125] S501: Based on the preliminary 3D model, geometric analysis technology is used to evaluate the model structure, mark feature points, and generate a model feature point set;

[0126] S502: Based on the model feature point set, a machine deep learning algorithm is used to dynamically adjust the feature points and iteratively optimize the strategy to generate an adjusted feature point set;

[0127] S503: Based on the adjusted feature point set, a dynamic model is constructed using real-time simulation technology, and a dynamic three-dimensional model is generated through a model reconstruction algorithm;

[0128] S504: Based on the dynamic three-dimensional model, a refinement adjustment algorithm is used to perform final structural fine-tuning to generate an optimized three-dimensional model.

[0129] First, the structure of the preliminary 3D model is evaluated through geometric analysis technology to determine the areas that need adjustment, mark the key model feature points, and construct a model feature point set.

[0130] Next, a machine deep learning algorithm is applied and a convolutional neural network model is used to perform real-time dynamic adjustment and iterative optimization of the model feature point set to ensure that the feature points gradually converge over multiple iterations to obtain more accurate feature point positions.

[0131] Subsequently, real-time simulation technology is used to convert the dynamically adjusted feature point set into a real-time dynamic model, and the latest dynamic three-dimensional model is generated through the model reconstruction algorithm to reflect the changes in the model structure in real time.

[0132] Finally, a refinement adjustment algorithm is used to fine-tune the dynamic three-dimensional model to further optimize the structure and details of the model, achieve the final structural fine-tuning effect, and generate the final optimized three-dimensional model.

[0133] In practice, appropriate technologies and algorithms are selected based on the specific model and application scenario, and parameters are adjusted at each step to ensure process effectiveness. The entire process aims to achieve real-time dynamic adjustment and optimization of 3D models through deep learning algorithms, enabling them to adapt to dynamically changing scenarios and improve model accuracy and realism.

[0134] Refer to Figure 7. Based on the optimized 3D model, the steps for generating a high-precision 3D model by using advanced image processing algorithms and performing detail processing are as follows:

[0135] S601: Based on the optimized three-dimensional model, surface texture processing and coloring optimization are performed through texture mapping to generate a texture-optimized three-dimensional model;

[0136] S602: Performing a lighting simulation based on the texture-optimized 3D model to generate a 3D model with optimized lighting effects;

[0137] S603: Based on the optimized three-dimensional model of the lighting effect, a rendering algorithm is used to generate a three-dimensional model with enhanced rendering effect;

[0138] S604: Based on the enhanced 3D model with rendering effect, final details are refined and image quality is improved through post-processing to generate a high-precision 3D model.

[0139] First, texture mapping technology is used to perform surface texture processing and coloring optimization. The texture information is mapped to the surface of the three-dimensional model. The color distribution is adjusted by optimizing the coloring algorithm to generate a texture-optimized three-dimensional model.

[0140] Next, the lighting simulation technology is used to simulate the real lighting effect, and the light and shadow effect of the model is optimized by adjusting the ambient light to generate a three-dimensional model with optimized lighting effect.

[0141] Subsequently, a rendering algorithm is applied to render the model in detail, and the rendering effect is improved by adjusting the rendering parameters and using advanced algorithms to generate a three-dimensional model with enhanced rendering effect.

[0142] Finally, post-processing technology is used to refine the final details and improve the image quality of the 3D model with enhanced rendering effect, generating a high-precision, realistic and visually superior 3D model.

[0143] In practice, appropriate algorithms and techniques are selected based on the specific application scenario, and parameters are adjusted to optimize image processing results. The entire process aims to achieve high accuracy and visual superiority in surface texture, lighting effects, rendering, and details of 3D models through advanced image processing algorithms.

[0144] Please refer to Figure 8, which shows a three-dimensional real-time measurement and reconstruction modeling system. The three-dimensional real-time measurement and reconstruction modeling system is used to execute the above-mentioned three-dimensional real-time measurement and reconstruction modeling method. The system includes a visual data acquisition module, a data preprocessing module, a feature analysis module, a three-dimensional structure capture module, a model optimization and adjustment module, a texture and lighting processing module, and a high-precision rendering module.

[0145] The visual data acquisition module is based on a stereo camera and uses image sequence acquisition and stereo matching algorithms to acquire visual data and perform time synchronization to generate a continuous visual data set;

[0146] The data preprocessing module uses noise filtering and outlier detection algorithms to clean the data based on multi-source original data sets and generate a denoised data set;

[0147] The feature analysis module uses a convolutional neural network to perform preliminary feature analysis based on the preprocessed unified data set and generates a preliminary feature extraction data set;

[0148] The 3D structure capture module uses structured light scanning technology to capture the 3D structure based on the feature fusion dataset and generate an initial point cloud dataset;

[0149] The model optimization and adjustment module uses geometric analysis technology to evaluate the model and mark feature points based on the preliminary 3D model to generate a model feature point set;

[0150] The texture and lighting processing module performs surface texture processing and shading optimization based on the optimized 3D model through texture mapping to generate a texture-optimized 3D model;

[0151] The high-precision rendering module optimizes the three-dimensional model based on the lighting effect, applies the rendering algorithm to perform detail rendering and effect optimization, and generates a three-dimensional model with enhanced rendering effect.

[0152] First, the visual data acquisition module combines stereo cameras and image sequence acquisition technology to obtain high-quality continuous visual data sets. The data preprocessing module uses noise filtering and outlier detection algorithms to clean multi-source raw data and generate high-quality, denoised data sets. This helps improve data integrity and quality, providing a more reliable foundation for subsequent processing.

[0153] The feature analysis module then applies a convolutional neural network to the unified dataset to achieve preliminary feature extraction. This process will help to gain a deeper understanding of the data characteristics and provide meaningful information for subsequent model building.

[0154] Next, the 3D Structure Capture module uses the feature fusion dataset to perform structured light scanning, generating an initial point cloud dataset. The Model Optimization and Adjustment module then evaluates the model, labels feature points, and adjusts them based on geometric analysis techniques. This improves the model's accuracy and realism, making it more realistic.

[0155] Finally, the texture and lighting processing module uses texture mapping and lighting simulation to perform surface texture processing and shading optimization based on the optimized 3D model. The high-precision rendering module utilizes advanced rendering algorithms for detailed rendering and optimized effects. This significantly enhances the realism and visual quality of the model's appearance. The entire system design is dedicated to real-time performance, striving for it wherever possible from data acquisition to model rendering. This makes the system applicable to a variety of fields, including industrial manufacturing, game development, and virtual reality, providing users with fast, high-quality 3D reconstruction services. The organic integration of modules improves overall performance. Through the synergy between modules, the system can achieve more efficient operational processes in data acquisition, processing, analysis, and rendering, improving overall performance and efficiency.

[0156] The beneficial effects of this system are outstanding in improving data quality, model accuracy and rendering realism, providing strong technical support for real-time 3D reconstruction and modeling, and will have a profound impact on scientific research and applications in many industries.

[0157] Please refer to Figure 9 , the visual data acquisition module includes an image sequence acquisition submodule, a point cloud acquisition submodule, a thermal imaging capture submodule, and a data fusion submodule;

[0158] The data preprocessing module includes a noise filtering submodule, a data improvement submodule, a format standardization submodule, and a data normalization submodule;

[0159] The feature analysis module includes a preliminary feature analysis submodule, a time feature analysis submodule, a dynamic feature analysis submodule, and a feature fusion submodule;

[0160] The 3D structure capture module includes a structured light scanning submodule, a point cloud processing submodule, a surface reconstruction submodule, and a detail optimization submodule;

[0161] The model optimization and adjustment module includes a geometric analysis submodule, a feature point dynamic adjustment submodule, a dynamic model construction submodule, and a refinement adjustment submodule;

[0162] The texture and lighting processing module includes a texture mapping submodule and a lighting simulation submodule;

[0163] The high-precision rendering module includes a detail rendering sub-module, a rendering effect optimization sub-module, and a post-processing sub-module.

[0164] In the visual data acquisition module, the image sequence acquisition submodule continuously captures image sequences to record dynamic scenes; the point cloud acquisition submodule is responsible for obtaining point cloud data in space through a stereo camera, providing a basis for three-dimensional modeling; the thermal imaging capture submodule captures the thermal distribution information of objects through thermal imaging technology, assisting in data collection in special scenarios; the data fusion submodule integrates and synchronizes the various types of data collected above to ensure data consistency and accuracy.

[0165] In the data preprocessing module, the noise filtering submodule removes noise from the data through algorithm processing to improve data quality; the data improvement submodule is used to fill in missing data information to ensure the integrity of the data set; the format standardization submodule unifies the format of the source data to facilitate subsequent processing; the data normalization submodule uses standardization processing to make the data at the same level and reduce calculation errors.

[0166] In the feature analysis module, the preliminary feature analysis submodule uses a convolutional neural network to perform preliminary feature extraction on the data; the time feature analysis submodule focuses on analyzing the characteristics of data changing over time; the dynamic feature analysis submodule processes feature changes in dynamic scenarios; and the feature fusion submodule combines the feature analysis results to form a richer and more accurate feature set.

[0167] In the 3D structure capture module, the structured light scanning submodule uses structured light technology to capture the 3D structure of an object; the point cloud processing submodule processes the captured point cloud data to remove redundancy and errors; the surface reconstruction submodule reconstructs the object surface based on the point cloud data; and the detail optimization submodule optimizes and adjusts the details of the reconstructed model.

[0168] In the model optimization and adjustment module, the geometric analysis submodule performs a detailed analysis of the geometric characteristics of the model; the feature point dynamic adjustment submodule adjusts the key feature points of the model according to the analysis results; the dynamic model construction submodule builds a more accurate model based on dynamic data; and the refinement adjustment submodule further optimizes model details to ensure the accuracy and realism of the model.

[0169] In the texture and lighting processing module, the texture mapping submodule is responsible for texture mapping the model surface to enhance the visual effect; the lighting simulation submodule improves the realism and visual effect of the model by simulating lighting conditions.

[0170] In the high-precision rendering module, the detail rendering sub-module performs high-precision detail rendering on the model to increase the model's delicacy; the rendering effect optimization sub-module optimizes rendering parameters to improve rendering quality; the post-processing sub-module performs final visual effect adjustment and optimization to ensure that the final model's visual effect is optimal.

[0171] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A three-dimensional real-time measurement reconstruction and modeling method, characterized in that, it includes the following steps: Based on a stereo camera, lidar, and infrared sensor device, a continuous sampling method is used to generate a multi-source raw data set; Based on the multi-source raw data set, a data cleaning and standardization algorithm is used to perform data preprocessing to generate a unified data set after preprocessing; Based on the unified data set after preprocessing, a deep learning algorithm is used to perform feature extraction and data fusion to generate a feature fusion data set; Based on the feature fusion data set, structured light technology and variational autoencoders are used to construct a three-dimensional space to generate a preliminary three-dimensional model; Based on the preliminary three-dimensional model, a machine deep learning algorithm is used to perform real-time dynamic adjustment and optimization of the model to generate an optimized three-dimensional model; Based on the optimized three-dimensional model, an advanced image processing algorithm is used to generate a high-precision three-dimensional model; The multi-source raw data set is specifically a data set collected by a stereo camera, lidar, and infrared sensor device. The unified data set after preprocessing includes data processed by noise removal, format unification, and range standardization. The feature fusion data set specifically refers to the data set processed by a deep learning algorithm. The preliminary three-dimensional model is a basic three-dimensional structure generated using structured light technology and variational autoencoders. The optimized three-dimensional model is specifically an advanced model optimized and adjusted by a machine deep learning algorithm. The high-precision three-dimensional model is specifically a model after detail enhancement and refinement processing through an advanced image processing algorithm.

2. The three-dimensional real-time measurement reconstruction and modeling method according to claim 1, characterized in that, The step of generating a multi-source raw data set based on a stereo camera, lidar, and infrared sensor device using a continuous sampling method is specifically as follows: Based on the stereo camera, an image sequence acquisition and stereo matching algorithm is used to perform visual data acquisition and time synchronization processing to generate a continuous visual data set; Based on the continuous visual data set, a point cloud acquisition and spatial positioning algorithm is used to perform spatial information acquisition and spatial calibration to generate a spatial point cloud data set; Based on the spatial point cloud data set, a thermal imaging capture and temperature difference analysis algorithm is used to perform thermal image data acquisition and hot spot calibration to generate an infrared thermal imaging data set; Based on the infrared thermal imaging data set, a data fusion algorithm is used to perform multi-source data merging and verification to generate a multi-source raw data set.

3. The three-dimensional real-time measurement reconstruction and modeling method according to claim 1, characterized in that, The step of performing data preprocessing based on the multi-source raw data set using a data cleaning and standardization algorithm to generate a unified data set after preprocessing is specifically as follows: Based on the multi-source raw data set, a noise filtering and outlier detection algorithm is used to perform data cleaning and data elimination to generate a data set after noise removal; Based on the data set after noise removal, a data interpolation and missing value filling algorithm is used to perform data improvement and data supplementation to generate a completed data set; Based on the completed data set, a format unification and data conversion algorithm is used to perform data standardization and format adjustment to generate a data set with unified format; Based on the format-unified dataset, the range normalization algorithm is used to perform data normalization and scale adjustment to generate a preprocessed unified dataset.

4. The three-dimensional real-time measurement and reconstruction modeling method according to claim 1, wherein, the steps of generating a feature fusion dataset by performing feature extraction and data fusion using a deep learning algorithm based on the preprocessed unified dataset are specifically as follows: Based on the preprocessed unified dataset, a convolutional neural network and feature extraction are used to perform preliminary feature analysis to generate a preliminary feature extraction dataset; Based on the preliminary feature extraction dataset, a recurrent neural network and time series analysis are used to perform time feature analysis to generate a time series feature dataset; Based on the time series feature dataset, a long short-term memory network and dynamic feature analysis are used to perform dynamic feature analysis to generate a dynamic feature dataset; Based on the dynamic feature dataset, a data fusion algorithm is used to perform feature integration and summarization to generate a feature fusion dataset.

5. The three-dimensional real-time measurement and reconstruction modeling method according to claim 1, wherein, the steps of generating a preliminary three-dimensional model by performing three-dimensional space construction using structured light technology and variational autoencoder based on the feature fusion dataset are specifically as follows: Based on the feature fusion dataset, structured light scanning is used to capture three-dimensional geometric structures to generate an initial point cloud dataset; Based on the initial point cloud dataset, point cloud processing is used to perform noise filtering and data smoothing to generate an optimized point cloud dataset; Based on the optimized point cloud dataset, a three-dimensional reconstruction algorithm is used to perform surface reconstruction to generate a preliminary three-dimensional mesh model; Based on the preliminary three-dimensional mesh model, a variational autoencoder is used to perform detail optimization and structural adjustment to generate a preliminary three-dimensional model.

6. The three-dimensional real-time measurement and reconstruction modeling method according to claim 1, wherein, the steps of generating an optimized three-dimensional model by performing real-time dynamic adjustment and optimization of the model using a machine deep learning algorithm based on the preliminary three-dimensional model are specifically as follows: Based on the preliminary three-dimensional model, geometric analysis technology is used to perform model structure evaluation and feature point marking to generate a model feature point set; Based on the model feature point set, a machine deep learning algorithm is used to perform dynamic adjustment of feature points and iterative optimization strategies to generate an adjusted feature point set; Based on the adjusted feature point set, real-time simulation technology is used to perform dynamic model construction, and through a model reconstruction algorithm, a dynamic three-dimensional model is generated; Based on the dynamic three-dimensional model, a refinement adjustment algorithm is used to perform final structural fine-tuning to generate an optimized three-dimensional model.

7. The three-dimensional real-time measurement and reconstruction modeling method according to claim 1, wherein, the steps of generating a high-precision three-dimensional model by performing detail processing using an advanced image processing algorithm based on the optimized three-dimensional model are specifically as follows: Based on the optimized three-dimensional model, surface texture processing and coloring optimization are performed through texture mapping to generate a three-dimensional model with optimized texture; Based on the three-dimensional model with optimized texture, light simulation is performed to generate a three-dimensional model with optimized lighting effect; Based on the three-dimensional model optimized by the lighting effect, a rendering algorithm is used to generate a three-dimensional model with enhanced rendering effect; Based on the three-dimensional model with enhanced rendering effect, final detail refinement and image quality improvement are performed through post-processing to generate a high-precision three-dimensional model.

8. A three-dimensional real-time measurement and reconstruction modeling system characterized in that According to the three-dimensional real-time measurement and reconstruction modeling method according to any one of claims 1-7, the system includes a visual data acquisition module, a data preprocessing module, a feature analysis module, a three-dimensional structure capture module, a model optimization and adjustment module, a texture and lighting processing module, and a high-precision rendering module.

9. The three-dimensional real-time measurement and reconstruction modeling system according to claim 8 characterized in that The visual data acquisition module is based on a stereo camera, and uses an image sequence acquisition and stereo matching algorithm to acquire visual data and perform time synchronization to generate a continuous visual data set; The data preprocessing module is based on a multi-source original data set, and uses a noise filtering and outlier detection algorithm to clean the data to generate a denoised data set; The feature analysis module is based on the preprocessed unified data set, and applies a convolutional neural network to perform preliminary feature analysis to generate a preliminary feature extraction data set; The three-dimensional structure capture module is based on the feature fusion data set, and uses structured light scanning technology to capture the three-dimensional structure to generate an initial point cloud data set; The model optimization and adjustment module is based on the preliminary three-dimensional model and applies geometric analysis technology to evaluate the model and mark feature points to generate a model feature point set; The texture and lighting processing module is based on the optimized three-dimensional model, and performs surface texture processing and shading optimization through texture mapping to generate a three-dimensional model with optimized texture; The high-precision rendering module is based on the three-dimensional model optimized by the lighting effect, and applies a rendering algorithm to perform detail rendering and effect optimization to generate a three-dimensional model with enhanced rendering effect.

10. The three-dimensional real-time measurement and reconstruction modeling system according to claim 8 characterized in that The visual data acquisition module includes an image sequence acquisition sub-module, a point cloud acquisition sub-module, a thermal imaging capture sub-module, and a data fusion sub-module; The data preprocessing module includes a noise filtering sub-module, a data improvement sub-module, a format standardization sub-module, and a data normalization sub-module; The feature analysis module includes a preliminary feature analysis sub-module, a time feature analysis sub-module, a dynamic feature analysis sub-module, and a feature fusion sub-module; The three-dimensional structure capture module includes a structured light scanning sub-module, a point cloud processing sub-module, a surface reconstruction sub-module, and a detail optimization sub-module; The model optimization and adjustment module includes a geometric analysis sub-module, a feature point dynamic adjustment sub-module, a dynamic model construction sub-module, and a refinement adjustment sub-module; The texture and lighting processing module includes a texture mapping sub-module and a lighting simulation sub-module; The high-precision rendering module includes a detail rendering sub-module, a rendering effect optimization sub-module, and a post-processing sub-module.

Images