An intelligent 3D modeling method and system based on machine learning

Abstract

The present invention relates to the technical field of intelligent modeling, and specifically discloses an intelligent 3D modeling method and system based on machine learning, including: S1: based on a preset sensor, data collection is performed on a target area to obtain a comprehensive image of the target area, and image processing is performed to obtain a comprehensive processed image; S2: combining intelligent 3D modeling requirements with a feature extraction scheme, extracting key image features from the comprehensive processed image, and obtaining key feature information; S3: performing regional classification based on regional characteristics of the target area, and combining with the intelligent 3D modeling requirements to obtain a 3D reconstruction model of the target area, and generating an initial 3D image in combination with the key feature information; S4: combining preset constraint conditions with the 3D reconstruction model, thereby optimizing the initial 3D image based on the optimization model to obtain a 3D modeling image; and combining the machine learning model to achieve high-precision 3D model reconstruction, improve modeling efficiency, and enhance the real-time performance of 3D modeling.

Description

Technical Field

[0001] The present invention relates to the field of intelligent modeling technology, and in particular to an intelligent three-dimensional modeling method and system based on machine learning. Background Art

[0002] At present, with the rapid development of computer vision and machine learning technologies, intelligent 3D modeling has shown great application potential in many fields such as urban planning, virtual reality, and cultural heritage protection. Especially in the field of industrial development, intelligent 3D modeling is particularly common.

[0003] There is often a large amount of data in the industrial field. However, existing 3D modeling methods often face problems such as insufficient accuracy, low efficiency and low degree of automation when dealing with complex scenes and large-scale data. The generated 3D modeling images also have problems such as insufficient image accuracy and cannot well reflect the characteristics of the target object.

[0004] Therefore, the present invention proposes an intelligent three-dimensional modeling method and system based on machine learning. Summary of the Invention

[0005] The present invention provides an intelligent 3D modeling method and system based on machine learning, which is used to obtain comprehensive and accurate integrated processing images through multi-view and multi-spectral data collection and processing. According to the intelligent modeling requirements and feature extraction scheme, key feature information is accurately extracted, making feature extraction more accurate and efficient. In combination with regional characteristic classification and modeling requirements, the initial machine learning model is determined and optimized to generate an initial 3D image adapted to the target area; and preset constraints are used to perform dual optimization of the model and image, effectively improving the quality of the 3D modeling image, meeting the needs of diverse applications, and improving real-time modeling performance.

[0006] The present invention provides an intelligent three-dimensional modeling method based on machine learning, comprising:

[0007] S1: Based on the preset sensor, multi-view and multi-spectral data of the target area are collected to obtain a comprehensive image of the target area, and image processing is performed to obtain a comprehensive processed image;

[0008] S2: Based on the requirements of intelligent 3D modeling, key image features are extracted from the comprehensive processed images in combination with feature extraction schemes to obtain key feature information;

[0009] S3: Classify the target area based on its regional characteristics, determine the initial machine learning model for each sub-area based on the requirements of intelligent 3D modeling, optimize and integrate the models, obtain a 3D reconstruction model of the target area, and generate an initial 3D image based on key feature information.

[0010] S4: Optimizing the 3D reconstruction model in combination with preset constraints, thereby optimizing the initial 3D image based on the optimized model to obtain a 3D modeling image.

[0011] Preferably, S1: performing multi-view and multi-spectral data acquisition on the target area based on a preset sensor to obtain a comprehensive image of the target area, and performing image processing to obtain a comprehensive processed image, including:

[0012] Based on the regional characteristics of the target area and the requirements of intelligent 3D modeling, the sensor for collecting data in the target area is determined as the preset sensor;

[0013] The target device equipped with a preset sensor collects data of the target area from different perspectives to obtain a first image of the target area in a multi-perspective state;

[0014] Acquire spectral data of different bands based on a preset camera, and obtain multi-band spectral data as a second image;

[0015] The first image and the second image are integrated to obtain a comprehensive image of the target area, and the comprehensive image is processed to obtain a comprehensive processed image.

[0016] Preferably, S2: based on the intelligent 3D modeling requirements, key image features are extracted from the comprehensive processed image in combination with a feature extraction solution to obtain key feature information, including:

[0017] Based on the requirements of intelligent 3D modeling, a feature extraction solution that matches the target area is selected from the preset feature extraction database;

[0018] Based on each feature extraction algorithm in the feature extraction scheme, key image features corresponding to intelligent 3D modeling are extracted from the comprehensive processed image, and the features are sorted and classified to obtain a set of key feature information.

[0019] Preferably, based on the intelligent 3D modeling requirements, a feature extraction scheme that matches the target area is screened from a preset feature extraction database, including:

[0020] Determine the 3D modeling requirements for intelligent 3D modeling based on real-time application scenarios, and determine the initial feature extraction plan based on the 3D modeling requirements;

[0021] Based on the initial feature extraction scheme, the feature types related to intelligent 3D modeling in the image are analyzed and processed comprehensively, and used as the initial feature types;

[0022] A comprehensive feature extraction method for intelligent 3D modeling by comprehensively determining the initial feature extraction scheme and the initial feature type;

[0023] Based on the feature extraction level requirements and feature fusion modality requirements, the comprehensive feature extraction method is optimized to obtain the optimized feature extraction method;

[0024] Based on the initial feature type, the corresponding type of image information is randomly extracted from the comprehensive processed image to verify the optimized feature extraction method, and the qualified optimized feature extraction method is used as the feature extraction scheme of the target area.

[0025] Preferably, S3: performing regional classification based on regional characteristics of the target area, determining an initial machine learning model for each sub-area in combination with intelligent 3D modeling requirements, and performing model optimization and integration to obtain a 3D reconstruction model of the target area, and generating an initial 3D image in combination with key feature information, including:

[0026] Based on the regional characteristics of the target area, the region is classified. Based on the regional classification results and the requirements of intelligent 3D modeling, an initial machine learning model that matches each sub-region is screened from the preset model database to obtain an initial machine learning model set for the target area.

[0027] Performing model training on each initial machine learning model in the initial machine learning model set based on the key feature information set and the comprehensive processed image, and performing corresponding model optimization based on each training result;

[0028] Obtaining a three-dimensional reconstruction model of the target area based on the model optimization results of each initial machine learning model in the machine learning model set;

[0029] The key feature information is input into the 3D reconstruction model to obtain the initial 3D image of the target area.

[0030] Preferably, the key feature information is input into the three-dimensional reconstruction model to obtain an initial three-dimensional image of the target area, including:

[0031] Randomly extract any frame image from the comprehensive processed image as the reference image of the target area;

[0032] Extracting key feature information corresponding to the current reference image, and based on the type of the key feature information, extracting processed images of frames adjacent to the current reference image from the integrated processed images of the same band to obtain a second reference image set;

[0033] Extracting key feature information of each second reference image in the second reference image set, and combining the key feature information corresponding to the reference image to obtain a first key feature information set of the target area;

[0034] Inputting the key feature information in the first key feature information set into the three-dimensional reconstruction model in combination with the regional parameters of the corresponding sub-region to obtain a three-dimensional skeleton image of the target region in the current sub-region;

[0035] Randomly extract the relative coordinates of any edge point in the benchmark modeling image and compare them with the relative coordinates of the corresponding edge point in the synthetically processed image;

[0036] If the comparison error is less than the real-time modeling accuracy requirement, the 3D skeleton image is used as the reference modeling image;

[0037] Otherwise, a comprehensive processed image of the adjacent bands of the current reference image is extracted as an auxiliary reference image, and key feature information corresponding to the auxiliary reference image is extracted to obtain a set of auxiliary images adjacent to the auxiliary reference image, thereby obtaining an auxiliary feature information set;

[0038] The auxiliary feature information set is combined with the regional parameters of the corresponding sub-region and input into the 3D reconstruction model to obtain a new 3D skeleton image, and compared with it to obtain a reference modeling image;

[0039] performing error compensation on the benchmark modeling image based on the image error influence of the benchmark modeling image, thereby optimizing the benchmark modeling image to obtain a benchmark optimized image;

[0040] The reference optimized image is used as the regional modeling reference image of the current sub-region, and is combined with the key feature information belonging to the current region in the first key feature information set and input into the 3D reconstruction model to obtain the regional 3D modeling image of the current sub-region;

[0041] Performing image stitching on the regional three-dimensional modeling images of each sub-region of the target region, thereby obtaining a three-dimensional stitched image of the target region;

[0042] Based on the real-time modeling accuracy requirement, the amount of stitching feature information is obtained, and based on the stitching feature information, a comprehensive processed image near each stitching area in the three-dimensional stitching image is obtained to obtain a stitching image set;

[0043] Randomly extracting feature information other than the key feature information from the spliced ​​image set, and when the feature information satisfies the splicing feature information amount, using the extracted feature information as a second key feature information set;

[0044] Performing three-dimensional filling on the three-dimensional stitched image based on feature information in the second key feature information set to obtain a three-dimensional optimized image;

[0045] Image fitting and image prediction are performed on each spliced ​​area image in the three-dimensional optimized image to obtain the initial three-dimensional image of the target area.

[0046] Preferably, S4: optimizing the three-dimensional reconstruction model in combination with preset constraints, thereby optimizing the initial three-dimensional image based on the optimized model to obtain a three-dimensional modeling image, including:

[0047] Based on the real-time application scenario, the preset constraints of the 3D processing model are optimized, and then the 3D processing model is constrained based on the constraints after the optimization, and the model is optimized in combination with the model optimization algorithm to obtain an optimized 3D model;

[0048] Perform intelligent 3D modeling of the target area based on the optimized 3D model to obtain a 3D optimized image of the target area;

[0049] The three-dimensional optimized image and the initial three-dimensional image are input into the same three-dimensional coordinate system to perform image comparison;

[0050] If the image error between the 3D optimized image and the initial 3D image is less than the real-time modeling accuracy requirement, the initial 3D image is used as the 3D modeling image of the target area;

[0051] Otherwise, the initial three-dimensional image is optimized based on the three-dimensional optimized image to obtain a three-dimensional modeling image of the target area.

[0052] Preferably, based on the real-time application scenario, the preset constraints of the three-dimensional processing model are constrained and optimized, thereby constraining the three-dimensional processing model based on the constraints after constraint optimization, and optimizing the model in combination with the model optimization algorithm to obtain an optimized three-dimensional model, including:

[0053] Obtain key scene information related to 3D modeling in real-time application scenarios;

[0054] Dynamically adjust the constraint parameters of the preset constraint conditions of the three-dimensional processing model according to the key scene information to obtain the constraint optimization parameters;

[0055] The constrained optimization parameters are combined with the parameters that are not constrained to be optimized in the preset constraint conditions to obtain the constrained optimization conditions, and the three-dimensional processing model is constrained based on the constrained optimization conditions;

[0056] The constrained three-dimensional processing model is optimized based on a preset optimization algorithm to obtain an optimized three-dimensional model.

[0057] The present invention provides a machine learning-based intelligent 3D modeling system for executing the machine learning-based intelligent 3D modeling method described in any one of Embodiments 1 to 8, comprising:

[0058] The acquisition and processing module is used to acquire multi-view and multi-spectral data of the target area based on the preset sensor to obtain a comprehensive image of the target area, and perform image processing to obtain a comprehensive processed image;

[0059] Feature extraction module: used to extract key image features from the comprehensive processed image based on the requirements of intelligent 3D modeling and combine feature extraction solutions to obtain key feature information;

[0060] Model Optimization Module: This module is used to classify regions based on their characteristics, determine the initial machine learning model for each sub-region based on the requirements of intelligent 3D modeling, optimize and integrate the models, obtain a 3D reconstruction model of the target region, and generate an initial 3D image based on key feature information.

[0061] Image optimization module: used to optimize the 3D reconstruction model in combination with preset constraints, thereby optimizing the initial 3D image based on the optimized model to obtain a 3D modeling image.

[0062] Compared with the existing technology, the beneficial effects of the present invention are as follows: through multi-view, multi-spectral data acquisition and processing, a comprehensive and accurate integrated processing image is obtained; according to the intelligent modeling requirements and feature extraction scheme, key feature information is accurately extracted, making feature extraction more accurate and efficient; and combined with the regional characteristic classification and modeling requirements, the initial machine learning model is determined and optimized to generate an initial three-dimensional image adapted to the target area; and preset constraints are used to perform dual optimization of the model and image, effectively improving the quality of the three-dimensional modeling image, meeting the needs of diverse applications, and improving real-time modeling performance.

[0063] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0064] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0066] Figure 1 Schematic diagram of an intelligent three-dimensional modeling method based on machine learning in an embodiment of the present invention;

[0067] Figure 2 This is a structural diagram of an intelligent three-dimensional modeling system based on machine learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0069] Example 1:

[0070] The present invention provides an intelligent three-dimensional modeling method based on machine learning. Figure 1 ,include:

[0071] S1: Based on the preset sensor, multi-view and multi-spectral data of the target area are collected to obtain a comprehensive image of the target area, and image processing is performed to obtain a comprehensive processed image;

[0072] S2: Based on the requirements of intelligent 3D modeling, key image features are extracted from the comprehensive processed images in combination with feature extraction schemes to obtain key feature information;

[0073] S3: Classify the target area based on its regional characteristics, determine the initial machine learning model for each sub-area based on the requirements of intelligent 3D modeling, optimize and integrate the models, obtain a 3D reconstruction model of the target area, and generate an initial 3D image based on key feature information.

[0074] S4: Optimizing the 3D reconstruction model in combination with preset constraints, thereby optimizing the initial 3D image based on the optimized model to obtain a 3D modeling image.

[0075] In this embodiment, the preset sensor refers to a device for data collection that is pre-selected and configured based on specific needs and objectives before starting data collection. For example, the sensor may include an optical camera, a multispectral camera, a lidar, etc.

[0076] In this embodiment, multi-view data acquisition is to acquire data of the target area from different angles and orientations. By changing the position, direction or posture of the sensor, images or data of the target area at different perspectives can be obtained.

[0077] In this embodiment, multispectral data acquisition utilizes sensors that can sense different spectral bands to simultaneously acquire data of the target area in multiple spectral bands.

[0078] In this embodiment, the composite image is an image obtained by fusing various data acquired through multi-view and multi-spectral data acquisition.

[0079] In this embodiment, image processing is a series of operations and transformations performed on the collected composite image to improve image quality, extract useful information, or prepare for subsequent analysis and processing. Common image processing operations include image enhancement, image filtering, image segmentation, etc.

[0080] In this embodiment, the comprehensive processed image is based on the comprehensive image by applying various image processing techniques to remove noise, enhance image features, extract useful information, etc., so that the image is more suitable for subsequent feature extraction and three-dimensional modeling operations.

[0081] In this embodiment, intelligent 3D modeling requirements refer to requirements put forward based on specific application scenarios and goals during the 3D modeling process, which may include modeling accuracy requirements, modeling speed requirements, modeling content requirements, and modeling automation requirements.

[0082] In this embodiment, the feature extraction scheme is a method and strategy for extracting key image features from the comprehensive processed image. For example, the feature extraction scheme includes selecting a suitable feature extraction algorithm, determining feature extraction parameters, and a feature extraction process.

[0083] In this embodiment, key image features are information extracted from the comprehensively processed image that reflects important characteristics of the target area. Key image features can include edges, corners, textures, color distribution, etc. in the image. For example, in 3D building modeling, building edges and corners can be used as key image features to determine the building's shape and structure.

[0084] In this embodiment, regional characteristics refer to the unique properties of the target area in terms of geography, environment, function, etc.

[0085] In this embodiment, region classification involves dividing a target region into different sub-regions based on its regional characteristics. For example, a target region can be divided into static, dynamic, complex, and simple regions. The purpose of region classification is to utilize more appropriate 3D modeling methods and techniques based on the characteristics of different sub-regions, thereby improving the accuracy and efficiency of 3D modeling.

[0086] In this embodiment, the initial machine learning model is a pre-selected machine learning model for 3D reconstruction based on the regional characteristics of each sub-region and the requirements for intelligent 3D modeling. The machine learning model is selected from a preset model database. For example, for sub-regions with relatively flat terrain, a rule-based modeling approach or a simple machine learning model can be selected; for sub-regions with complex terrain and dense buildings, a more complex model such as a deep learning model can be selected.

[0087] In this embodiment, model optimization and integration is the process of optimizing and integrating the initial machine learning models for each sub-region. Model integration is the process of fusing the optimized models of each sub-region to form a 3D reconstruction model that can cover the entire target area.

[0088] In this embodiment, the 3D reconstruction model is obtained by model optimization and integration, and is capable of representing the 3D structure and features of the target area. For example, the 3D reconstruction model can be a digital elevation model (DEM), a 3D point cloud model, a 3D mesh model, or a 3D solid model.

[0089] In this embodiment, the initial three-dimensional image is a three-dimensional image generated by inputting the pendant feature information into the three-dimensional reconstruction model of the target area.

[0090] In this embodiment, the preset constraint condition is a constraint condition followed when optimizing the 3D reconstructed model. The preset constraint condition may be a physical constraint, a geometric constraint, or an application constraint.

[0091] In this embodiment, model optimization is the process of adjusting and improving the 3D reconstruction model based on preset constraints. For example, during the model optimization process, various optimization algorithms (such as gradient descent algorithms and genetic algorithms) are used to adjust the model parameters so that the model achieves better performance and accuracy while satisfying the constraints.

[0092] In this embodiment, image optimization is the process of improving the initial 3D image based on the optimized 3D reconstruction model. By performing operations such as denoising, smoothing, and texture mapping on the initial 3D image, the quality and realism of the image can be improved, making it more consistent with the actual target area characteristics.

[0093] The beneficial effects of the above technologies are: through multi-view, multi-spectral data acquisition and processing, comprehensive and accurate integrated processing images are obtained, and key feature information is accurately extracted according to intelligent modeling requirements and feature extraction solutions, making feature extraction more accurate and efficient. In combination with regional characteristic classification and modeling requirements, the initial machine learning model is determined and optimized to generate an initial three-dimensional image adapted to the target area; and preset constraints are used to perform dual optimization of the model and image, effectively improving the quality of three-dimensional modeling images, meeting diverse application needs, and improving real-time modeling performance.

[0094] Example 2:

[0095] Based on Example 1, an intelligent three-dimensional modeling method based on machine learning, S1: Based on a preset sensor, multi-view and multi-spectral data of a target area is collected to obtain a comprehensive image of the target area, and image processing is performed to obtain a comprehensive processed image, including:

[0096] Based on the regional characteristics of the target area and the requirements of intelligent 3D modeling, the sensor for collecting data in the target area is determined as the preset sensor;

[0097] The target device equipped with a preset sensor collects data of the target area from different perspectives to obtain a first image of the target area in a multi-perspective state;

[0098] Acquire spectral data of different bands based on a preset camera, and obtain multi-band spectral data as a second image;

[0099] The first image and the second image are integrated to obtain a comprehensive image of the target area, and the comprehensive image is processed to obtain a comprehensive processed image.

[0100] In this embodiment, regional characteristics refer to the unique properties of the target area in terms of geography, environment, function, etc.

[0101] In this embodiment, intelligent 3D modeling requirements refer to requirements put forward based on specific application scenarios and goals during the 3D modeling process, which may include modeling accuracy requirements, modeling speed requirements, modeling content requirements, and modeling automation requirements.

[0102] In this embodiment, the preset sensor refers to a device for data collection that is pre-selected and configured based on specific needs and objectives before starting data collection. For example, the sensor may include an optical camera, a multispectral camera, a lidar, etc.

[0103] In this embodiment, the target device is a vehicle equipped with pre-set sensors for collecting data from the target area. This can be an unmanned aerial vehicle (UAV), a manned aircraft, a ground vehicle, a fixed measurement platform, or the like. Choosing an appropriate target device requires consideration of factors such as the sensor's weight, size, and power consumption, as well as the target area's terrain and environmental conditions. For example, for a large target area, a drone may be a better choice due to its high maneuverability and wide coverage.

[0104] In this embodiment, multi-view data acquisition involves acquiring data about the target area from different angles and orientations. By changing the position, orientation, or posture of the sensor, images or data of the target area at different perspectives can be obtained, and a first image is obtained based on the multi-view data acquisition results.

[0105] In this embodiment, multispectral data acquisition utilizes a sensor that can sense different spectral bands to simultaneously acquire data of the target area in multiple spectral bands, and obtains a second image based on the multispectral data acquisition result.

[0106] In this embodiment, the composite image is an image obtained by integrating the first image and the second image, and includes both the geometric shape, texture, color and other appearance information of the target area at different viewing angles, and the material characteristic information of the target area at different spectral bands.

[0107] In this embodiment, image processing is a series of operations and transformations performed on the composite image to improve image quality, extract useful information, or prepare for subsequent analysis and processing. For example, image processing operations include image enhancement, image filtering, image segmentation, feature extraction, etc.

[0108] In this embodiment, the comprehensive processed image is an image obtained after image processing.

[0109] The beneficial effects of the above technology are: by determining the preset sensors through regional characteristics and modeling requirements, it is possible to accurately adapt to the collection requirements, and use the target device equipped with the preset sensor to collect data from multiple perspectives to obtain a comprehensive first image, and then combine the preset camera to obtain multi-band spectral data to form a second image, and integrate and process it into a comprehensive processed image, providing rich, accurate and comprehensive data for three-dimensional modeling, which helps to improve the accuracy of three-dimensional modeling.

[0110] Example 3:

[0111] Based on Example 2, a machine learning-based intelligent 3D modeling method, S2: based on the intelligent 3D modeling requirements, in combination with a feature extraction scheme, extracts key image features from the integrated processed image to obtain key feature information, including:

[0112] Based on the requirements of intelligent 3D modeling, a feature extraction solution that matches the target area is selected from the preset feature extraction database;

[0113] Based on each feature extraction algorithm in the feature extraction scheme, key image features corresponding to intelligent 3D modeling are extracted from the comprehensive processed image, and the features are sorted and classified to obtain a set of key feature information.

[0114] In this embodiment, intelligent 3D modeling requirements refer to requirements put forward based on specific application scenarios and goals during the 3D modeling process, which may include modeling accuracy requirements, modeling speed requirements, modeling content requirements, and modeling automation requirements.

[0115] In this embodiment, the feature extraction scheme is a method and strategy for extracting key image features from the comprehensive processed image. For example, the feature extraction scheme includes selecting a suitable feature extraction algorithm, determining feature extraction parameters, and a feature extraction process.

[0116] In this embodiment, key image features are information extracted from the comprehensively processed image that reflects important characteristics of the target area. Key image features can include edges, corners, textures, color distribution, etc. in the image. For example, in 3D building modeling, building edges and corners can be used as key image features to determine the building's shape and structure.

[0117] In this embodiment, the key feature information set is a whole formed by summarizing the key image features after feature sorting and classification, and includes all the key feature information required for the target area in the intelligent 3D modeling process.

[0118] The beneficial effects of the above technology are: by combining intelligent modeling needs and feature extraction solutions, key feature information can be accurately extracted, making feature extraction more accurate and efficient, thereby making the three-dimensional modeling of the target area more accurate and obtaining a more effective three-dimensional modeling image.

[0119] Example 4:

[0120] Based on Example 3, a machine learning-based intelligent 3D modeling method is provided, which selects a feature extraction scheme that matches the target area from a preset feature extraction database based on the intelligent 3D modeling requirements, including:

[0121] Determine the 3D modeling requirements for intelligent 3D modeling based on real-time application scenarios, and determine the initial feature extraction plan based on the 3D modeling requirements;

[0122] Based on the initial feature extraction scheme, the feature types related to intelligent 3D modeling in the image are analyzed and processed comprehensively, and used as the initial feature types;

[0123] A comprehensive feature extraction method for intelligent 3D modeling by comprehensively determining the initial feature extraction scheme and the initial feature type;

[0124] Based on the feature extraction level requirements and feature fusion modality requirements, the comprehensive feature extraction method is optimized to obtain the optimized feature extraction method;

[0125] Based on the initial feature type, the corresponding type of image information is randomly extracted from the comprehensive processed image to verify the optimized feature extraction method, and the qualified optimized feature extraction method is used as the feature extraction scheme of the target area.

[0126] In this embodiment, real-time application scenarios refer to application scenarios that actually occur at a specific time and under specific circumstances and require an immediate response. For example, in a drone inspection scenario, there may be real-time conditions such as temporary obstacles and environmental wind effects.

[0127] In this embodiment, the initial feature extraction scheme is a method and strategy for extracting features from image data determined according to the 3D modeling requirements, for example, including selecting a specific feature extraction algorithm, determining the algorithm parameter settings, and planning the algorithm application order.

[0128] In this embodiment, the initial feature type is a feature category related to intelligent 3D modeling determined by analyzing the integrated processed image based on the initial feature extraction scheme. The chef feature type may include geometric features, texture features, spectral features, etc.

[0129] In this embodiment, the comprehensive feature extraction method is a feature extraction method for intelligent 3D modeling, determined by comprehensively considering the initial feature extraction scheme and the initial feature type. It combines the algorithm selection and parameter settings in the initial feature extraction scheme with the feature range covered by the initial feature type, aiming to more comprehensively and accurately extract feature information useful for 3D modeling from the synthetically processed image. The comprehensive feature extraction method is one of the key steps in achieving 3D model construction from image data.

[0130] In this embodiment, feature extraction hierarchical requirements refer to the different levels of feature extraction requirements during intelligent 3D modeling. For example, low-level feature extraction focuses on basic image elements such as pixels and edges, while high-level feature extraction prioritizes semantically meaningful features such as object category and posture. The feature extraction hierarchical requirements impact the complexity and effectiveness of comprehensive feature extraction methods.

[0131] In this embodiment, feature fusion modality requirements refer to the methods and requirements for fusing different features in intelligent 3D modeling. Feature fusion can integrate feature information from different sensors, different viewpoints, or different feature extraction algorithms to improve the accuracy and robustness of 3D modeling. For example, feature fusion modality requirements may include selecting an appropriate fusion algorithm and determining the timing and level of fusion.

[0132] In this embodiment, method optimization is the process of improving and perfecting the comprehensive feature extraction method based on the feature extraction level requirements and feature fusion modality requirements. The optimization method may include adjusting algorithm parameters, introducing new feature extraction algorithms, improving feature fusion strategies, etc.

[0133] In this embodiment, the optimized feature extraction method is obtained after method optimization and can better meet the requirements of intelligent 3D modeling. The optimized feature extraction method has improved the accuracy, efficiency and robustness of feature extraction compared to the comprehensive feature extraction method.

[0134] In this embodiment, verification refers to a process of randomly extracting image information of corresponding types from the comprehensive processed image based on the initial feature types, and testing and evaluating the optimized feature extraction method.

[0135] In this embodiment, the feature extraction scheme of the target area is a verified and qualified optimized feature extraction method, which is ultimately used for the feature extraction scheme of the intelligent three-dimensional modeling of the target area.

[0136] The beneficial effects of the above technology are: by combining intelligent modeling needs and feature extraction solutions, key feature information can be accurately extracted, making feature extraction more accurate and efficient, thereby making the three-dimensional modeling of the target area more accurate and obtaining a more effective three-dimensional modeling image.

[0137] Example 5:

[0138] Based on Example 3, a machine learning-based intelligent 3D modeling method, S3: performing regional classification based on regional characteristics of the target area, determining an initial machine learning model for each sub-area in combination with intelligent 3D modeling requirements, and performing model optimization and integration to obtain a 3D reconstruction model of the target area, and generating an initial 3D image in combination with key feature information, including:

[0139] Based on the regional characteristics of the target area, the region is classified. Based on the regional classification results and the requirements of intelligent 3D modeling, an initial machine learning model that matches each sub-region is screened from the preset model database to obtain an initial machine learning model set for the target area.

[0140] Performing model training on each initial machine learning model in the initial machine learning model set based on the key feature information set and the comprehensive processed image, and performing corresponding model optimization based on each training result;

[0141] Obtaining a three-dimensional reconstruction model of the target area based on the model optimization results of each initial machine learning model in the machine learning model set;

[0142] The key feature information is input into the 3D reconstruction model to obtain the initial 3D image of the target area.

[0143] In this embodiment, regional characteristics refer to the unique properties of the target area in terms of geography, environment, function, etc.

[0144] In this embodiment, region classification involves dividing a target region into different sub-regions based on its regional characteristics. For example, a target region can be divided into static, dynamic, complex, and simple regions. The purpose of region classification is to utilize more appropriate 3D modeling methods and techniques based on the characteristics of different sub-regions, thereby improving the accuracy and efficiency of 3D modeling.

[0145] In this embodiment, the preset model database is a pre-built database that stores a variety of different types of machine learning models. These models are pre-trained and optimized for different application scenarios and data characteristics. The preset model database provides a rich selection of options for subsequent screening of initial machine learning models based on regional classification results, enabling rapid identification of models that match the target regional sub-regions, reducing the time and cost of model development.

[0146] In this embodiment, the initial machine learning model is a pre-selected machine learning model for 3D reconstruction based on the regional characteristics of each sub-region and the requirements for intelligent 3D modeling. The machine learning model is selected from a preset model database. For example, for sub-regions with relatively flat terrain, a rule-based modeling approach or a simple machine learning model can be selected; for sub-regions with complex terrain and dense buildings, a more complex model such as a deep learning model can be selected.

[0147] In this embodiment, the initial machine learning model set is a set formed by aggregating the initial machine learning models corresponding to the sub-regions of the target region obtained by regional classification. The set includes multiple initial machine learning models for different sub-regions of the target region.

[0148] In this embodiment, the key feature information set is a collection of key image features relevant to intelligent 3D modeling extracted from the comprehensive processed image, sorted and classified. This key feature information accurately describes the shape, structure, texture, material, and other attributes of the target area and is crucial data for building a 3D model.

[0149] In this embodiment, model training is the process of learning each initial machine learning model in the initial machine learning model set using a set of key feature information and a comprehensive processing image portion data as input data.

[0150] In this embodiment, model optimization is the process of improving and refining the model based on the training results of each initial machine learning model. Model optimization may include adjusting the model's hyperparameters, improving the model's structure, and adopting more advanced optimization algorithms.

[0151] In this embodiment, the machine learning model set is an initial machine learning model set after model optimization.

[0152] In this embodiment, the 3D reconstruction model is derived from the optimization results of each initial machine learning model in the machine learning model set and is used to achieve 3D reconstruction of the target area. The 3D reconstruction model is the core of intelligent 3D modeling, transforming key input feature information into the 3D geometry and spatial structure of the target area.

[0153] In this embodiment, the initial three-dimensional image is a three-dimensional image of the target area generated after key feature information is input into the three-dimensional reconstruction model and processed by the model.

[0154] The beneficial effects of the above technology are: by utilizing key feature information and comprehensive image training to optimize the model, the model is made to better fit the characteristics of the target area, and the accuracy of three-dimensional reconstruction is improved. The initial three-dimensional image of the target area finally obtained can effectively improve the efficiency and quality of three-dimensional modeling and reduce the difficulty and cost of modeling.

[0155] Example 6:

[0156] Based on Example 3, an intelligent 3D modeling method based on machine learning inputs key feature information into a 3D reconstruction model to obtain an initial 3D image of the target area, including:

[0157] Randomly extract any frame image from the comprehensive processed image as the reference image of the target area;

[0158] Extracting key feature information corresponding to the current reference image, and based on the type of the key feature information, extracting processed images of frames adjacent to the current reference image from the integrated processed images of the same band to obtain a second reference image set;

[0159] Extracting key feature information of each second reference image in the second reference image set, and combining the key feature information corresponding to the reference image to obtain a first key feature information set of the target area;

[0160] Inputting the key feature information in the first key feature information set into the three-dimensional reconstruction model in combination with the regional parameters of the corresponding sub-region to obtain a three-dimensional skeleton image of the target region in the current sub-region;

[0161] Randomly extract the relative coordinates of any edge point in the benchmark modeling image and compare them with the relative coordinates of the corresponding edge point in the synthetically processed image;

[0162] If the comparison error is less than the real-time modeling accuracy requirement, the 3D skeleton image is used as the reference modeling image;

[0163] Otherwise, a comprehensive processed image of the adjacent bands of the current reference image is extracted as an auxiliary reference image, and key feature information corresponding to the auxiliary reference image is extracted to obtain a set of auxiliary images adjacent to the auxiliary reference image, thereby obtaining an auxiliary feature information set;

[0164] The auxiliary feature information set is combined with the regional parameters of the corresponding sub-region and input into the 3D reconstruction model to obtain a new 3D skeleton image, and compared with it to obtain a reference modeling image;

[0165] performing error compensation on the benchmark modeling image based on the image error influence of the benchmark modeling image, thereby optimizing the benchmark modeling image to obtain a benchmark optimized image;

[0166] The reference optimized image is used as the regional modeling reference image of the current sub-region, and is combined with the key feature information belonging to the current region in the first key feature information set and input into the 3D reconstruction model to obtain the regional 3D modeling image of the current sub-region;

[0167] Performing image stitching on the regional three-dimensional modeling images of each sub-region of the target region, thereby obtaining a three-dimensional stitched image of the target region;

[0168] Based on the real-time modeling accuracy requirement, the amount of stitching feature information is obtained, and based on the stitching feature information, a comprehensive processed image near each stitching area in the three-dimensional stitching image is obtained to obtain a stitching image set;

[0169] Randomly extracting feature information other than the key feature information from the spliced ​​image set, and when the feature information satisfies the splicing feature information amount, using the extracted feature information as a second key feature information set;

[0170] Performing three-dimensional filling on the three-dimensional stitched image based on feature information in the second key feature information set to obtain a three-dimensional optimized image;

[0171] Image fitting and image prediction are performed on each spliced ​​area image in the three-dimensional optimized image to obtain the initial three-dimensional image of the target area.

[0172] In this embodiment, the reference image is a frame of image randomly selected from the comprehensive processed image and serves as the basis for three-dimensional modeling.

[0173] In this embodiment, key feature information refers to representative and distinguishing information in the image, such as edges, corners, textures, etc., which can reflect the important structure and attributes of the target area.

[0174] In this embodiment, the second reference image set is a set consisting of processed images of frames adjacent to the current reference image extracted from the integrated processed images of the same wavelength band.

[0175] In this embodiment, the first key feature information set is a combination of the key feature information of the reference image and each second reference image in the second reference image set. It contains more comprehensive key feature information of the target area within the current sub-area, providing more accurate data support for 3D reconstruction.

[0176] In this embodiment, the region parameters are parameters related to the sub-regions of the target region, such as the size, shape, position, etc. of the sub-regions, reflecting the geometric features and spatial positional relationships of the sub-regions.

[0177] In this embodiment, the 3D reconstruction model is a mathematical model or algorithmic framework used to convert key feature information and regional parameters into a 3D skeleton image. Through computational processing, the 3D reconstruction model can convert 2D image information into a 3D spatial representation, achieving 3D reconstruction of the target area.

[0178] In this embodiment, the reference modeling image is a three-dimensional image obtained after preliminary processing and comparison during the three-dimensional reconstruction process.

[0179] In this embodiment, the comparison error refers to the relative coordinates of any edge point in the reference modeling image, and is compared with the relative coordinates of the corresponding edge point in the integrated processing image to obtain a relative error value T;

[0180]

[0181] Wherein, T is the comparison error, a1, a2, and a3 are the first coordinate value, second coordinate value, and third coordinate value of any edge point of the reference modeling image, b1 is the relative coordinate value of the corresponding integrated processed image, b2 and b3 are the second edge coordinate value and third edge coordinate value of the corresponding edge point in the integrated processed image, δ is the data transmission error, wherein the direction of the first coordinate value is consistent with the relative coordinate value, the direction of the second coordinate value is consistent with the direction of the second edge coordinate value, and the direction of the third coordinate value is consistent with the direction of the third edge coordinate value, α1 is the first error influence weight in the direction of the first coordinate value, α2 is the second error influence weight in the direction of the second coordinate value, and α3 is the third error influence weight in the direction of the third coordinate value, wherein the sum of the weights of the first error influence weight, the second error influence weight, and the third error influence weight is 1.

[0182] In this embodiment, the error influence weight in each direction is related to the rate of change of the current edge point of the reference modeling image in the current direction. The greater the rate of change, the greater the error influence weight, and the smaller the rate of change, the smaller the error influence weight.

[0183] In this embodiment, for example, the directions corresponding to the first coordinate value, the second coordinate value, and the third coordinate value may be the X-axis direction, the Y-axis direction, and the Z-axis direction.

[0184] In this embodiment, the auxiliary reference image is a corresponding reference image extracted from the integrated processed image of adjacent bands of the current reference image when the comparison error of the reference modeling image exceeds the real-time modeling accuracy requirement. The auxiliary reference image is used to provide additional key feature information to help improve the accuracy of 3D reconstruction.

[0185] In this embodiment, the auxiliary image set is a set of adjacent images in the same wavelength band as the auxiliary reference image. The images in the auxiliary image set are spatially correlated with the auxiliary reference image, which helps to extract key feature information related to the auxiliary reference image.

[0186] In this embodiment, the auxiliary feature information set is a set obtained by combining the auxiliary reference image and key feature information of each auxiliary image in the auxiliary image set.

[0187] In this embodiment, the regional 3D modeling image refers to the regional image obtained by using the baseline optimized image as the regional modeling baseline image for the current subregion and combining it with the key feature information of the current region in the first key feature information set and inputting it into the 3D reconstruction model. This image reflects the 3D structure and morphology of the target region within the current subregion.

[0188] In this embodiment, the 3D stitched image is an image obtained by stitching the 3D modeling images of each sub-region of the target region, integrating the 3D information of each sub-region of the target region to form a complete 3D representation of the target region.

[0189] In this embodiment, the amount of stitching feature information is the amount of feature information that needs to be extracted from the 3D stitched image, determined based on the real-time modeling accuracy requirements. This represents the richness of feature information that needs to be extracted from the 3D stitched image to meet the real-time modeling accuracy requirements. For example, the amount of stitching feature information can be 100.

[0190] In this embodiment, the stitched image set is a set consisting of comprehensive processed images near each stitched area in the three-dimensional stitched image.

[0191] In this embodiment, the second key feature information set is a set of randomly extracted feature information from the stitched image set, excluding the key feature information, obtained when the feature information satisfies the stitching feature information requirement. This set includes feature information used for 3D filling, helping to improve the integrity and accuracy of the 3D stitching edge.

[0192] In this embodiment, the 3D optimized image refers to an image obtained by performing 3D filling on the 3D stitched image based on the feature information in the second key feature information set, and is an image that further optimizes and improves the 3D stitched image.

[0193] In this embodiment, the initial three-dimensional image is an image obtained by performing image fitting and image prediction on each splicing region image in the three-dimensional optimized image.

[0194] The beneficial effects of the above technology are: by utilizing key feature information and comprehensive image training to optimize the model, the model is made to better fit the characteristics of the target area, the accuracy of three-dimensional reconstruction is improved, and finally the initial three-dimensional image is obtained, which can effectively improve the efficiency and quality of three-dimensional modeling and reduce the difficulty and cost of modeling.

[0195] Example 7:

[0196] Based on Example 6, an intelligent 3D modeling method based on machine learning, S4: optimizing the 3D reconstruction model in combination with preset constraints, thereby optimizing the initial 3D image based on the optimized model to obtain a 3D modeled image, including:

[0197] Based on the real-time application scenario, the preset constraints of the 3D processing model are optimized, and then the 3D processing model is constrained based on the constraints after the optimization, and the model is optimized in combination with the model optimization algorithm to obtain an optimized 3D model;

[0198] Perform intelligent 3D modeling of the target area based on the optimized 3D model to obtain a 3D optimized image of the target area;

[0199] The three-dimensional optimized image and the initial three-dimensional image are input into the same three-dimensional coordinate system to perform image comparison;

[0200] If the image error between the 3D optimized image and the initial 3D image is less than the real-time modeling accuracy requirement, the initial 3D image is used as the 3D modeling image of the target area;

[0201] Otherwise, the initial three-dimensional image is optimized based on the three-dimensional optimized image to obtain a three-dimensional modeling image of the target area.

[0202] In this embodiment, the preset constraint condition is a constraint condition followed when optimizing the 3D reconstruction model, and the preset constraint condition may be a physical constraint, a geometric constraint, or an application constraint.

[0203] In this embodiment, the real-time application scenario refers to a real-time application scenario when the three-dimensional processing model performs three-dimensional modeling. For example, the real-time application scenario includes an application field, an application area, and the like.

[0204] In this embodiment, model optimization is the process of adjusting and improving the 3D reconstruction model based on preset constraints. For example, during the model optimization process, various optimization algorithms (such as gradient descent algorithms and genetic algorithms) are used to adjust the model parameters so that the model achieves better performance and accuracy while satisfying the constraints.

[0205] In this embodiment, model constraint is the process of applying preset constraint conditions to the 3D processing model. Through model constraint, the behavior of the model during the generation or optimization process can be restricted so that the output result meets the preset requirements.

[0206] In this embodiment, the model optimization algorithm is an algorithm used to improve and enhance the three-dimensional processing model. These algorithms adjust the model's parameters, structure, or training process to make the model perform better on given evaluation indicators (such as accuracy, efficiency, generalization ability, etc.).

[0207] In this embodiment, the optimized three-dimensional model is a three-dimensional processed model obtained after being processed by model constraints and model optimization algorithms.

[0208] In this embodiment, the three-dimensional optimized image is an image obtained by performing intelligent three-dimensional modeling on the target area based on the optimized three-dimensional model.

[0209] In this embodiment, the image error is the difference between the 3D optimized image and the initial 3D image when the two are compared in the same 3D coordinate system. The image error can be measured by various indicators, such as mean square error (MSE).

[0210] In this embodiment, the real-time modeling accuracy requirement refers to the accuracy requirement for the three-dimensional modeling results of the target area in a real-time application scenario. The real-time modeling accuracy requirement is usually determined based on the specific application scenario and task objectives.

[0211] In this embodiment, the three-dimensional modeling image is a three-dimensional image of the target area obtained by optimizing the initial three-dimensional image based on the three-dimensional optimization image.

[0212] The beneficial effects of the above technology are: by constraining the three-dimensional processing model through preset constraints and combining it with optimization algorithms for optimization, the model accuracy and stability can be improved, and a more realistic optimized three-dimensional model can be obtained, thereby obtaining a three-dimensional optimized image and comparing it with the initial three-dimensional image. The optimization performance is judged in combination with the real-time modeling accuracy requirements. On the one hand, the modeling quality is guaranteed, and the waste of resources caused by excessive optimization is avoided, effectively improving the efficiency and accuracy of three-dimensional modeling.

[0213] Example 8:

[0214] Based on Example 7, a machine learning-based intelligent 3D modeling method performs constraint optimization on preset constraints of a 3D processing model based on a real-time application scenario, thereby constraining the 3D processing model based on the constraint optimization, and optimizing the model in combination with a model optimization algorithm to obtain an optimized 3D model, including:

[0215] Obtain key scene information related to 3D modeling in real-time application scenarios;

[0216] Dynamically adjust the constraint parameters of the preset constraint conditions of the three-dimensional processing model according to the key scene information to obtain the constraint optimization parameters;

[0217] The constrained optimization parameters are combined with the parameters that are not constrained to be optimized in the preset constraint conditions to obtain the constrained optimization conditions, and the three-dimensional processing model is constrained based on the constrained optimization conditions;

[0218] The constrained three-dimensional processing model is optimized based on a preset optimization algorithm to obtain an optimized three-dimensional model.

[0219] In this embodiment, the constraint optimization condition is to merge the dynamically adjusted constraint optimization parameters with the unadjusted preset constraint condition parameters to form a complete constraint set. For example, if the preset constraint condition includes "the model must be a closed grid", and "the model surface roughness must be less than 0.1" is added after dynamic adjustment, the integrated constraint condition is a combination of the two.

[0220] In this embodiment, the model constraint is to embed the synthesized constraint conditions into the three-dimensional processing model in a mathematical form.

[0221] In this embodiment, the preset optimization algorithm is an algorithm for optimizing a three-dimensional model, such as a gradient descent method, a genetic algorithm, reinforcement learning, and the like.

[0222] The beneficial effects of the above technology are: by constraining the three-dimensional processing model through preset constraints and combining it with optimization algorithms for optimization, the model accuracy and stability can be improved, thereby making the optimization of the initial three-dimensional image more accurate and efficient, effectively improving the efficiency and accuracy of three-dimensional modeling.

[0223] Example 9:

[0224] The present invention provides an intelligent three-dimensional modeling system based on machine learning, which is used to execute the intelligent three-dimensional modeling method based on machine learning described in any one of embodiments 1 to 8, with reference to Figure 2 ,include:

[0225] The acquisition and processing module is used to acquire multi-view and multi-spectral data of the target area based on the preset sensor to obtain a comprehensive image of the target area, and perform image processing to obtain a comprehensive processed image;

[0226] Feature extraction module: used to extract key image features from the comprehensive processed image based on the requirements of intelligent 3D modeling and combine feature extraction solutions to obtain key feature information;

[0227] Model Optimization Module: This module is used to classify regions based on their characteristics, determine the initial machine learning model for each sub-region based on the requirements of intelligent 3D modeling, optimize and integrate the models, obtain a 3D reconstruction model of the target region, and generate an initial 3D image based on key feature information.

[0228] Image optimization module: used to optimize the 3D reconstruction model in combination with preset constraints, thereby optimizing the initial 3D image based on the optimized model to obtain a 3D modeling image.

[0229] The beneficial effects of the above technologies are: through multi-view, multi-spectral data acquisition and processing, comprehensive and accurate integrated processing images are obtained, and key feature information is accurately extracted according to intelligent modeling requirements and feature extraction solutions, making feature extraction more accurate and efficient. In combination with regional characteristic classification and modeling requirements, the initial machine learning model is determined and optimized to generate an initial three-dimensional image adapted to the target area; and preset constraints are used to perform dual optimization of the model and image, effectively improving the quality of three-dimensional modeling images, meeting diverse application needs, and improving real-time modeling performance.

[0230] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. An intelligent 3D modeling method based on machine learning, characterized in that: include: S1: Based on the preset sensor, multi-view and multi-spectral data of the target area are collected to obtain a comprehensive image of the target area, and image processing is performed to obtain a comprehensive processed image; S2: Based on the requirements of intelligent 3D modeling, key image features are extracted from the comprehensive processed images in combination with feature extraction schemes to obtain key feature information; S3: Classify the target area based on its regional characteristics, determine the initial machine learning model for each sub-area based on the requirements of intelligent 3D modeling, optimize and integrate the models, obtain a 3D reconstruction model of the target area, and generate an initial 3D image based on key feature information. S4: optimizing the 3D reconstruction model in combination with preset constraints, thereby optimizing the initial 3D image based on the optimized model to obtain a 3D modeling image; Among them, S2: Based on the requirements of intelligent 3D modeling, combined with the feature extraction solution, key image features are extracted from the comprehensive processed image to obtain key feature information, including: Based on the requirements of intelligent 3D modeling, a feature extraction solution that matches the target area is selected from the preset feature extraction database; Based on each feature extraction algorithm in the feature extraction scheme, key image features corresponding to intelligent 3D modeling are extracted from the comprehensive processed image, and the features are sorted and classified to obtain a key feature information set; S3: Classify the target area based on its regional characteristics, determine the initial machine learning model for each sub-area based on the requirements of intelligent 3D modeling, optimize and integrate the models, obtain a 3D reconstruction model of the target area, and generate an initial 3D image based on key feature information, including: Based on the regional characteristics of the target area, the region is classified. Based on the regional classification results and the requirements of intelligent 3D modeling, an initial machine learning model that matches each sub-region is screened from the preset model database to obtain an initial machine learning model set for the target area. Performing model training on each initial machine learning model in the initial machine learning model set based on the key feature information set and the comprehensive processed image, and performing corresponding model optimization based on each training result; Obtaining a three-dimensional reconstruction model of the target area based on the model optimization results of each initial machine learning model in the machine learning model set; Input key feature information into the 3D reconstruction model to obtain an initial 3D image of the target area; Inputting the key feature information into the three-dimensional reconstruction model to obtain an initial three-dimensional image of the target area includes: Randomly extract any frame image from the comprehensive processed image as the reference image of the target area; Extracting key feature information corresponding to the current reference image, and based on the type of the key feature information, extracting processed images of frames adjacent to the current reference image from the integrated processed images of the same band to obtain a second reference image set; Extracting key feature information of each second reference image in the second reference image set, and combining the key feature information corresponding to the reference image to obtain a first key feature information set of the target area; Inputting the key feature information in the first key feature information set into the three-dimensional reconstruction model in combination with the regional parameters of the corresponding sub-region to obtain a three-dimensional skeleton image of the target region in the current sub-region; Randomly extract the relative coordinates of any edge point in the benchmark modeling image and compare them with the relative coordinates of the corresponding edge point in the synthetically processed image; If the comparison error is less than the real-time modeling accuracy requirement, the 3D skeleton image is used as the reference modeling image; Otherwise, a comprehensive processed image of the adjacent bands of the current reference image is extracted as an auxiliary reference image, and key feature information corresponding to the auxiliary reference image is extracted to obtain a set of auxiliary images adjacent to the auxiliary reference image, thereby obtaining an auxiliary feature information set; The auxiliary feature information set is combined with the regional parameters of the corresponding sub-region and input into the 3D reconstruction model to obtain a new 3D skeleton image, and compared with it to obtain a reference modeling image; performing error compensation on the benchmark modeling image based on the image error influence of the benchmark modeling image, thereby optimizing the benchmark modeling image to obtain a benchmark optimized image; The reference optimized image is used as the regional modeling reference image of the current sub-region, and is combined with the key feature information belonging to the current region in the first key feature information set and input into the 3D reconstruction model to obtain the regional 3D modeling image of the current sub-region; Performing image stitching on the regional three-dimensional modeling images of each sub-region of the target region, thereby obtaining a three-dimensional stitched image of the target region; Based on the real-time modeling accuracy requirement, the amount of stitching feature information is obtained, and based on the stitching feature information, a comprehensive processed image near each stitching area in the three-dimensional stitching image is obtained to obtain a stitching image set; Randomly extracting feature information other than the key feature information from the spliced ​​image set, and when the feature information satisfies the splicing feature information amount, using the extracted feature information as a second key feature information set; Performing three-dimensional filling on the three-dimensional stitched image based on feature information in the second key feature information set to obtain a three-dimensional optimized image; Perform image fitting and image prediction on each stitching area image in the three-dimensional optimized image to obtain an initial three-dimensional image of the target area; S4: optimizing the three-dimensional reconstruction model in combination with preset constraints, thereby optimizing the initial three-dimensional image based on the optimized model to obtain a three-dimensional modeling image, including: Based on the real-time application scenario, the preset constraints of the 3D processing model are optimized, and then the 3D processing model is constrained based on the constraints after the optimization, and the model is optimized in combination with the model optimization algorithm to obtain an optimized 3D model; Perform intelligent 3D modeling of the target area based on the optimized 3D model to obtain a 3D optimized image of the target area; The three-dimensional optimized image and the initial three-dimensional image are input into the same three-dimensional coordinate system to perform image comparison; If the image error between the 3D optimized image and the initial 3D image is less than the real-time modeling accuracy requirement, the initial 3D image is used as the 3D modeling image of the target area; Otherwise, the initial three-dimensional image is optimized based on the three-dimensional optimized image to obtain a three-dimensional modeling image of the target area.

2. The intelligent three-dimensional modeling method based on machine learning according to claim 1, characterized in that: S1: Based on the preset sensor, multi-view and multi-spectral data of the target area are collected to obtain a comprehensive image of the target area, and image processing is performed to obtain a comprehensive processed image, including: Based on the regional characteristics of the target area and the requirements of intelligent 3D modeling, the sensor for collecting data in the target area is determined as the preset sensor; The target device equipped with a preset sensor collects data of the target area from different perspectives to obtain a first image of the target area in a multi-perspective state; Acquire spectral data of different bands based on a preset camera, and obtain multi-band spectral data as a second image; The first image and the second image are integrated to obtain a comprehensive image of the target area, and the comprehensive image is processed to obtain a comprehensive processed image.

3. The intelligent three-dimensional modeling method based on machine learning according to claim 1, characterized in that: Based on the requirements of intelligent 3D modeling, feature extraction solutions that match the target area are selected from the preset feature extraction database, including: Determine the 3D modeling requirements for intelligent 3D modeling based on real-time application scenarios, and determine the initial feature extraction plan based on the 3D modeling requirements; Based on the initial feature extraction scheme, the feature types related to intelligent 3D modeling in the image are analyzed and processed comprehensively, and used as the initial feature types; A comprehensive feature extraction method for intelligent 3D modeling by comprehensively determining the initial feature extraction scheme and the initial feature type; Based on the feature extraction level requirements and feature fusion modality requirements, the comprehensive feature extraction method is optimized to obtain the optimized feature extraction method; Based on the initial feature type, the corresponding type of image information is randomly extracted from the comprehensive processed image to verify the optimized feature extraction method, and the qualified optimized feature extraction method is used as the feature extraction scheme of the target area.

4. The intelligent three-dimensional modeling method based on machine learning according to claim 1, characterized in that: Based on the real-time application scenario, the preset constraints of the 3D processing model are optimized, and then the 3D processing model is constrained based on the constraints after the optimization. The model is optimized in combination with the model optimization algorithm to obtain an optimized 3D model, including: Obtain key scene information related to 3D modeling in real-time application scenarios; Dynamically adjust the constraint parameters of the preset constraint conditions of the three-dimensional processing model according to the key scene information to obtain the constraint optimization parameters; The constrained optimization parameters are combined with the parameters that are not constrained to be optimized in the preset constraint conditions to obtain the constrained optimization conditions, and the three-dimensional processing model is constrained based on the constrained optimization conditions; The constrained three-dimensional processing model is optimized based on a preset optimization algorithm to obtain an optimized three-dimensional model.

5. An intelligent 3D modeling system based on machine learning, characterized in that: Used to execute the intelligent three-dimensional modeling method based on machine learning as described in any one of claims 1 to 4, comprising: The acquisition and processing module is used to acquire multi-view and multi-spectral data of the target area based on the preset sensor to obtain a comprehensive image of the target area, and perform image processing to obtain a comprehensive processed image; Feature extraction module: used to extract key image features from the comprehensive processed image based on the requirements of intelligent 3D modeling and combine feature extraction solutions to obtain key feature information; Model Optimization Module: This module is used to classify regions based on their characteristics, determine the initial machine learning model for each sub-region based on the requirements of intelligent 3D modeling, optimize and integrate the models, obtain a 3D reconstruction model of the target region, and generate an initial 3D image based on key feature information. Image optimization module: used to optimize the 3D reconstruction model in combination with preset constraints, thereby optimizing the initial 3D image based on the optimized model to obtain a 3D modeling image.

Images