Intelligent three-dimensional modeling method and system based on machine learning

Abstract

The invention relates to the technical field of intelligent modeling, and particularly discloses an intelligent three-dimensional modeling method and system based on machine learning, and the method comprises the steps: S1, carrying out the data collection of a target region based on a preset sensor, obtaining a comprehensive image of the target region, and carrying out the image processing, and obtaining a comprehensive processing image; s2, combining an intelligent three-dimensional modeling demand with a feature extraction scheme, and extracting key image features from the comprehensive processing image to obtain key feature information; s3, carrying out region classification based on the region characteristics of the target region, obtaining a target region three-dimensional reconstruction model in combination with an intelligent three-dimensional modeling demand, and generating an initial three-dimensional image in combination with the key characteristic information; s4, performing model optimization on the three-dimensional reconstruction model in combination with a preset constraint condition, thereby performing image optimization on the initial three-dimensional image based on the optimization model, and obtaining a three-dimensional modeling image; the method is used for achieving high-precision three-dimensional model reconstruction in combination with a machine learning model, the modeling efficiency is improved, and the real-time performance of three-dimensional modeling is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent modeling, and particularly relates to an intelligent three-dimensional modeling method and system based on machine learning. Background Art

[0002] At present, with the rapid development of computer vision and machine learning technologies, intelligent three-dimensional modeling has shown great application potential in many fields such as urban planning, virtual reality, cultural heritage protection, etc. Especially in the field of industrial development, intelligent three-dimensional modeling is particularly common.

[0003] There are often a large amount of data in the industrial field. However, existing three-dimensional modeling methods often face problems such as insufficient accuracy, low efficiency, and low automation when dealing with complex scenes and large-scale data. The generated three-dimensional modeling images also have problems such as insufficient image accuracy and inability to well reflect the characteristics of the target object.

[0004] Therefore, the present invention proposes an intelligent three-dimensional modeling method and system based on machine learning. Summary of the Invention

[0005] The present invention provides an intelligent three-dimensional modeling method and system based on machine learning, which is used to obtain a comprehensive and accurate integrated processed image through multi-view and multi-spectral data acquisition and processing, accurately extract key feature information according to the intelligent modeling requirements and feature extraction scheme, make the feature extraction more accurate and efficient, and combine the regional characteristics classification and modeling requirements to determine and optimize the initial machine learning model, generate an initial three-dimensional image adapted to the target area; and use preset constraint conditions to double-optimize the model and the image, effectively improve the quality of the three-dimensional modeling image, meet diverse application requirements, and improve the real-time modeling performance.

[0006] The present invention provides an intelligent three-dimensional modeling method based on machine learning, including:

[0007] S1: Based on a preset sensor, perform multi-view and multi-spectral data acquisition on the target area to obtain a comprehensive image of the target area, and perform image processing to obtain an integrated processed image;

[0008] S2: Based on the intelligent three-dimensional modeling requirements, combine the feature extraction scheme to extract key image features from the integrated processed image to obtain key feature information;

[0009] S3: Perform regional classification based on the regional characteristics of the target area, and combine the intelligent three-dimensional modeling requirements to determine the initial machine learning model for each sub-region, and perform model optimization and integration to obtain a three-dimensional reconstruction model of the target area, and generate an initial three-dimensional image in combination with the key feature information;

[0010] S4: Optimize the 3D reconstruction model in combination with preset constraint conditions, and then optimize the initial 3D image based on the optimized model to obtain a 3D modeling image.

[0011] Preferably, S1: Collect multi-view and multi-spectral data of the target area based on a preset sensor to obtain a comprehensive image of the target area, and perform image processing on it to obtain a comprehensively processed image, including:

[0012] Determine the sensor for data collection of the target area as the preset sensor based on the regional characteristics of the target area and the intelligent 3D modeling requirements;

[0013] Collect data of the target area from different perspectives based on the target device equipped with the preset sensor to obtain the first image of the target area in a multi-view state;

[0014] Obtain multi-band spectral data as the second image based on the spectral data of different bands acquired by the preset camera;

[0015] Integrate the first image and the second image to obtain a comprehensive image of the target area, and perform image processing on the comprehensive image to obtain a comprehensively processed image.

[0016] Preferably, S2: Extract key image features from the comprehensively processed image in combination with a feature extraction scheme based on the intelligent 3D modeling requirements to obtain key feature information, including:

[0017] Screen the feature extraction scheme that matches the target area from the preset feature extraction database based on the intelligent 3D modeling requirements;

[0018] Extract the key image features corresponding to the intelligent 3D modeling from the comprehensively processed image based on each feature extraction algorithm in the feature extraction scheme, and perform feature sorting and classification to obtain a set of key feature information.

[0019] Preferably, screening the feature extraction scheme that matches the target area from the preset feature extraction database based on the intelligent 3D modeling requirements includes:

[0020] Determine the 3D modeling requirements for intelligent 3D modeling based on the real-time application scenario, and determine the initial feature extraction scheme based on the 3D modeling requirements;

[0021] Analyze the feature types related to intelligent 3D modeling in the comprehensively processed image based on the initial feature extraction scheme as the initial feature types;

[0022] Comprehensively determine the comprehensive feature extraction method for intelligent 3D modeling by combining the initial feature extraction scheme and the initial feature types;

[0023] Optimize the comprehensive feature extraction method based on the requirements of the feature extraction hierarchy and the feature fusion mode to obtain an optimized feature extraction method;

[0024] Verify the optimized feature extraction method by randomly extracting image information of the corresponding type from the comprehensively processed image based on the initial feature type, and use the verified optimized feature extraction method as the feature extraction scheme for the target area.

[0025] Preferably, S3: Classify the region based on the region characteristics of the target area, determine the initial machine learning model for each sub-region in combination with the intelligent 3D modeling requirements, and perform model optimization and integration to obtain the 3D reconstruction model of the target area, and generate an initial 3D image in combination with the key feature information, including:

[0026] Classify the region based on the region characteristics of the target area, and thus screen the initial machine learning models matching each sub-region from the preset model database based on the region classification results in combination with the intelligent 3D modeling requirements to obtain the initial machine learning model set of the target area;

[0027] Train each initial machine learning model in the initial machine learning model set based on the key feature information set and the comprehensively processed image, and perform corresponding model optimization based on each training result;

[0028] Obtain the 3D reconstruction model of the target area based on the model optimization results of each initial machine learning model in the machine learning model set;

[0029] Input the key feature information into the 3D reconstruction model to obtain the initial 3D image of the target area.

[0030] Preferably, input the key feature information into the 3D reconstruction model to obtain the initial 3D image of the target area, including:

[0031] Randomly extract any frame of image from the comprehensively processed image as the reference image of the target area;

[0032] Extract the key feature information corresponding to the current reference image, and based on the type of the key feature information, extract the processed images adjacent to the current reference image from the comprehensively processed images of the same band to obtain the second reference image set;

[0033] Extract the key feature information of each second reference image in the second reference image set, and combine it with the key feature information corresponding to the reference image to obtain the first key feature information set of the target area;

[0034] Input the key feature information in the first key feature information set and the region parameters of the corresponding sub-region into the 3D reconstruction model to obtain the 3D skeleton image of the target area in the current sub-region;

[0035] Randomly extract the relative coordinates of any edge point in the reference modeling image, and compare them with the relative coordinates of the corresponding edge point in the comprehensively processed image;

[0036] If the comparison error is less than the real-time modeling accuracy requirement, then use the three-dimensional skeleton image as the reference modeling image;

[0037] Otherwise, extract the comprehensively processed image of the adjacent band of the current reference image as the auxiliary reference image, and extract the key feature information corresponding to the auxiliary reference image to obtain an auxiliary image set adjacent to the auxiliary reference image, thereby obtaining an auxiliary feature information set;

[0038] Input the auxiliary feature information set and the region parameters of the corresponding sub-region into the three-dimensional reconstruction model to obtain a new three-dimensional skeleton image, and compare to obtain the reference modeling image;

[0039] Perform error compensation on the reference modeling image based on the image error influence of the reference modeling image, thereby optimizing the reference modeling image to obtain a reference optimized image;

[0040] Use the reference optimized image as the region modeling reference image of the current sub-region, and input the key feature information belonging to the current region in the first key feature information set into the three-dimensional reconstruction model to obtain the region three-dimensional modeling image of the current sub-region;

[0041] Stitch the region three-dimensional modeling images of each sub-region of the target region to obtain the three-dimensional stitched image of the target region;

[0042] Obtain the stitching feature information amount based on the real-time modeling accuracy requirement, and obtain the comprehensively processed image near each stitching region in the three-dimensional stitched image based on the stitching feature information amount to obtain a stitched image set;

[0043] Randomly extract the feature information except the key feature information in the stitched image set. When the feature information meets the stitching feature information amount, use the extracted feature information as the second key feature information set;

[0044] Perform three-dimensional filling on the three-dimensional stitched image based on the feature information in the second key feature information set to obtain a three-dimensional optimized image;

[0045] Perform image fitting and image prediction on each stitching region image in the three-dimensional optimized image to obtain the initial three-dimensional image of the target region.

[0046] Preferably, S4: Combine preset constraint conditions to optimize the three-dimensional reconstruction model, thereby performing image optimization on the initial three-dimensional image based on the optimized model to obtain a three-dimensional modeling image, including:

[0047] Optimize the preset constraint conditions of the 3D processing model based on the real-time application scenario, thereby performing model constraint on the 3D processing model based on the optimized constraint conditions, and combining with the model optimization algorithm to optimize the model, obtaining an optimized 3D model;

[0048] Perform intelligent 3D modeling on the target area based on the optimized 3D model, obtaining a 3D optimized image of the target area;

[0049] Input the 3D optimized image and the initial 3D image into the same 3D coordinate system, thereby performing image comparison;

[0050] If the image error between the 3D optimized image and the initial 3D image is less than the real-time modeling accuracy requirement, then use the initial 3D image as the 3D modeling image of the target area;

[0051] Otherwise, perform image optimization on the initial 3D image based on the 3D optimized image, obtaining the 3D modeling image of the target area.

[0052] Preferably, optimizing the preset constraint conditions of the 3D processing model based on the real-time application scenario, thereby performing model constraint on the 3D processing model based on the optimized constraint conditions, and combining with the model optimization algorithm to optimize the model, obtaining an optimized 3D model, including:

[0053] Obtain the key scenario information related to 3D modeling in the real-time application scenario;

[0054] Dynamically adjust the constraint parameters of the preset constraint conditions of the 3D processing model according to the key scenario information, obtaining optimized constraint parameters;

[0055] Integrate the optimized constraint parameters with the parameters in the preset constraint conditions that have not been optimized to obtain optimized constraint conditions, and perform constraint on the 3D processing model based on the optimized constraint conditions;

[0056] Perform model optimization on the constrained 3D processing model based on the preset optimization algorithm, obtaining an optimized 3D model.

[0057] The present invention provides an intelligent 3D modeling system based on machine learning, which is used to execute the intelligent 3D modeling method based on machine learning described in any one of Embodiments 1 to 8, including:

[0058] An acquisition and processing module, which is used to collect multi-view and multi-spectral data of the target area based on a preset sensor, obtain a comprehensive image of the target area, and perform image processing to obtain a comprehensively processed image;

[0059] A feature extraction module: which is used to extract key image features from the comprehensively processed image based on the intelligent 3D modeling requirements and in combination with the feature extraction scheme, obtaining key feature information;

[0060] Model optimization module: used to classify regions based on the regional characteristics of the target region, determine the initial machine learning model for each sub-region in combination with the intelligent 3D modeling requirements, and perform model optimization and integration to obtain the 3D reconstruction model of the target region, and generate an initial 3D image in combination with key feature information;

[0061] Image optimization module: used to optimize the 3D reconstruction model in combination with preset constraint conditions, so as to optimize the initial 3D image based on the optimized model to obtain a 3D modeling image.

[0062] The beneficial effects of the present invention compared with the prior art are as follows: through multi-view and multi-spectral data acquisition and processing, a comprehensive and accurate comprehensive processed image is obtained. According to the intelligent modeling requirements and feature extraction schemes, key feature information is accurately extracted, making the feature extraction more accurate and efficient. In combination with regional characteristics classification and modeling requirements, the initial machine learning model is determined and optimized to generate an initial 3D image adapted to the target region; and the model and image are double-optimized using preset constraint conditions, effectively improving the quality of the 3D modeling image, meeting diverse application requirements, and improving real-time modeling performance.

[0063] Other features and advantages of the present invention will be described in the following description, and part of them will become obvious from the description, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0064] The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention. In the accompanying drawings:

[0066] Figure 1 It is a schematic diagram of an intelligent 3D modeling method based on machine learning in an embodiment of the present invention;

[0067] Figure 2 It is a structural diagram of an intelligent 3D modeling system based on machine learning in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0068] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0069] Embodiment 1:

[0070] The present invention provides an intelligent three-dimensional modeling method based on machine learning, with reference to Figure 1 , including:

[0071] S1: Based on a preset sensor, multi-view and multi-spectral data of the target area are collected to obtain a comprehensive image of the target area, and image processing is performed to obtain a comprehensively processed image;

[0072] S2: Based on the intelligent three-dimensional modeling requirements, key image features are extracted from the comprehensively processed image in combination with a feature extraction scheme to obtain key feature information;

[0073] S3: Region classification is performed based on the regional characteristics of the target area, and the initial machine learning model of each sub-region is determined in combination with the intelligent three-dimensional modeling requirements, and model optimization and integration are performed to obtain a three-dimensional reconstruction model of the target area, and an initial three-dimensional image is generated in combination with the key feature information;

[0074] S4: The three-dimensional reconstruction model is optimized in combination with preset constraint conditions, so that the initial three-dimensional image is optimized based on the optimized model to obtain a three-dimensional modeling image.

[0075] In this embodiment, the preset sensor refers to a device for data collection that is pre-selected and set according to specific requirements and goals before starting the data collection work. For example, the sensor may include an optical camera, a multi-spectral camera, a lidar, etc.

[0076] In this embodiment, multi-view data collection is to obtain data of the target area from different angles and orientations. By changing the position, direction or attitude of the sensor, images or data of the target area from different perspectives can be obtained.

[0077] In this embodiment, multi-spectral data collection is to use a sensor that can sense different spectral bands to collect data of the target area in multiple spectral bands at the same time.

[0078] In this embodiment, the comprehensive image is an image obtained by fusing various data obtained through multi-view and multi-spectral data collection.

[0079] In this embodiment, image processing is to perform a series of operations and transformations on the collected comprehensive image to improve the image quality, extract useful information, or prepare for subsequent analysis and processing. Common image processing operations include image enhancement, image filtering, image segmentation, etc.

[0080] In this embodiment, the comprehensively processed image is based on the comprehensive image, and by applying various image processing technologies, noise is removed, image features are enhanced, useful information is extracted, etc., so that the image is more suitable for subsequent feature extraction and three-dimensional modeling operations.

[0081] In this embodiment, the intelligent 3D modeling requirements refer to the requirements put forward according to specific application scenarios and objectives during the 3D modeling process. They may include requirements for modeling accuracy, modeling speed, modeling content, and the degree of automation of modeling, etc.

[0082] In this embodiment, the feature extraction scheme is a method and strategy for extracting key image features from the comprehensively processed image. For example, the feature extraction scheme includes selecting a suitable feature extraction algorithm, determining the parameters of feature extraction, and the process of feature extraction.

[0083] In this embodiment, the key image features are the information extracted from the comprehensively processed image that can reflect the important characteristics of the target area. The key image features can be edges, corners, textures, color distributions, etc. in the image. For example, in the 3D modeling of a building, the edges and corners of the building can be used as key image features to determine the shape and structure of the building.

[0084] In this embodiment, the regional characteristics refer to the unique properties of the target area in terms of geography, environment, function, etc.

[0085] In this embodiment, the regional classification is to divide the target area into different sub-areas according to the regional characteristics of the target area. For example, a target area can be divided into static areas, dynamic areas, complex areas, simple areas, etc. The purpose of regional classification is to adopt more appropriate 3D modeling methods and technologies for different sub-areas to improve the accuracy and efficiency of 3D modeling.

[0086] In this embodiment, the initial machine learning model is a machine learning model pre-selected for 3D reconstruction according to the regional characteristics of each sub-area and the intelligent 3D modeling requirements. The machine learning model is selected from a preset model database. For example, for a relatively flat sub-area of the terrain, a rule-based modeling method or a simple machine learning model can be selected; for a sub-area with complex terrain and dense buildings, a more complex model such as a deep learning model can be selected.

[0087] In this embodiment, the model optimization and integration is a process of optimizing and integrating the initial machine learning model of each sub-area. Model integration is to fuse the optimized models of each sub-area to form a 3D reconstruction model that can cover the entire target area.

[0088] In this embodiment, the 3D reconstruction model is obtained through model optimization and integration, and it is a model that can represent the 3D structure and features of the target area. For example, the 3D reconstruction model can be a digital elevation model (DEM), a 3D point cloud model, a 3D mesh model, or a 3D solid model, etc.

[0089] In this embodiment, the initial three-dimensional image is a three-dimensional image generated by inputting the hanging part feature information into the three-dimensional reconstruction model of the target area.

[0090] In this embodiment, the preset constraint conditions are the limiting conditions followed when optimizing the three-dimensional reconstruction model. The preset constraint conditions can be physical constraints, geometric constraints, or application constraints.

[0091] In this embodiment, model optimization is a process of adjusting and improving the three-dimensional reconstruction model according to the preset constraint conditions. For example, in the process of model optimization, various optimization algorithms (such as gradient descent algorithm, genetic algorithm, etc.) are used to adjust the parameters of the model, so that the model can achieve better performance and accuracy on the premise of meeting the constraint conditions.

[0092] In this embodiment, image optimization is a process of improving the initial three-dimensional image based on the optimized three-dimensional reconstruction model. By performing operations such as denoising, smoothing, and texture mapping on the initial three-dimensional image, the quality and realism of the image can be improved, making it more in line with the actual target area characteristics.

[0093] The beneficial effects of the above technologies are as follows: Through multi-view and multi-spectral data acquisition and processing, a comprehensive and accurate comprehensive processed image is obtained. According to the intelligent modeling requirements and feature extraction schemes, key feature information is accurately extracted, making feature extraction more accurate and efficient. Combining with the regional characteristics classification and modeling requirements, the initial machine learning model is determined and optimized to generate an initial three-dimensional image adapted to the target area; and the model and image are double-optimized using the preset constraint conditions, effectively improving the quality of the three-dimensional modeling image, meeting diverse application requirements, and improving the real-time modeling performance.

[0094] Embodiment 2:

[0095] Based on the intelligent three-dimensional modeling method based on machine learning in Embodiment 1, S1: Perform multi-view and multi-spectral data acquisition on the target area based on a preset sensor to obtain a comprehensive image of the target area, and perform image processing to obtain a comprehensive processed image, including:

[0096] Based on the regional characteristics of the target area and the intelligent three-dimensional modeling requirements, determine the sensor for data acquisition of the target area as the preset sensor;

[0097] Based on the target device equipped with the preset sensor, perform data acquisition on the target area from different perspectives to obtain the first image of the target area in a multi-view state;

[0098] Based on the preset camera, obtain spectral data of different bands to obtain multi-band spectral data as the second image;

[0099] Integrate the first image and the second image to obtain a comprehensive image of the target area, and perform image processing on the comprehensive image to obtain a comprehensively processed image.

[0100] In this embodiment, the regional characteristics refer to the unique properties of the target area in terms of geography, environment, function, etc.

[0101] In this embodiment, the intelligent 3D modeling requirements refer to the requirements put forward according to specific application scenarios and objectives during the 3D modeling process. It may include requirements for modeling accuracy, modeling speed, modeling content, and the degree of automation of modeling, etc.

[0102] In this embodiment, the preset sensor refers to the device selected and set in advance according to specific requirements and objectives for data collection before starting the data collection work. For example, the sensor may include an optical camera, a multispectral camera, a lidar, etc.

[0103] In this embodiment, the target device is a carrier equipped with a preset sensor for data collection of the target area. It can be a drone, a manned aircraft, a ground mobile vehicle, a fixed measurement platform, etc. Selecting a suitable target device requires considering factors such as the weight, size, and power consumption of the sensor, as well as the terrain and environment of the target area. For example, for a large-area target area, a drone may be a better choice because of its strong mobility and wide coverage.

[0104] In this embodiment, multi-view data collection is to obtain data of the target area from different angles and orientations. By changing the position, direction, or attitude of the sensor, images or data of the target area from different perspectives can be obtained, and the first image is obtained based on the multi-view data collection results.

[0105] In this embodiment, multispectral data collection is to use a sensor capable of sensing different spectral bands to simultaneously collect data of the target area in multiple spectral bands, and the second image is obtained based on the multispectral data collection results.

[0106] In this embodiment, the comprehensive image is the image obtained by integrating the first image and the second image. It contains both the appearance information such as the geometric shape, texture, and color of the target area from different perspectives, and the material characteristic information of the target area in different spectral bands.

[0107] In this embodiment, image processing is to perform a series of operations and transformations on the comprehensive image to improve the image quality, extract useful information, or prepare for subsequent analysis and processing. For example, image processing operations include image enhancement, image filtering, image segmentation, feature extraction, etc.

[0108] In this embodiment, the comprehensively processed image is the image obtained after image processing.

[0109] The beneficial effects of the above technologies are as follows: By determining the preset sensors according to the regional characteristics and modeling requirements, it is possible to achieve precise adaptation to the acquisition requirements, and use the target devices equipped with the preset sensors to collect data from multiple perspectives to obtain comprehensive first image images. Then, combined with the preset cameras to obtain multi-band spectral data to form second image images, and integrate and process them into comprehensively processed images, providing rich, accurate and comprehensive data for 3D modeling, which helps to improve the accuracy of 3D modeling.

[0110] Embodiment 3:

[0111] Based on the intelligent 3D modeling method based on machine learning in Embodiment 2, S2: Based on the intelligent 3D modeling requirements, combined with the feature extraction scheme, extract key image features from the comprehensively processed image to obtain key feature information, including:

[0112] Screen the feature extraction scheme that matches the target area from the preset feature extraction database based on the intelligent 3D modeling requirements;

[0113] Based on each feature extraction algorithm in the feature extraction scheme, extract the key image features corresponding to the intelligent 3D modeling from the comprehensively processed image, and perform feature sorting and classification to obtain the key feature information set.

[0114] In this embodiment, the intelligent 3D modeling requirements refer to the requirements put forward according to the specific application scenarios and targets during the 3D modeling process. It may include requirements for modeling accuracy, modeling speed, modeling content, and the degree of automation of modeling, etc.

[0115] In this embodiment, the feature extraction scheme is the method and strategy for extracting key image features from the comprehensively processed image. For example, the feature extraction scheme includes selecting appropriate feature extraction algorithms, determining the parameters of feature extraction, and the process of feature extraction.

[0116] In this embodiment, the key image features are the information that can reflect the important characteristics of the target area extracted from the comprehensively processed image. The key image features can be edges, corners, textures, color distributions, etc. in the image. For example, in the 3D modeling of a building, the edges and corners of the building can be used as key image features to determine the shape and structure of the building.

[0117] In this embodiment, the key feature information set is an overall formed by summarizing the key image features after feature sorting and classification. It contains all the key feature information required for the target area during the intelligent 3D modeling process.

[0118] The beneficial effects of the above technology are as follows: By combining the intelligent modeling requirements and the feature extraction scheme, the key feature information is accurately extracted, making the feature extraction more accurate and efficient, so that the 3D modeling of the target area is more accurate, and a more effective 3D modeling image is obtained.

[0119] Embodiment 4:

[0120] Based on Embodiment 3, an intelligent 3D modeling method based on machine learning, which screens a feature extraction scheme that matches the target area from a preset feature extraction database based on the intelligent 3D modeling requirements, includes:

[0121] Determine the 3D modeling requirements for intelligent 3D modeling based on the real-time application scenario, and determine the initial feature extraction scheme based on the 3D modeling requirements;

[0122] Analyze the feature types related to intelligent 3D modeling in the comprehensive processed image based on the initial feature extraction scheme as the initial feature types;

[0123] Comprehensively determine the comprehensive feature extraction method for intelligent 3D modeling by combining the initial feature extraction scheme and the initial feature types;

[0124] Optimize the comprehensive feature extraction method based on the feature extraction level requirements and the feature fusion mode requirements to obtain an optimized feature extraction method;

[0125] Randomly extract the image information of the corresponding type from the comprehensive processed image based on the initial feature types to verify the optimized feature extraction method, and use the verified optimized feature extraction method as the feature extraction scheme for the target area.

[0126] In this embodiment, the real-time application scenario refers to an application situation that actually occurs and requires an immediate response at a specific time and in a specific environment. For example, in the scenario of drone inspection, there will be real-time situations such as the influence of temporary obstacles and environmental wind power.

[0127] In this embodiment, the initial feature extraction scheme is a method and strategy for extracting features from image data determined according to the 3D modeling requirements. For example, it includes selecting a specific feature extraction algorithm, determining the parameter settings of the algorithm, and planning the application order of the algorithm.

[0128] In this embodiment, the initial feature types are the feature categories related to intelligent 3D modeling determined after analyzing the comprehensive processed image based on the initial feature extraction scheme. The chef feature types may include geometric features, texture features, spectral features, etc.

[0129] In this embodiment, the comprehensive feature extraction method is a feature extraction method for intelligent 3D modeling determined by comprehensively considering the initial feature extraction scheme and the initial feature types. It synthesizes the algorithm selection and parameter settings in the initial feature extraction scheme, as well as the feature scope covered by the initial feature types, aiming to more comprehensively and accurately extract the feature information useful for 3D modeling from the comprehensively processed images. The comprehensive feature extraction method is one of the key steps to realize the construction of 3D models from image data.

[0130] In this embodiment, the feature extraction level requirements refer to the different level requirements for feature extraction in the process of intelligent 3D modeling. For example, in low-level feature extraction, more attention is paid to the basic elements of the image, such as pixel points and edges; while in high-level feature extraction, more emphasis is placed on extracting features with semantic meanings, such as the category and pose of objects. The feature extraction level requirements will affect the complexity and effectiveness of the comprehensive feature extraction method.

[0131] In this embodiment, the feature fusion modality requirements refer to the ways and requirements for fusing different features in intelligent 3D modeling. Feature fusion can integrate the feature information from different sensors, different perspectives, or different feature extraction algorithms to improve the accuracy and robustness of 3D modeling. For example, the feature fusion modality requirements may include selecting appropriate fusion algorithms, determining the timing and level of fusion, etc.

[0132] In this embodiment, method optimization is a process of improving and perfecting the comprehensive feature extraction method based on the feature extraction level requirements and the feature fusion modality requirements. The optimization methods may include adjusting algorithm parameters, introducing new feature extraction algorithms, improving feature fusion strategies, etc.

[0133] In this embodiment, the optimized feature extraction method is obtained through method optimization and can better meet the requirements of intelligent 3D modeling. The optimized feature extraction method has improved in terms of the accuracy, efficiency, and robustness of feature extraction compared to the comprehensive feature extraction method.

[0134] In this embodiment, verification refers to the process of randomly extracting the corresponding type of image information from the comprehensively processed images based on the initial feature types to test and evaluate the optimized feature extraction method.

[0135] In this embodiment, the feature extraction scheme for the target area is the optimized feature extraction method that has passed the verification and is finally used for the feature extraction in the intelligent 3D modeling of the target area.

[0136] The beneficial effects of the above technologies are as follows: By combining the intelligent modeling requirements and the feature extraction scheme, key feature information is accurately extracted, making the feature extraction more accurate and efficient, thereby making the 3D modeling of the target area more accurate and obtaining a more effective 3D modeling image.

[0137] Example 5:

[0138] Based on Example 3, an intelligent 3D modeling method based on machine learning. S3: Classify regions based on the regional characteristics of the target region, and determine the initial machine learning model for each sub-region in combination with the intelligent 3D modeling requirements, and perform model optimization and integration to obtain a 3D reconstruction model of the target region, and generate an initial 3D image in combination with key feature information, including:

[0139] Classify regions based on the regional characteristics of the target region, and thus screen the initial machine learning models matching each sub-region from the preset model database based on the region classification results to obtain the initial machine learning model set of the target region;

[0140] Perform model training on each initial machine learning model in the initial machine learning model set based on the key feature information set and the comprehensive processed image, and perform corresponding model optimization based on each training result;

[0141] Obtain a 3D reconstruction model of the target region based on the model optimization results of each initial machine learning model in the machine learning model set;

[0142] Input the key feature information into the 3D reconstruction model to obtain the initial 3D image of the target region.

[0143] In this embodiment, the regional characteristics refer to the unique properties of the target region in terms of geography, environment, function, etc.

[0144] In this embodiment, region classification is to divide the target region into different sub-regions according to the regional characteristics of the target region. For example, a target region can be divided into static regions, dynamic regions, complex regions, simple regions, etc. The purpose of region classification is to adopt more appropriate 3D modeling methods and technologies for the characteristics of different sub-regions to improve the accuracy and efficiency of 3D modeling.

[0145] In this embodiment, the preset model database is a pre-constructed database that stores various types of machine learning models. These models have been pre-trained and optimized and are applicable to different application scenarios and data characteristics. The preset model database provides a rich selection for subsequent screening of initial machine learning models based on region classification results, enabling quick finding of models matching the target region sub-regions and reducing the time and cost of model development.

[0146] In this embodiment, the initial machine learning model is a machine learning model pre-selected for 3D reconstruction according to the regional characteristics of each sub-region and the requirements of intelligent 3D modeling. The machine learning model is selected from a preset model database. For example, for a sub-region with relatively flat terrain, a rule-based modeling method or a simple machine learning model can be selected; for a sub-region with complex terrain and dense buildings, a more complex model such as a deep learning model can be selected.

[0147] In this embodiment, the initial machine learning model set is an overall formed by summarizing the initial machine learning models corresponding to each sub-region obtained by classifying the target region according to regions. This set contains multiple initial machine learning models for different sub-regions of the target region.

[0148] In this embodiment, the key feature information set is an information set obtained after sorting and classifying the key image features related to intelligent 3D modeling extracted from the comprehensively processed image. The key feature information can accurately describe the attributes of the target region such as shape, structure, texture, and material, and is the key data for constructing the 3D model.

[0149] In this embodiment, model training is a process of using the key feature information set and part of the data of the comprehensively processed image as input data to learn each initial machine learning model in the initial machine learning model set.

[0150] In this embodiment, model optimization is a process of improving and perfecting the model based on the training results of each initial machine learning model. Model optimization may include adjusting the hyperparameters of the model, improving the structure of the model, adopting more advanced optimization algorithms, etc.

[0151] In this embodiment, the machine learning model set is the initial machine learning model set after model optimization.

[0152] In this embodiment, the 3D reconstruction model is obtained based on the model optimization results of each initial machine learning model in the machine learning model set, and is a model used to achieve the 3D reconstruction of the target region. The 3D reconstruction model can convert the input key feature information into the 3D geometric shape and spatial structure of the target region, and is the core of intelligent 3D modeling.

[0153] In this embodiment, the initial 3D image is a 3D image of the target region generated after inputting the key feature information into the 3D reconstruction model and being processed by the model.

[0154] The beneficial effects of the above technologies are as follows: By using the key feature information and the comprehensive image to train and optimize the model, the model is more in line with the characteristics of the target region, improving the accuracy of 3D reconstruction. The finally obtained initial 3D image of the target region can effectively improve the efficiency and quality of 3D modeling, and reduce the modeling difficulty and cost.

[0155] Example 6:

[0156] Based on Example 3, an intelligent 3D modeling method based on machine learning inputs key feature information into a 3D reconstruction model to obtain an initial 3D image of the target area, including:

[0157] Randomly extract any frame of image from the comprehensive processed image as the reference image of the target area;

[0158] Extract the key feature information corresponding to the current reference image, and based on the type of the key feature information, extract the processed images adjacent to the current reference image from the comprehensive processed images of the same band to obtain a second set of reference images;

[0159] Extract the key feature information of each second reference image in the second set of reference images, and combine it with the key feature information corresponding to the reference image to obtain a first set of key feature information of the target area;

[0160] Input the key feature information in the first set of key feature information and the regional parameters of the corresponding sub-region into the 3D reconstruction model to obtain the 3D skeleton image of the target area in the current sub-region;

[0161] Randomly extract the relative coordinates of any edge point in the reference modeling image and compare them with the relative coordinates of the corresponding edge point in the comprehensive processed image;

[0162] If the comparison error is less than the real-time modeling accuracy requirement, use the 3D skeleton image as the reference modeling image;

[0163] Otherwise, extract the comprehensive processed image of the adjacent band of the current reference image as the auxiliary reference image, and extract the key feature information corresponding to the auxiliary reference image to obtain an auxiliary image set adjacent to the auxiliary reference image, thereby obtaining an auxiliary feature information set;

[0164] Input the auxiliary feature information set and the regional parameters of the corresponding sub-region into the 3D reconstruction model to obtain a new 3D skeleton image, and compare to obtain the reference modeling image;

[0165] Perform error compensation on the reference modeling image based on the image error influence of the reference modeling image, thereby optimizing the reference modeling image to obtain a reference optimized image;

[0166] Use the reference optimized image as the regional modeling reference image of the current sub-region, and combine the key feature information belonging to the current region in the first set of key feature information and input it into the 3D reconstruction model to obtain the regional 3D modeling image of the current sub-region;

[0167] Perform image stitching on the regional 3D modeling images of each sub-region in the target region to obtain the 3D stitched image of the target region;

[0168] Obtain the stitching feature information volume based on the real-time modeling accuracy requirement, and obtain the comprehensive processing images near each stitching region in the 3D stitched image based on the stitching feature information volume to obtain a set of stitched images;

[0169] Randomly extract the feature information except the key feature information in the set of stitched images. When the feature information meets the stitching feature information volume, use the extracted feature information as the second set of key feature information;

[0170] Perform 3D filling on the 3D stitched image based on the feature information in the second set of key feature information to obtain a 3D optimized image;

[0171] Perform image fitting and image prediction on each stitching region image in the 3D optimized image to obtain the initial 3D image of the target region.

[0172] In this embodiment, the reference image is a randomly selected frame image from the comprehensive processing images and serves as the basis for 3D modeling.

[0173] In this embodiment, the key feature information refers to the information in the image that is representative and distinguishable, such as edges, corners, textures, etc., which can reflect the important structures and attributes of the target region.

[0174] In this embodiment, the second set of reference images is a set composed of the processing images of the adjacent frames to the current reference image extracted from the comprehensive processing images in the same waveband.

[0175] In this embodiment, the first set of key feature information is a set obtained by combining the key feature information of the reference image and each second reference image in the second set of reference images. It contains more comprehensive key feature information of the target region in the current sub-region and provides more accurate data support for 3D reconstruction.

[0176] In this embodiment, the regional parameters are the parameters related to the sub-regions of the target region, such as the size, shape, position, etc. of the sub-regions, which reflect the geometric features and spatial position relationships of the sub-regions.

[0177] In this embodiment, the 3D reconstruction model is a mathematical model or algorithm framework for converting key feature information and regional parameters into a 3D skeleton image. Through computational processing, the 3D reconstruction model can convert 2D image information into a 3D spatial representation to achieve the 3D reconstruction of the target region.

[0178] In this embodiment, the reference modeling image is a 3D image obtained after preliminary processing and comparison during the 3D reconstruction process.

[0179] In this embodiment, the comparison error refers to the relative coordinates of any edge point in the reference modeling image, and the relative error value T obtained by comparing with the relative coordinates of the corresponding edge point in the comprehensively processed image;

[0180]

[0181] where T is the comparison error, a1, a2, and a3 are the first coordinate value, the second coordinate value, and the third coordinate value of any edge point in the reference modeling image, b1 is the relative coordinate value of the corresponding comprehensively processed image, b2 and b3 are the second edge coordinate value and the third edge coordinate value of the corresponding edge point in the comprehensively processed image, δ is the data transmission error, where the direction of the first coordinate value is the same as that of the relative coordinate value, the direction of the second coordinate value is the same as that of the second edge coordinate value, and the direction of the third coordinate value is the same as that of the third edge coordinate value, α1 is the first error influence weight in the direction of the first coordinate value, α2 is the second error influence weight in the direction of the second coordinate value, α3 is the third error influence weight in the direction of the third coordinate value, and the sum of the weights of the first error influence weight, the second error influence weight, and the third error influence weight is 1.

[0182] In this embodiment, the error influence weight in each direction is related to the change rate of the current edge point of the reference modeling image in the current direction. The greater the change rate, the greater the error influence weight, and the smaller the change rate, the smaller the error influence weight.

[0183] In this embodiment, for example, the directions corresponding to the first coordinate value, the second coordinate value, and the third coordinate value can be the X-axis direction, the Y-axis direction, and the Z-axis direction.

[0184] In this embodiment, the auxiliary reference image is the corresponding reference image extracted from the comprehensively processed image of the adjacent band of the current reference image when the comparison error of the reference modeling image is greater than the real-time modeling accuracy requirement. The auxiliary reference image is used to provide additional key feature information to help improve the accuracy of 3D reconstruction.

[0185] In this embodiment, the auxiliary image set is a set composed of adjacent images in the same band as the auxiliary reference image. The images in the auxiliary image set are spatially related to the auxiliary reference image, which helps to extract the key feature information related to the auxiliary reference image.

[0186] In this embodiment, the auxiliary feature information set is a set obtained by combining the key feature information of the auxiliary reference image and each auxiliary image in the auxiliary image set.

[0187] In this embodiment, the regional three-dimensional modeling image refers to the regional modeling reference image obtained by taking the reference optimized image as the regional modeling reference image of the current sub-region and inputting the key feature information belonging to the current region in the first key feature information set into the three-dimensional reconstruction model. It reflects the three-dimensional structure and shape of the target region within the current sub-region.

[0188] In this embodiment, the three-dimensional stitching image is the image obtained by stitching the regional three-dimensional modeling images of each sub-region of the target region. It integrates the three-dimensional information of each sub-region of the target region and forms a complete three-dimensional representation of the target region.

[0189] In this embodiment, the stitching feature information quantity is the quantity of feature information to be extracted from the three-dimensional stitching image determined based on the real-time modeling accuracy requirement. It is the richness of the feature information that needs to be extracted from the three-dimensional stitching image to meet the real-time modeling accuracy requirement. For example, the stitching feature information quantity can be 100.

[0190] In this embodiment, the stitching image set is the set composed of the comprehensive processing images near each stitching region in the three-dimensional stitching image.

[0191] In this embodiment, the second key feature information set is the set obtained by randomly extracting the feature information other than the key feature information in the stitching image set when the feature information meets the stitching feature information quantity. It contains the feature information for three-dimensional filling and helps to improve the integrity and accuracy of the three-dimensional stitching edge.

[0192] In this embodiment, the three-dimensional optimized image refers to the image obtained by performing three-dimensional filling on the three-dimensional stitching image based on the feature information in the second key feature information set. It is the image that further optimizes and improves the three-dimensional stitching image.

[0193] In this embodiment, the initial three-dimensional image is the image obtained by performing image fitting and image prediction on each stitching region image in the three-dimensional optimized image.

[0194] The beneficial effects of the above technologies are as follows: By using the key feature information and the comprehensive image to train and optimize the model, the model is more in line with the characteristics of the target region, improving the accuracy of three-dimensional reconstruction. Finally, the initial three-dimensional image is obtained, which can effectively improve the efficiency and quality of three-dimensional modeling and reduce the modeling difficulty and cost.

[0195] Embodiment 7:

[0196] Based on Embodiment 6, an intelligent three-dimensional modeling method based on machine learning, S4: Combining preset constraint conditions to optimize the three-dimensional reconstruction model, and thus performing image optimization on the initial three-dimensional image based on the optimized model to obtain the three-dimensional modeling image, including:

[0197] Optimize the preset constraint conditions of the 3D processing model based on the real-time application scenario, so as to perform model constraint on the 3D processing model based on the optimized constraint conditions, and combine with the model optimization algorithm to optimize the model, obtaining an optimized 3D model;

[0198] Perform intelligent 3D modeling on the target area based on the optimized 3D model, obtaining a 3D optimized image of the target area;

[0199] Input the 3D optimized image and the initial 3D image into the same 3D coordinate system, so as to perform image comparison;

[0200] If the image error between the 3D optimized image and the initial 3D image is less than the real-time modeling accuracy requirement, then use the initial 3D image as the 3D modeling image of the target area;

[0201] Otherwise, perform image optimization on the initial 3D image based on the 3D optimized image, obtaining the 3D modeling image of the target area.

[0202] In this embodiment, the preset constraint conditions are the limiting conditions followed when the 3D reconstruction model is optimized. The preset constraint conditions can be physical constraints, geometric constraints or application constraints.

[0203] In this embodiment, the real-time application scenario refers to the real-time application scenario when the 3D processing model performs 3D modeling. For example, the real-time application scenario includes the application field, application area, etc.

[0204] In this embodiment, model optimization is a process of adjusting and improving the 3D reconstruction model according to the preset constraint conditions. For example, in the model optimization process, various optimization algorithms (such as gradient descent algorithm, genetic algorithm, etc.) are used to adjust the parameters of the model, so that the model can achieve better performance and accuracy on the premise of meeting the constraint conditions.

[0205] In this embodiment, model constraint is the process of applying the preset constraint conditions to the 3D processing model. Through model constraint, the behavior of the model in the generation or optimization process can be restricted, so that the output result meets the preset requirements.

[0206] In this embodiment, the model optimization algorithm is an algorithm used to improve and enhance the 3D processing model. These algorithms make the model perform better on the given evaluation indexes (such as accuracy, efficiency, generalization ability, etc.) by adjusting the parameters, structure or training process of the model.

[0207] In this embodiment, the optimized 3D model is the 3D processing model obtained after being processed by model constraint and model optimization algorithm.

[0208] In this embodiment, the 3D optimized image is the image obtained by performing intelligent 3D modeling on the target area based on the optimized 3D model.

[0209] In this embodiment, the image error is the degree of difference between the three-dimensional optimized image and the initial three-dimensional image when they are compared in the same three-dimensional coordinate system. The image error can be measured by various metrics, such as the mean square error (MSE), etc.

[0210] In this embodiment, the real-time modeling accuracy requirement refers to the accuracy requirement for the three-dimensional modeling result of the target area in a real-time application scenario. The real-time modeling accuracy requirement is usually determined according to specific application scenarios and task objectives.

[0211] In this embodiment, the three-dimensional modeling image is a three-dimensional image of the target area obtained by optimizing the initial three-dimensional image based on the three-dimensional optimized image.

[0212] The beneficial effects of the above technologies are as follows: By presetting constraint conditions to constrain the three-dimensional processing model and combining with an optimization algorithm for optimization, the model accuracy and stability can be improved, a more practical optimized three-dimensional model can be obtained, thereby obtaining a three-dimensional optimized image, comparing it with the initial three-dimensional image, and judging the optimization performance in combination with the real-time modeling accuracy requirement. On the one hand, the modeling quality is guaranteed, and on the other hand, the resource waste caused by over-optimization is avoided, effectively improving the efficiency and accuracy of three-dimensional modeling.

[0213] Embodiment 8:

[0214] Based on Embodiment 7, an intelligent three-dimensional modeling method based on machine learning, which performs constraint optimization on the preset constraint conditions of the three-dimensional processing model based on a real-time application scenario, thereby performing model constraint on the three-dimensional processing model based on the constraint-optimized constraint conditions, and combining with a model optimization algorithm for model optimization to obtain an optimized three-dimensional model, including:

[0215] Obtain key scene information related to three-dimensional modeling in the real-time application scenario;

[0216] Dynamically adjust the constraint parameters of the preset constraint conditions of the three-dimensional processing model according to the key scene information to obtain constraint-optimized parameters;

[0217] Combine the constraint-optimized parameters with the parameters in the preset constraint conditions that have not been constraint-optimized to obtain constraint-optimized conditions, and perform constraints on the three-dimensional processing model based on the constraint-optimized conditions;

[0218] Perform model optimization on the constrained three-dimensional processing model based on a preset optimization algorithm to obtain an optimized three-dimensional model.

[0219] In this embodiment, the constraint optimization condition is to fuse the dynamically adjusted constraint optimization parameters with the unadjusted preset constraint condition parameters to form a complete constraint set. For example, if the preset constraint condition includes "the model must be a closed grid" and "the surface roughness of the model needs to be less than 0.1" is added after dynamic adjustment, the combined constraint condition is the combination of the two.

[0220] In this embodiment, the model constraint is to embed the combined constraint conditions into the three-dimensional processing model in a mathematical form.

[0221] In this embodiment, the preset optimization algorithm is an algorithm for optimizing the three-dimensional model, such as the gradient descent method, genetic algorithm, reinforcement learning, etc.

[0222] The beneficial effects of the above technologies are as follows: By constraining the three-dimensional processing model with preset constraint conditions and combining optimization algorithms for optimization, the accuracy and stability of the model can be improved, so that the optimization of the initial three-dimensional image is more accurate and efficient, effectively improving the efficiency and accuracy of three-dimensional modeling.

[0223] Embodiment 9:

[0224] The present invention provides an intelligent three-dimensional modeling system based on machine learning for performing the intelligent three-dimensional modeling method according to any one of Embodiments 1 to 8, referring to Figure 2 , including:

[0225] An acquisition and processing module, configured to collect multi-view and multi-spectral data of a target area based on a preset sensor, obtain a comprehensive image of the target area, and perform image processing to obtain a comprehensively processed image;

[0226] A feature extraction module: configured to extract key image features from the comprehensively processed image based on the intelligent three-dimensional modeling requirements and in combination with a feature extraction scheme to obtain key feature information;

[0227] A model optimization module: configured to perform region classification based on the regional characteristics of the target area, determine an initial machine learning model for each sub-region in combination with the intelligent three-dimensional modeling requirements, and perform model optimization and integration to obtain a three-dimensional reconstruction model of the target area, and generate an initial three-dimensional image in combination with the key feature information;

[0228] An image optimization module: configured to perform model optimization on the three-dimensional reconstruction model in combination with preset constraint conditions, and thus perform image optimization on the initial three-dimensional image based on the optimized model to obtain a three-dimensional modeling image.

[0229] The beneficial effects of the above technology are as follows: Through multi-perspective and multi-spectral data collection and processing, a comprehensive and accurate integrated processed image is obtained. According to the intelligent modeling requirements and feature extraction schemes, key feature information is accurately extracted, making the feature extraction more accurate and efficient. Combining with the regional characteristics classification and modeling requirements, the initial machine learning model is determined and optimized to generate an initial three-dimensional image adapted to the target area. And the model and image are double-optimized using preset constraint conditions, effectively improving the quality of the three-dimensional modeling image, meeting diverse application requirements, and enhancing the real-time modeling performance.

[0230] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these changes and modifications.

Claims

1. An intelligent three-dimensional modeling method based on machine learning, characterized in that Including: S1: Based on a preset sensor, multi-view and multi-spectral data of the target area are collected to obtain a comprehensive image of the target area, and image processing is performed to obtain a comprehensively processed image; S2: Based on the requirements of intelligent 3D modeling, key image features are extracted from the comprehensively processed image in combination with a feature extraction scheme to obtain key feature information; S3: Region classification is performed based on the regional characteristics of the target area, and the initial machine learning model of each sub-region is determined in combination with the requirements of intelligent 3D modeling, and model optimization and integration are performed to obtain a 3D reconstruction model of the target area, and an initial 3D image is generated in combination with the key feature information; S4: The 3D reconstruction model is optimized in combination with preset constraint conditions, so that the initial 3D image is optimized based on the optimized model to obtain a 3D modeling image.

2. The intelligent three-dimensional modeling method based on machine learning according to claim 1, wherein S1: Based on a preset sensor, multi-view and multi-spectral data of the target area are collected to obtain a comprehensive image of the target area, and image processing is performed to obtain a comprehensively processed image, including: Based on the regional characteristics of the target area and the requirements of intelligent 3D modeling, the sensor for data collection of the target area is determined as the preset sensor; Based on the target device equipped with the preset sensor, data of the target area are collected from different perspectives to obtain the first image of the target area in a multi-view state; Based on a preset camera, spectral data of different bands are obtained to obtain multi-band spectral data as the second image; The first image and the second image are integrated to obtain a comprehensive image of the target area, and image processing is performed on the comprehensive image to obtain a comprehensively processed image.

3. The intelligent three-dimensional modeling method based on machine learning according to claim 2, wherein, S2: Based on the requirements of intelligent 3D modeling, key image features are extracted from the comprehensively processed image in combination with a feature extraction scheme to obtain key feature information, including: Based on the requirements of intelligent 3D modeling, a feature extraction scheme that matches the target area is selected from a preset feature extraction database; Based on each feature extraction algorithm in the feature extraction scheme, key image features corresponding to intelligent 3D modeling are extracted from the comprehensively processed image, and feature sorting and classification are performed to obtain a key feature information set.

4. The intelligent three-dimensional modeling method based on machine learning according to claim 3, characterized in that, Selecting a feature extraction scheme that matches the target area from a preset feature extraction database based on the requirements of intelligent 3D modeling, including: Based on the real-time application scenario, the 3D modeling requirements for intelligent 3D modeling are determined, and an initial feature extraction scheme is determined based on the 3D modeling requirements; Based on the initial feature extraction scheme, the feature types related to intelligent 3D modeling in the comprehensively processed image are analyzed as the initial feature types; The initial feature extraction scheme and the initial feature types are comprehensively determined to obtain a comprehensive feature extraction method for intelligent 3D modeling; Based on the requirements of the feature extraction level and the feature fusion mode, the comprehensive feature extraction method is optimized to obtain an optimized feature extraction method; Based on the initial feature types, image information of the corresponding type is randomly extracted from the comprehensively processed image to verify the optimized feature extraction method, and the verified optimized feature extraction method is used as the feature extraction scheme of the target area.

5. The intelligent three-dimensional modeling method based on machine learning according to claim 3, wherein S3: Perform region classification based on the regional characteristics of the target area, determine the initial machine learning model for each sub-region in combination with the intelligent 3D modeling requirements, and perform model optimization and integration to obtain the 3D reconstruction model of the target area, and generate the initial 3D image in combination with the key feature information, including: Perform region classification based on the regional characteristics of the target area, so as to screen the initial machine learning models matching each sub-region from the preset model database based on the region classification results in combination with the intelligent 3D modeling requirements, and obtain the initial machine learning model set of the target area; Perform model training on each initial machine learning model in the initial machine learning model set based on the key feature information set and the comprehensively processed image, and perform corresponding model optimization based on each training result; Obtain the 3D reconstruction model of the target area based on the model optimization results of each initial machine learning model in the machine learning model set; Input the key feature information into the 3D reconstruction model, so as to obtain the initial 3D image of the target area.

6. The intelligent three-dimensional modeling method based on machine learning according to claim 5, characterized in that, Input the key feature information into the 3D reconstruction model, so as to obtain the initial 3D image of the target area, including: Randomly extract any frame of image from the comprehensively processed image as the reference image of the target area; Extract the key feature information corresponding to the current reference image, and extract the processed images adjacent to the current reference image from the comprehensively processed images of the same waveband based on the type of the key feature information to obtain the second reference image set; Extract the key feature information of each second reference image in the second reference image set, and combine the key feature information corresponding to the reference image to obtain the first key feature information set of the target area; Input the key feature information in the first key feature information set and the regional parameters of the corresponding sub-region into the 3D reconstruction model to obtain the 3D skeleton image of the target area in the current sub-region; Randomly extract the relative coordinates of any edge point in the reference modeling image, and compare them with the relative coordinates of the corresponding edge point in the comprehensively processed image; If the comparison error is less than the real-time modeling accuracy requirement, use the 3D skeleton image as the reference modeling image; Otherwise, extract the comprehensively processed image of the adjacent waveband of the current reference image as the auxiliary reference image, and extract the key feature information corresponding to the auxiliary reference image to obtain the auxiliary image set adjacent to the auxiliary reference image, so as to obtain the auxiliary feature information set; Input the auxiliary feature information set and the regional parameters of the corresponding sub-region into the 3D reconstruction model to obtain a new 3D skeleton image, and compare to obtain the reference modeling image; Perform error compensation on the reference modeling image based on the image error influence of the reference modeling image, so as to optimize the reference modeling image to obtain the reference optimized image; Use the reference optimized image as the regional modeling reference image of the current sub-region, and input the key feature information belonging to the current region in the first key feature information set into the 3D reconstruction model to obtain the regional 3D modeling image of the current sub-region; Perform image stitching on the regional 3D modeling images of each sub-region of the target area to obtain the 3D stitched image of the target area; Obtain the amount of splicing feature information based on the real-time modeling accuracy requirement, and obtain the comprehensive processing image near each splicing area in the three-dimensional splicing image based on the amount of splicing feature information to obtain a set of splicing images; Randomly extract the feature information other than the key feature information in the set of splicing images. When the feature information meets the amount of splicing feature information, use the extracted feature information as the second set of key feature information; Perform three-dimensional filling on the three-dimensional splicing image based on the feature information in the second set of key feature information to obtain a three-dimensional optimized image; Perform image fitting and image prediction on each splicing area image in the three-dimensional optimized image to obtain the initial three-dimensional image of the target area.

7. A machine learning-based intelligent three-dimensional modeling method according to claim 5, characterized in that, S4: Optimize the three-dimensional reconstruction model in combination with the preset constraint conditions, and thus optimize the initial three-dimensional image based on the optimized model to obtain a three-dimensional modeling image, including: Conduct constraint optimization on the preset constraint conditions of the three-dimensional processing model based on the real-time application scenario, and thus perform model constraint on the three-dimensional processing model based on the constraint conditions after constraint optimization, and combine the model optimization algorithm to perform model optimization to obtain an optimized three-dimensional model; Perform intelligent three-dimensional modeling on the target area based on the optimized three-dimensional model to obtain a three-dimensional optimized image of the target area; Input the three-dimensional optimized image and the initial three-dimensional image into the same three-dimensional coordinate system for image comparison; If the image error between the three-dimensional optimized image and the initial three-dimensional image is less than the real-time modeling accuracy requirement, use the initial three-dimensional image as the three-dimensional modeling image of the target area; Otherwise, optimize the initial three-dimensional image based on the three-dimensional optimized image to obtain the three-dimensional modeling image of the target area.

8. An intelligent three-dimensional modeling method based on machine learning according to claim 7, characterized in that, Conduct constraint optimization on the preset constraint conditions of the three-dimensional processing model based on the real-time application scenario, and thus perform model constraint on the three-dimensional processing model based on the constraint conditions after constraint optimization, and combine the model optimization algorithm to perform model optimization to obtain an optimized three-dimensional model, including: Obtain the key scene information related to three-dimensional modeling in the real-time application scenario; Dynamically adjust the constraint parameters of the preset constraint conditions of the three-dimensional processing model according to the key scene information to obtain constraint optimization parameters; Combine the constraint optimization parameters with the parameters in the preset constraint conditions that have not been constraint optimized to obtain constraint optimization conditions, and perform constraints on the three-dimensional processing model based on the constraint optimization conditions; Perform model optimization on the constrained three-dimensional processing model based on the preset optimization algorithm to obtain an optimized three-dimensional model.

9. An intelligent three-dimensional modeling system based on machine learning, characterized in that, Used to execute the intelligent three-dimensional modeling method based on machine learning described in any one of claims 1 to 8, including: An acquisition and processing module, used to collect multi-view and multi-spectral data of the target area based on a preset sensor to obtain a comprehensive image of the target area, and perform image processing to obtain a comprehensive processing image; A feature extraction module: used to extract key image features from the comprehensive processing image based on the intelligent three-dimensional modeling requirement and in combination with the feature extraction scheme to obtain key feature information; Model optimization module: It is used to classify regions based on the regional characteristics of the target region, determine the initial machine learning model for each sub-region in combination with the requirements of intelligent 3D modeling, and perform model optimization and integration to obtain a 3D reconstruction model of the target region, and generate an initial 3D image in combination with key feature information; Image optimization module: It is used to optimize the 3D reconstruction model in combination with preset constraint conditions, so as to optimize the initial 3D image based on the optimized model to obtain a 3D modeling image.

Images