Indoor 3D reconstruction method based on feature descriptor

CN118229913BActive Publication Date: 2024-11-12BEIJING DAXI TECH CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202410374468.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-29
Publication Date
2024-11-12
Estimated Expiration
2044-03-29

Smart Images

  • Figure CN118229913B_ABST
    Figure CN118229913B_ABST
Patent Text Reader

Abstract

The present application provides a method for indoor three-dimensional reconstruction based on feature descriptors, and relates to the field of three-dimensional reconstruction technology. The method includes: reading a source data set of a reconstruction target; pre-processing the source data set based on a preprocessing model to determine preprocessed source data, wherein the preprocessing model includes preprocessing branches corresponding to each data modality; extracting multimodal data features from the preprocessed source data to determine feature descriptors; performing multimodal fitting and dimensional conversion on the feature descriptors to determine a reconstruction feature set; performing a three-dimensional reconstruction operation based on the reconstruction feature set to obtain a three-dimensional reconstruction target. The present application can solve the technical problem that the existing three-dimensional reconstruction methods have poor precision, accuracy and comprehensiveness in the selection of reconstruction feature data, resulting in poor precision and quality of target three-dimensional reconstruction, and can improve the precision, accuracy and comprehensiveness of the selection of reconstruction feature data, thereby achieving the effect of improving the precision and quality of target three-dimensional reconstruction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional reconstruction, and in particular to a method for indoor three-dimensional reconstruction based on feature descriptors. Background Art

[0002] Three-dimensional reconstruction refers to the process of using computer vision technology to extract targets from two-dimensional images or videos and restore them to three-dimensional structures. It has a wide range of application scenarios, such as: 3D printing, augmented reality, urban planning, virtual reality, cultural relics protection, etc.

[0003] In the process of target 3D reconstruction, in addition to considering factors such as lighting conditions and occlusion problems, the integrity and accuracy of target data acquisition are crucial. The existing indoor target 3D reconstruction methods are unable to accurately determine which feature data are necessary for reconstruction, and therefore are unable to accurately and effectively select the reconstruction feature data, resulting in poor precision, accuracy and comprehensiveness in the selection of reconstruction feature data, which ultimately affects the precision and quality of target 3D reconstruction.

[0004] In summary, the existing indoor target 3D reconstruction methods have the technical problems of poor precision, accuracy and comprehensiveness in selecting reconstruction feature data, which leads to poor precision and quality of target 3D reconstruction. Summary of the invention

[0005] The purpose of this application is to provide a feature descriptor-based indoor 3D reconstruction method to solve the technical problem that the existing indoor 3D reconstruction methods have poor precision, accuracy and comprehensiveness in the selection of reconstruction feature data, resulting in poor precision and quality of target 3D reconstruction.

[0006] In view of the above problems, the present application provides a method for indoor 3D reconstruction based on feature descriptors.

[0007] In a first aspect, the present application provides a method for indoor three-dimensional reconstruction based on feature descriptors, and the method is implemented by an indoor three-dimensional reconstruction system based on feature descriptors, wherein the method includes: reading a source data set of a reconstruction target, wherein the source data set includes multiple data modalities and multiple data dimensions, and there is a homologous mapping identifier; based on a preprocessing model, pre-processing the source data set to determine the preprocessed source data, wherein the preprocessing model includes parallel preprocessing branches corresponding to each data modality; performing multimodal data feature extraction based on a feature extraction module on the preprocessed source data to determine a feature descriptor, wherein the feature descriptor has a modal identifier; performing multimodal fitting and dimensional conversion on the feature descriptor to determine a reconstruction feature set; determining a reconstruction space based on a visual simulation platform, performing a three-dimensional reconstruction operation of the reconstruction target based on the reconstruction feature set, and obtaining a three-dimensional reconstruction target.

[0008] In a second aspect, the present application further provides a feature descriptor-based indoor 3D reconstruction system, which is used to execute a feature descriptor-based indoor 3D reconstruction method as described in the first aspect, wherein the system includes: a source data set reading module, the source data set reading module is used to read the source data set of the reconstruction target, the source data set includes multiple data modalities and multiple data dimensions, and there is a homology mapping identifier; a pre-processing source data determination module, the pre-processing source data determination module is used to pre-process the source data set based on a pre-processing model to determine the pre-processed source data, wherein the pre-processing model includes parallel pre-processing branches corresponding to each data modality; a feature descriptor determination module, the feature descriptor determination module is used to perform multi-modal data feature extraction based on a feature extraction module on the pre-processed source data, determine the feature descriptor, and the feature descriptor has a modality identifier; a reconstruction feature set determination module, the reconstruction feature set determination module is used to perform multi-modal fitting and dimensional conversion on the feature descriptor to determine the reconstruction feature set; a 3D reconstruction module, the 3D reconstruction module is used to determine a reconstruction space based on a visual simulation platform, perform a 3D reconstruction operation of the reconstruction target based on the reconstruction feature set, and obtain a 3D reconstruction target.

[0009] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0010] By reading the source data set of the reconstruction target, the source data set includes multiple data modes and multiple data dimensions, and there is a homologous mapping identifier; based on the preprocessing model, the source data set is pre-processed to determine the preprocessed source data, wherein the preprocessing model includes parallel preprocessing branches corresponding to each data mode; for the preprocessed source data, multimodal data feature extraction based on the feature extraction module is performed to determine the feature descriptor, and the feature descriptor has a modal identifier; multimodal fitting and dimension conversion are performed on the feature descriptor to determine the reconstruction feature set; a reconstruction space based on a visual simulation platform is determined, and a three-dimensional reconstruction operation of the reconstruction target is performed based on the reconstruction feature set to obtain a three-dimensional reconstruction target. In other words, by combining feature descriptor technology for feature extraction, and performing multimodal fitting and dimension conversion on the extracted feature descriptors to obtain a reconstruction feature set, the technical goal of improving the precision, accuracy and comprehensiveness of the selection of reconstruction feature data can be achieved. Finally, in the reconstruction space, a three-dimensional reconstruction operation is performed based on the reconstruction feature set, which can achieve the technical effect of improving the precision and quality of the target three-dimensional reconstruction.

[0011] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented according to the contents of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are specifically cited below. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0013] Figure 1 A schematic diagram of a process flow of an indoor 3D reconstruction method based on feature descriptors in this application;

[0014] Figure 2 A schematic diagram of a process for pre-processing the source data set in a method for indoor 3D reconstruction based on a feature descriptor of the present application;

[0015] Figure 3 This is a structural schematic diagram of an indoor 3D reconstruction system based on feature descriptors in this application.

[0016] Description of reference numerals:

[0017] A source data set reading module 11 , a pre-processed source data determination module 12 , a feature descriptor determination module 13 , a reconstruction feature set determination module 14 , and a three-dimensional reconstruction module 15 . DETAILED DESCRIPTION

[0018] This application provides a method for indoor 3D reconstruction based on feature descriptors, which solves the technical problem that the existing methods for indoor target 3D reconstruction have poor precision, accuracy and comprehensiveness in selecting reconstruction feature data, resulting in poor precision and quality of target 3D reconstruction. The technical goal of improving the precision, accuracy and comprehensiveness of reconstruction feature data selection can be achieved, and the technical effect of improving the precision and quality of target 3D reconstruction can be achieved.

[0019] Below, the technical solutions in the present application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments of the present application. It should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. It should also be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, rather than all of them.

[0020] Embodiment 1

[0021] Please refer to the attached Figure 1 The present application provides a method for indoor three-dimensional reconstruction based on feature descriptors, wherein the method is applied to an indoor three-dimensional reconstruction system based on feature descriptors, and the method specifically comprises the following steps:

[0022] Step 1: Read the source data set of the reconstruction target, where the source data set includes multiple data modalities and multiple data dimensions, and has a homologous mapping identifier;

[0023] Specifically, the method provided in the present application is used to extract reconstruction features from source data of indoor targets based on feature descriptor technology. Feature descriptors refer to necessary information that can easily distinguish and identify objects, and then complete three-dimensional reconstruction of indoor targets based on the extracted reconstruction features. The method is specifically implemented in a feature descriptor-based indoor three-dimensional reconstruction system.

[0024] Read the source data set of the reconstruction target. The reconstruction target is an indoor target and can be set according to the actual situation, such as indoor furniture such as tables and sofas. The source data set includes multiple data modalities and multiple data dimensions, and there is a homologous mapping identifier, where the data modality refers to the data type in the source data set, and the data temperature refers to the data feature or data attribute in the source data set, such as time dimension, space dimension, two-dimensional data, three-dimensional data, etc. The homologous mapping identifier refers to the mapping mark of multiple modal data of the same target part, that is, the data is mapped to the target part. By obtaining the source data set of the reconstruction target, the original data support is provided for the next step of reconstructing feature data extraction.

[0025] Step 2: Pre-processing the source data set based on a pre-processing model to determine pre-processed source data, wherein the pre-processing model includes parallel pre-processing branches corresponding to each data modality;

[0026] Specifically, a preprocessing model is constructed, wherein the preprocessing model includes multiple preprocessing branches, wherein each data modality corresponds to a preprocessing branch, and multiple preprocessing branches exist in parallel. Then, the source data set is preprocessed by the preprocessing model, wherein preprocessing refers to preliminary image processing before feature extraction, including steps such as image size adjustment, grayscale processing, and local area enhancement, to obtain preprocessed source data.

[0027] By pre-processing the source data set, interference from other factors can be reduced, data feature extraction can be facilitated, and the efficiency and accuracy of data feature extraction can be improved.

[0028] Step 3: performing multimodal data feature extraction based on a feature extraction module on the preprocessed source data to determine a feature descriptor, wherein the feature descriptor has a modality identifier;

[0029] Specifically, first, a feature extraction module is constructed, which is based on the histogram of oriented gradients, where the histogram of oriented gradients is a popular feature descriptor technology in computer vision and image processing, which describes the shape and appearance of an object by analyzing the distribution of edge directions within the object, where the feature descriptor creates a gradient histogram by counting the number of occurrences of gradient directions in a local part of the image.

[0030] Then, based on the feature extraction module, the multimodal data features in the preprocessed source data are extracted to determine the feature descriptor, which is a descriptor for multimodal features, that is, the necessary information required to identify multimodal features, wherein the feature descriptor has a modality identifier. For modal data other than the image modality in the preprocessed source data, feature extraction can be performed directly without determining the feature descriptor through the feature extraction module. By determining the feature descriptor, the necessary features of the reconstructed target can be accurately obtained, and at the same time, support is provided for obtaining the reconstruction feature set in the next step.

[0031] Step 4: performing multimodal fitting and dimension conversion on the feature descriptor to determine a reconstructed feature set;

[0032] Specifically, the feature descriptor is further subjected to multimodal fitting and dimensional conversion, wherein multimodal fitting refers to fitting analysis and data fusion of multiple types of data, and dimensional conversion refers to converting the two-dimensional features in the feature descriptor into three-dimensional features, and setting the features after multimodal fitting and dimensional conversion as reconstruction features to determine the reconstruction feature set. By determining the reconstruction feature set, a basis is provided for the next step of three-dimensional reconstruction of the reconstruction target.

[0033] Step 5: Determine a reconstruction space based on a visual simulation platform, perform a three-dimensional reconstruction operation on the reconstruction target based on the reconstruction feature set, and obtain a three-dimensional reconstruction target.

[0034] Specifically, the reconstruction space is built based on a visual simulation platform, where the visual simulation platform is a technology that uses computer graphics and virtual reality technology to graphically display objects, scenes, processes, etc. in the real world or virtual world, so that users can interact, analyze, optimize, etc. Commonly used visual simulation platforms include CAD software, catla software, solidworks software, etc. Technical personnel in this field can select an appropriate visual simulation platform to build the reconstruction space based on actual conditions.

[0035] Then, in the reconstruction space, a three-dimensional reconstruction operation of the reconstruction target is performed according to the reconstruction feature set to obtain a three-dimensional reconstruction target. The three-dimensional reconstruction operation can be performed by three-dimensional reconstruction software, and common three-dimensional reconstruction software includes Autodesk ReCap software, Blender software, etc., which can be selected according to actual application scenarios and needs.

[0036] The indoor three-dimensional reconstruction method based on feature descriptors is applied to an indoor three-dimensional reconstruction system based on feature descriptors, which can solve the technical problem that the existing indoor target three-dimensional reconstruction method has poor precision, accuracy and comprehensiveness in the selection of reconstruction feature data, resulting in poor precision and quality of target three-dimensional reconstruction. First, read the source data set of the reconstruction target, the source data set includes multiple data modalities and multiple data dimensions, and there is a homologous mapping identifier; then, based on the preprocessing model, pre-process the source data set to determine the preprocessed source data, wherein the preprocessing model includes parallel preprocessing branches corresponding to each data modality; then, perform multimodal data feature extraction based on the feature extraction module on the preprocessed source data to determine the feature descriptor, and the feature descriptor has a modal identifier; next, perform multimodal fitting and dimensional conversion on the feature descriptor to determine the reconstruction feature set; finally, determine the reconstruction space based on the visual simulation platform, perform the three-dimensional reconstruction operation of the reconstruction target based on the reconstruction feature set, and obtain the three-dimensional reconstruction target. By combining feature descriptor technology to extract features, and performing multimodal fitting and dimensionality conversion on the extracted feature descriptors, a reconstruction feature set is obtained. This can achieve the technical goal of improving the precision, accuracy and comprehensiveness of reconstruction feature data selection. Finally, within the reconstruction space, three-dimensional reconstruction operations are performed based on the reconstruction feature set, which can achieve the technical effect of improving the target three-dimensional reconstruction accuracy and quality.

[0037] Further, as attached Figure 2 As shown, step 2 of this application includes:

[0038] Identify the reconstruction criteria and set the image size and cell division size based on the reconstruction accuracy;

[0039] Based on the image size and the cell division size, in combination with the image preprocessing branch, the image modality data is subjected to aspect ratio adjustment and cell division to determine a one-step preprocessing image;

[0040] The one-step preprocessed image is subjected to grayscale processing and local enhancement processing to determine preprocessed image data, and is added to the preprocessed source data.

[0041] Specifically, first, extract the image modality data in the source data set, the image modality data refers to the data of the source data set whose data type is image, such as pictures, videos, etc.; then identify the reconstruction standard, which can be set according to the actual reconstruction requirements of the reconstruction target, including reconstruction accuracy, integrity index, reconstruction dimension, reconstruction mode, etc. Then set the image size and cell division size according to the reconstruction accuracy, wherein the image size can be set based on the calculation requirements of the oriented gradient histogram, and the image size is usually set to a size with an aspect ratio of 1 to 2, such as: 64 by 128 images; wherein the higher the reconstruction accuracy, the smaller the cell division size, for example: the cell division size can be set to 8 by 8 cells.

[0042] Then, based on the image size, the image modality data is adjusted in aspect ratio in combination with the image preprocessing branch, that is, the aspect ratio of the image modality data is adjusted to 1 to 2; the image modality data after the aspect ratio adjustment is further divided into cells based on the cell division size to obtain a one-step preprocessed image. Then, the one-step preprocessed image is subjected to image grayscale processing and local enhancement processing, wherein image grayscale processing refers to the process of converting a color image into a grayscale image. A grayscale image is a monochrome image in which each pixel has only one brightness value, usually represented by a numerical value between 0 and 255. Grayscale processing can simplify image information, reduce the amount of calculation, and retain sufficient information for subsequent processing; wherein local enhancement processing refers to image enhancement of local areas with obvious segmentation features in the image, such as contour areas in the image; the preprocessed image data after grayscale processing and local enhancement processing is obtained, and added to the preprocessed source data to obtain the preprocessed source data.

[0043] By preprocessing the image modal data in the source data set, the accuracy of image modal data recognition and analysis can be improved, and support can be provided for the next step of image modal data feature extraction.

[0044] Further, step 4 of this application includes:

[0045] Based on the reconstruction standard, determine the reconstruction dimension and reconstruction state;

[0046] Based on the remodeled modality, activating a modal conversion unit, performing modal conversion processing on the feature descriptor, and determining a conversion feature set, wherein the conversion feature set is identified by a mapping angle;

[0047] The mapping angle is identified, the dimension of the conversion feature set is converted, multi-angle fusion reconstruction of the same feature is performed, and the target feature set is determined.

[0048] Specifically, first, the reconstruction dimension and reconstruction mode are determined based on the reconstruction standard, wherein the reconstruction dimension refers to the data dimension of the reconstructed target, for example, converting two-dimensional image data into three-dimensional image data, and the reconstruction mode refers to the data type of the reconstructed target. Then, based on the reconstruction mode, a modality conversion unit is activated, and the modality conversion unit is used to perform data type conversion, for example, a functional conversion branch of speech mode-image mode.

[0049] The modal conversion unit performs modal conversion processing on the feature descriptor to obtain a conversion feature set, wherein the conversion feature set is marked with a mapping angle, wherein the mapping angle refers to the recognition angle corresponding to the conversion feature, for example: when converting a two-dimensional image feature into a feature based on a three-dimensional image, each image feature is marked with a recognition angle. The mapping angle is further identified, and the conversion feature set is dimensionally converted. During the dimensional conversion process, multi-angle fusion reconstruction of the same feature is performed based on the mapping angle to obtain a target feature set.

[0050] Furthermore, the present application also includes the following steps:

[0051] If the target feature set does not conform to the reconstructed dimension, determining a dimension processing instruction, wherein the dimension processing includes a dimension increase processing and a dimension decrease processing;

[0052] Based on the dimension processing instruction, the dimension processing unit is activated to process the fused feature set to determine the target feature set. The dimension processing unit can perform bidirectional processing between high dimension and low dimension.

[0053] Specifically, after performing multi-angle fusion of the same feature, it is determined whether the target feature set conforms to the reconstruction dimension. If the target feature set does not conform to the reconstruction dimension, the dimension processing instruction is determined, wherein the dimension processing includes dimensionality increase processing and dimensionality reduction processing. Dimensionality increase processing refers to mapping the image from a low-dimensional space to a high-dimensional space. The dimensionality increase processing can improve the effect of image classification or recognition; dimensionality reduction processing refers to mapping the image from a high-dimensional space to a low-dimensional space. The dimensionality reduction processing can simplify the image information and reduce the amount of calculation. At the same time, it can remove redundant information or noise, thereby improving the efficiency and accuracy of image processing.

[0054] Then, based on the dimension processing instruction, the dimension processing unit is activated to process the fused feature set, where the fused feature set refers to feature data obtained after multi-angle fusion reconstruction of the same feature to obtain the target feature set, wherein the dimension processing unit can perform bidirectional processing between high dimension and low dimension, that is, it can perform both dimensionality increase and dimensionality decrease operations.

[0055] Furthermore, the present application also includes the following steps:

[0056] Verify whether the target feature set meets the reconstruction standard, and if not, locate the extended part;

[0057] Performing big data mining to determine extended features based on the extended portion;

[0058] The extended features are interspersed into the target feature set.

[0059] Specifically, after determining the target feature set, first, check whether the target feature set meets the reconstruction standard. If the target feature set does not meet the reconstruction standard, locate the extended part, which refers to some features in the target feature set that do not meet the reconstruction standard. For example, insufficient source data information leads to insufficient extracted target features, making it impossible to guarantee the degree of restoration and integrity of the reconstructed target.

[0060] Big data technology is a series of technologies and methods that enable the value contained in massive data to be mined and displayed, including data planning, data collection, analysis and mining, etc. Then, the extended part is used as the search condition, data mining is performed based on big data technology, and extended features based on the extended part are determined. The extended features refer to other features of the extended part except the target features, and the extended features are inserted into the target feature set to obtain the target feature set.

[0061] By expanding the features of the insufficient parts based on big data technology, the completeness of the target feature set can be improved, thereby improving the restoration and completeness of the target reconstruction.

[0062] Furthermore, step five of this application also includes:

[0063] Determining a preset feature library based on the source data set;

[0064] Randomly determining a predetermined number of verification elements based on the preset element library, performing consistency verification on the three-dimensional reconstruction target, and determining the verification consistency;

[0065] If the verification consistency meets a preset threshold, the three-dimensional reconstruction target is stored in a reconstruction file.

[0066] Specifically, after obtaining the three-dimensional reconstruction target, a preset element library based on the source data set is determined. The preset element library contains a variety of inspection elements and can be set according to actual conditions, such as data integrity, data format, data type, image color and other indicators.

[0067] Then, a predetermined number of verification elements are randomly selected from the preset element library, and the predetermined number is set based on the inspection requirements. The three-dimensional reconstructed target is consistency checked according to the inspection elements to obtain the inspection consistency, wherein the purpose of the consistency check is to determine the conformity of the reconstructed target with the source data set, and then determine the restoration degree of the reconstructed target.

[0068] A preset threshold is obtained, wherein the preset threshold includes a consistency index, and the consistency index can be set according to actual conditions, for example, the consistency index is set to 80%. When the verification consistency is greater than the preset threshold, the 3D reconstruction target is stored in a reconstruction file.

[0069] Furthermore, the present application also includes the following steps:

[0070] If the verification consistency does not meet the preset threshold, generating a polyphonic instruction;

[0071] Performing a difference analysis between the preset element library and the three-dimensional reconstruction target, converting and determining a pre-adjustment feature based on the reconstruction state and the reconstruction dimension, wherein the pre-adjustment feature is identified with an adjustment direction and an adjustment step;

[0072] Based on the pre-adjustment feature, the three-dimensional reconstruction target is adjusted and stored in the reconstruction file.

[0073] Specifically, when the verification consistency is less than or equal to the preset threshold, a polyphonic instruction is generated. Then, based on the polyphonic instruction, a difference analysis is performed on the preset element library and the three-dimensional reconstruction target, and then a pre-adjustment feature based on the reconstruction state and the reconstruction dimension is determined according to the difference analysis result, wherein the pre-adjustment feature refers to a feature that needs to be optimized and adjusted, and the pre-adjustment feature is identified with an adjustment direction and an adjustment step, wherein the adjustment direction and the adjustment step are determined according to the difference analysis result, wherein the larger the deviation, the larger the adjustment step.

[0074] Finally, the three-dimensional reconstruction target is adjusted according to the adjustment direction and the adjustment step in the pre-adjustment feature to obtain an adjusted three-dimensional reconstruction target, and the adjusted three-dimensional reconstruction target is stored in the reconstruction file.

[0075] By performing consistency check on the three-dimensional reconstruction target based on the preset element library of the source data set and optimizing and adjusting the three-dimensional reconstruction target that does not meet the consistency index, the accuracy and quality of the three-dimensional reconstruction target can be further improved.

[0076] In summary, the indoor 3D reconstruction method based on feature descriptors provided in this application has the following technical effects:

[0077] 1. By combining feature descriptor technology to extract features, and performing multimodal fitting and dimensional conversion on the extracted feature descriptors, a reconstruction feature set is obtained, which can achieve the technical goal of improving the precision, accuracy and comprehensiveness of reconstruction feature data selection. Finally, in the reconstruction space, three-dimensional reconstruction operations are performed based on the reconstruction feature set, which can achieve the technical effect of improving the target three-dimensional reconstruction accuracy and quality.

[0078] 2. By expanding the features of the insufficient parts based on big data technology, the completeness of the target feature set can be improved, thereby improving the restoration and completeness of the target reconstruction.

[0079] 3. By performing consistency check on the three-dimensional reconstruction target based on the preset element library of the source data set, and optimizing and adjusting the three-dimensional reconstruction target that does not meet the consistency index, the accuracy and quality of the three-dimensional reconstruction target can be further improved.

[0080] Embodiment 2

[0081] Based on the same inventive concept as the indoor 3D reconstruction method based on feature descriptors in the aforementioned embodiment, the present application also provides an indoor 3D reconstruction system based on feature descriptors, see the attached Figure 3 , the system comprising:

[0082] A source data set reading module 11, wherein the source data set reading module 11 is used to read a source data set of a reconstruction target, wherein the source data set includes multiple data modalities and multiple data dimensions, and has a homology mapping identifier;

[0083] A preprocessing source data determination module 12, the preprocessing source data determination module 12 is used to pre-process the source data set based on a preprocessing model to determine the preprocessing source data, wherein the preprocessing model includes parallel preprocessing branches corresponding to each data modality;

[0084] A feature descriptor determination module 13, the feature descriptor determination module 13 is used to perform multimodal data feature extraction based on the feature extraction module on the preprocessed source data, and determine a feature descriptor, wherein the feature descriptor has a modality identifier;

[0085] A reconstruction feature set determination module 14, the reconstruction feature set determination module 14 is used to perform multimodal fitting and dimensional conversion on the feature descriptor to determine a reconstruction feature set;

[0086] The three-dimensional reconstruction module 15 is used to determine a reconstruction space based on a visual simulation platform, perform a three-dimensional reconstruction operation on the reconstruction target based on the reconstruction feature set, and obtain a three-dimensional reconstruction target.

[0087] Furthermore, the pre-processing source data determination module 12 in the system is also used for:

[0088] Identify the reconstruction criteria and set the image size and cell division size based on the reconstruction accuracy;

[0089] Based on the image size and the cell division size, in combination with the image preprocessing branch, the image modality data is subjected to aspect ratio adjustment and cell division to determine a one-step preprocessing image;

[0090] The one-step preprocessed image is subjected to grayscale processing and local enhancement processing to determine preprocessed image data, and is added to the preprocessed source data.

[0091] Further, the reconstruction feature set determination module 14 in the system is also used for:

[0092] Based on the reconstruction standard, determine the reconstruction dimension and reconstruction state;

[0093] Based on the remodeled modality, activating a modal conversion unit, performing modal conversion processing on the feature descriptor, and determining a conversion feature set, wherein the conversion feature set is identified by a mapping angle;

[0094] The mapping angle is identified, the dimension of the conversion feature set is converted, multi-angle fusion reconstruction of the same feature is performed, and the target feature set is determined.

[0095] Further, the reconstruction feature set determination module 14 in the system is also used for:

[0096] If the target feature set does not conform to the reconstructed dimension, determining a dimension processing instruction, wherein the dimension processing includes a dimension increase processing and a dimension decrease processing;

[0097] Based on the dimension processing instruction, the dimension processing unit is activated to process the fused feature set to determine the target feature set. The dimension processing unit can perform bidirectional processing between high dimension and low dimension.

[0098] Further, the reconstruction feature set determination module 14 in the system is also used for:

[0099] Verify whether the target feature set meets the reconstruction standard, and if not, locate the extended part;

[0100] Performing big data mining to determine extended features based on the extended portion;

[0101] The extended features are interspersed into the target feature set.

[0102] Furthermore, the three-dimensional reconstruction module 15 in the system is also used for:

[0103] Determining a preset feature library based on the source data set;

[0104] Randomly determining a predetermined number of verification elements based on the preset element library, performing consistency verification on the three-dimensional reconstruction target, and determining the verification consistency;

[0105] If the verification consistency meets a preset threshold, the three-dimensional reconstruction target is stored in a reconstruction file.

[0106] Furthermore, the three-dimensional reconstruction module 15 in the system is also used for:

[0107] If the verification consistency does not meet the preset threshold, generating a polyphonic instruction;

[0108] Performing a difference analysis between the preset element library and the three-dimensional reconstruction target, converting and determining a pre-adjustment feature based on the reconstruction state and the reconstruction dimension, wherein the pre-adjustment feature is identified with an adjustment direction and an adjustment step;

[0109] Based on the pre-adjustment feature, the three-dimensional reconstruction target is adjusted and stored in the reconstruction file.

[0110] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The indoor three-dimensional reconstruction method based on feature descriptors and the specific examples in the aforementioned embodiment 1 are also applicable to the indoor three-dimensional reconstruction system based on feature descriptors in this embodiment. Through the aforementioned detailed description of the indoor three-dimensional reconstruction method based on feature descriptors, those skilled in the art can clearly know the indoor three-dimensional reconstruction system based on feature descriptors in this embodiment, so for the sake of brevity of the specification, it will not be described in detail here. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0111] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0112] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technology, the present application is also intended to include these modifications and variations.

Claims

1. A method for indoor 3D reconstruction based on feature descriptors, characterized in that: The method comprises: Reading a source data set of a reconstruction target, wherein the source data set includes multiple data modalities and multiple data dimensions and has a homology mapping identifier; Pre-processing the source data set based on a pre-processing model to determine pre-processed source data, wherein the pre-processing model includes parallel pre-processing branches corresponding to each data modality; Performing multimodal data feature extraction based on a feature extraction module on the preprocessed source data to determine a feature descriptor, wherein the feature descriptor has a modality identifier; Performing multimodal fitting and dimensionality conversion on the feature descriptors to determine a reconstructed feature set; A reconstruction space based on a visual simulation platform is determined, and a three-dimensional reconstruction operation of the reconstruction target is performed based on the reconstruction feature set to obtain a three-dimensional reconstruction target.

2. The method according to claim 1, characterized in that Extracting image modality data in the source data set and pre-processing the source data set, the method further includes: Identify the reconstruction criteria and set the image size and cell division size based on the reconstruction accuracy; Based on the image size and the cell division size, in combination with the image preprocessing branch, the image modality data is subjected to aspect ratio adjustment and cell division to determine a one-step preprocessing image; The one-step preprocessed image is subjected to grayscale processing and local enhancement processing to determine preprocessed image data, and is added to the preprocessed source data.

3. The method according to claim 1, characterized in that Performing multimodal fitting and dimensional conversion on the feature descriptor, the method further includes: Based on the reconstruction standard, determine the reconstruction dimension and reconstruction state; Based on the remodeled modality, activating a modal conversion unit, performing modal conversion processing on the feature descriptor, and determining a conversion feature set, wherein the conversion feature set is identified by a mapping angle; The mapping angle is identified, the dimension of the conversion feature set is converted, multi-angle fusion reconstruction of the same feature is performed, and the target feature set is determined.

4. The method according to claim 3, characterized in that After performing multi-angle fusion of the same feature, the method further includes: If the target feature set does not conform to the reconstructed dimension, determining a dimension processing instruction, wherein the dimension processing includes a dimension increase processing and a dimension decrease processing; Based on the dimension processing instruction, the dimension processing unit is activated to process the fused feature set to determine the target feature set. The dimension processing unit can perform bidirectional processing between high dimensions and low dimensions, wherein the fused feature set refers to feature data obtained after performing multi-angle fusion reconstruction of the same feature.

5. The method according to claim 3, characterized in that After determining the target feature set, the method further includes: Verify whether the target feature set meets the reconstruction standard, and if not, locate the extended part; Performing big data mining to determine extended features based on the extended portion; The extended features are interspersed into the target feature set.

6. The method according to claim 3, characterized in that After acquiring the three-dimensional reconstruction target, the method further includes: Determining a preset feature library based on the source data set; Randomly determining a predetermined number of verification elements based on the preset element library, performing consistency verification on the three-dimensional reconstruction target, and determining the verification consistency; If the verification consistency meets a preset threshold, the three-dimensional reconstruction target is stored in a reconstruction file.

7. The method according to claim 6, characterized in that After determining the verification consistency, the method further includes: If the verification consistency does not meet the preset threshold, generating a polyphonic instruction; Performing a difference analysis between the preset element library and the three-dimensional reconstruction target, converting and determining a pre-adjustment feature based on the reconstruction state and the reconstruction dimension, wherein the pre-adjustment feature is identified with an adjustment direction and an adjustment step; Based on the pre-adjustment feature, the three-dimensional reconstruction target is adjusted and stored in the reconstruction file.

8. Indoor 3D reconstruction system based on feature descriptors, characterized in that: For implementing the steps of the method according to any one of claims 1 to 7, the system comprises: A source data set reading module, wherein the source data set reading module is used to read a source data set of a reconstruction target, wherein the source data set includes multiple data modalities and multiple data dimensions, and has a homology mapping identifier; A preprocessing source data determination module, the preprocessing source data determination module is used to pre-process the source data set based on a preprocessing model to determine the preprocessing source data, wherein the preprocessing model includes parallel preprocessing branches corresponding to each data modality; A feature descriptor determination module, the feature descriptor determination module is used to perform multimodal data feature extraction based on the feature extraction module on the preprocessed source data, and determine a feature descriptor, wherein the feature descriptor has a modality identifier; A reconstruction feature set determination module, the reconstruction feature set determination module is used to perform multimodal fitting and dimensional conversion on the feature descriptor to determine a reconstruction feature set; A three-dimensional reconstruction module is used to determine a reconstruction space based on a visual simulation platform, perform a three-dimensional reconstruction operation on the reconstruction target based on the reconstruction feature set, and obtain a three-dimensional reconstruction target.

Citation Information

Patent Citations

  • Three-dimensional image reconstruction method and device, equipment and storage medium

    CN113554742A

  • Three-dimensional reconstruction model generation method and system, electronic equipment and storage medium

    CN114022639A

  • Virtual-real fusion augmented reality presentation method based on multi-view three-dimensional reconstruction

    CN115063562A

  • Organ three-dimensional reconstruction method and device, electronic equipment and storage medium

    CN116797726A

  • Medical single-view three-dimensional reconstruction device based on prior knowledge guidance

    CN116934970A