A data automatic conversion and fusion method for three-dimensional model components

Abstract

The present application discloses a method for automatic data conversion and fusion of three-dimensional model components, which relates to the technical field of data processing. The above method is based on the image data of the three-dimensional model, extracts the two-dimensional image of the three-dimensional model through the SAM model, segments the two-dimensional image to obtain the first component segmentation result, and then determines the spatial geometric features of the component in the three-dimensional model through the NeRF model based on the image data of the three-dimensional model and the first component segmentation result to obtain the second component segmentation result. After that, the second component segmentation result is mapped to the three-dimensional model, the three-dimensional model is segmented to obtain the components, and the attribute information of the components is determined through the semantic recognition model. Finally, based on the components and the attribute information, the standard data of the components is generated, improving the accuracy of the three-dimensional model component data conversion.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to a method for automatically converting and integrating data of three-dimensional model components. Background Art

[0002] In the construction industry, Building Information Modeling (BIM) can store and manage geometric, attribute, and relationship data of building components. When converting an ordinary three-dimensional model into BIM data, a key step is to identify each component in the three-dimensional model and assign corresponding attribute information to each component.

[0003] Currently, the method for converting a three-dimensional model into BIM data is mainly that professionals disassemble and mark the three-dimensional model according to a rule library and common sense. Then, based on the rule library, they identify and classify the attributes and parameters of different components, and manually add attribute information to each component. However, the above-mentioned rule library is difficult to adapt to all types of three-dimensional models. When facing diversified and complex three-dimensional models, errors are likely to occur in component identification and attribute association, resulting in low accuracy in the conversion of three-dimensional models into BIM data.

[0004] The above content is only used to assist in understanding the technical solution of this application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a method for automatically converting and integrating data of three-dimensional model components, aiming to solve the technical problem of low accuracy in the conversion of standard data of three-dimensional model components.

[0006] To achieve the above purpose, this application proposes a method for automatically converting and integrating data of three-dimensional model components, and the method includes:

[0007] Based on the image data of the three-dimensional model, extract the two-dimensional image of the three-dimensional model through the SAM model, and segment the two-dimensional image to obtain the first component segmentation result;

[0008] Based on the image data of the three-dimensional model and the first component segmentation result, determine the spatial geometric features of the components in the three-dimensional model through the NeRF model to obtain the second component segmentation result;

[0009] Map the second component segmentation result to the three-dimensional model, segment the three-dimensional model to obtain the components, and determine the attribute information of the components through the semantic recognition model;

[0010] Generate the standard data of the components based on the components and the attribute information.

[0011] In one embodiment, the step of extracting the two-dimensional image of the three-dimensional model from the image data based on the three-dimensional model and segmenting the two-dimensional image to obtain the first component segmentation result includes:

[0012] Frame-by-frame extraction of the image data of the three-dimensional model is performed to obtain the two-dimensional images from different perspectives;

[0013] Based on the geometric features of the two-dimensional image, the two-dimensional image is segmented into at least one segmentation region, and each segmentation region corresponds to a component in the three-dimensional model;

[0014] Based on the positions of the segmentation regions in the two-dimensional images from different perspectives, the spatial positions of the segmentation regions are determined;

[0015] Based on the segmentation regions and the spatial positions of the segmentation regions, the first component segmentation result is determined.

[0016] In one embodiment, the step of determining the spatial geometric features of the component in the three-dimensional model based on the image data of the three-dimensional model and the first component segmentation result and obtaining the second component segmentation result includes:

[0017] Based on the image data of the three-dimensional model, the spatial geometric features of the component in the three-dimensional model are determined;

[0018] Based on the first component segmentation result and the spatial geometric features of the component in the three-dimensional model, the second component segmentation result is determined.

[0019] In one embodiment, before the step of determining the second component segmentation result based on the first component segmentation result and the spatial geometric features of the component in the three-dimensional model, the following steps are further included:

[0020] Determine the area of the segmentation region in the first component segmentation result;

[0021] If the area of the segmentation region is less than a preset area threshold, then based on the color and / or texture between the segmentation region and the adjacent region, the similarity between the segmentation region and the adjacent region is determined;

[0022] If the similarity between the segmentation region and the adjacent region is greater than a preset similarity threshold, then the segmentation region and the adjacent region are merged to obtain a new first component segmentation result.

[0023] In one embodiment, the attribute information includes first attribute information, and the step of determining the attribute information of the component through the semantic recognition model includes:

[0024] Determine the initial attribute features of the component based on the segmentation results of the component and the second component;

[0025] Input the initial attribute features into the semantic recognition model;

[0026] Obtain the first attribute information output by the semantic recognition model based on the initial attribute features, where the first attribute information includes the type information and function information of the component.

[0027] In one embodiment, the attribute information includes second attribute information. After the step of obtaining the first attribute information output by the semantic recognition model based on the initial attribute features, the method further includes:

[0028] Input the first attribute information and the image data of the 3D model into the semantic recognition model;

[0029] Determine the second attribute information of the component through the semantic recognition model, where the second attribute information includes the type information, function information, material information, and manufacturing information of the component.

[0030] In one embodiment, after the step of determining the attribute information of the component through the semantic recognition model, the method further includes:

[0031] Obtain the attribute correction information input by the user;

[0032] Input the attribute correction information and the attribute information into the semantic recognition model again;

[0033] Obtain the new attribute information output by the semantic recognition model.

[0034] In one embodiment, the step of generating the standard data of the component based on the component and the attribute information includes:

[0035] Obtain the standard component data attribute table;

[0036] Based on the standard component data attribute table, associate and save the attribute information with the component to generate the standard data of the component.

[0037] The present application provides a method for automatic data conversion and fusion of three-dimensional model components. Based on the image data of the three-dimensional model, the two-dimensional image of the three-dimensional model is extracted through the SAM model, and the two-dimensional image is segmented to obtain the first component segmentation result. Then, based on the image data of the three-dimensional model and the first component segmentation result, the spatial geometric features of the component in the three-dimensional model are determined through the NeRF model to obtain the second component segmentation result. After that, the second component segmentation result is mapped to the three-dimensional model, and the three-dimensional model is segmented to obtain the components, and the attribute information of the components is determined through the semantic recognition model. Finally, based on the components and the attribute information, the standard data of the components is generated.

[0038] By combining the collaborative work of the SAM model and the NeRF model, this method realizes high-precision segmentation of three-dimensional model components, effectively overcoming the problems of inaccurate component recognition and insufficient conversion accuracy faced by traditional rule-based methods when dealing with the complexity and diversity of three-dimensional models. In addition, this method also introduces a semantic recognition model to deeply analyze each component to obtain its detailed attribute information. Finally, based on the segmented components and their attribute information, standard three-dimensional model component data is generated. This process reduces the errors that may be introduced by manual annotation, thus significantly improving the accuracy of three-dimensional model component data conversion when converting three-dimensional models into BIM data. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0040] To more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0041] Figure 1 It is a schematic flowchart provided for Embodiment 1 of a method for automatic data conversion and fusion of three-dimensional model components of the present application;

[0042] Figure 2 It is a schematic flowchart provided for Embodiment 2 of a method for automatic data conversion and fusion of three-dimensional model components of the present application;

[0043] Figure 3 It is a schematic flowchart provided for Embodiment 3 of a method for automatic data conversion and fusion of three-dimensional model components of the present application;

[0044] Figure 4 It is a schematic flowchart of a method for automatic data conversion and fusion of three-dimensional model components in an embodiment of the present application;

[0045] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the data automatic conversion and fusion method for three-dimensional model components in the embodiments of this application.

[0046] The implementation, functional features, and advantages of this application will be further described with reference to the embodiments and the accompanying drawings. Specific implementation manners

[0047] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not used to limit this application.

[0048] To better understand the technical solutions of this application, the following will be described in detail with reference to the drawings in the specification and specific implementation manners.

[0049] In the construction industry, building information models can store and manage geometric, attribute, and relationship data of building components. When converting a common three-dimensional model into BIM data, a key step is to identify each component in the three-dimensional model and assign corresponding attribute information to each component. Currently, the method for converting a three-dimensional model into BIM data is mainly that professionals disassemble and label the three-dimensional model according to a rule library and common sense, and then, based on the rule library, identify and classify the attributes and parameters of different components, and manually add attribute information to each component. However, the above-mentioned rule library is difficult to adapt to all types of three-dimensional models. When facing diversified and complex three-dimensional models, errors are likely to occur in component identification and attribute association, resulting in a low accuracy of converting the three-dimensional model into BIM data.

[0050] In view of the above problems, this application proposes a data automatic conversion and fusion method for three-dimensional model components. Based on the image data of the three-dimensional model, the two-dimensional image of the three-dimensional model is extracted through the SAM model, and the two-dimensional image is segmented to obtain the first component segmentation result. Then, based on the image data of the three-dimensional model and the first component segmentation result, the spatial geometric features of the components in the three-dimensional model are determined through the NeRF model to obtain the second component segmentation result. After that, the second component segmentation result is mapped to the three-dimensional model, and the three-dimensional model is segmented to obtain the components, and the attribute information of the components is determined through the semantic recognition model. Finally, based on the components and the attribute information, the standard data of the components is generated.

[0051] Through the collaborative work of the SAM model and the NeRF model, this method achieves high-precision segmentation of 3D model components, effectively overcoming the problems of inaccurate component recognition and insufficient conversion accuracy faced by traditional rule-based methods when dealing with the complexity and diversity of 3D models. In addition, this method also introduces a semantic recognition model to deeply analyze each component to obtain its detailed attribute information. Finally, based on the segmented components and their attribute information, 3D model component data that meets the standards is generated. This process reduces the errors that may be introduced by manual annotation, thus significantly improving the accuracy of generating standard data for 3D model components when converting 3D models into BIM data.

[0052] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of implementing the above functions. Hereinafter, taking the 3D model component standard data generation device as an example, this embodiment and the following embodiments will be described.

[0053] Based on this, the first embodiment proposed in this application provides a data automatic conversion and fusion method for 3D model components. Referring to Figure 1 , in this embodiment, the data automatic conversion and fusion method for 3D model components includes steps S10 to S40:

[0054] Step S10, based on the image data of the 3D model, extract the 2D image of the 3D model through the SAM model and segment the 2D image to obtain the first component segmentation result.

[0055] It should be noted that SAM is short for Segment Anything Model, which is an image segmentation model that can automatically identify and segment objects in images. In this embodiment, the SAM model is used to extract 2D images from the 3D model and accurately segment these 2D images to initially identify different components in the 3D model. The image data of the 3D model refers to the digital model data containing spatial information such as buildings and objects generated by 3D modeling software or obtained through scanning, including information such as the appearance, shape, and texture of the 3D model. The 2D image refers to the 2D projection image generated from different perspectives of the 3D model. The first component segmentation result includes geometric information such as the boundaries, shapes, and textures of each component in the 2D image. For example, it can be a 3D line mask layer, where the lines represent the boundaries of the components, and the attributes such as the transparency and color of the mask layer represent the texture color information of the components.

[0056] Exemplarily, first, preprocess the input 3D model, including format conversion and coordinate system unification. For example, convert the 3D model into a format that the SAM model can receive, ensuring that all model data is in a unified coordinate system, etc. Then, use the SAM model to extract 2D images from the preprocessed 3D model. Specifically, project the 3D model onto different planes to obtain 2D images from different angles. For example, project the 3D model along the X, Y, and Z axes. Each projection can be regarded as a 2D image observed from a specific perspective. Then, use the SAM model to segment the input 3D model from at least three mutually orthogonal perspectives to identify different components in the 2D images.

[0057] Optionally, step S10 includes steps S11 to S14:

[0058] In step S11, extract the image data of the 3D model frame by frame to obtain the 2D images from different perspectives.

[0059] Exemplarily, input the image data of the 3D model into the SAM model. When the SAM model processes continuous or dynamic 3D model image data, it splits it into independent, static 2D image frames to generate a series of 2D images from different perspectives. These 2D images retain key information in the 3D model, such as the shape, size, and position of objects, etc.

[0060] Optionally, first generate 2D views of the 3D model from different perspectives through 3D modeling software, and then input the 2D views of the 3D model from different perspectives into the SAM model.

[0061] Exemplarily, in 3D modeling software, simulate perspectives in different directions by setting the position and orientation of the camera. According to the perspective requirements, adjust parameters such as the focal length, field of view, position, and orientation of the camera. Then, render the 3D model into a 2D view according to the set perspective and camera parameters. During the rendering process, different parameters such as rendering quality, lighting effects, and texture mapping can be selected to generate high-quality 2D views and improve the accuracy of SAM model segmentation.

[0062] In step S12, based on the geometric features of the 2D image, segment the 2D image into at least one segmentation region, and the segmentation region corresponds to a component in the 3D model.

[0063] It should be noted that the geometric features of a two-dimensional image refer to the attributes such as the shape, size, position, and orientation of components in the two-dimensional image. From two-dimensional images of different perspectives, the relative positions or spatial relationships between different components can also be reflected. After inputting the image data of the three-dimensional model into the SAM model, the SAM model will perform segmentation based on the geometric features of the components in the image data and the two-dimensional images of different perspectives, obtaining a preliminary first component segmentation result, and the first component segmentation result includes the boundaries and positions of each component.

[0064] Exemplarily, the SAM model first identifies the geometric features in the two-dimensional image, including edges, textures, corners, and contours, etc. For example, the Canny edge detection algorithm is used to identify the edge contours in the image, and the Harris corner detection algorithm is used to identify the corners in the image, etc. Then, based on the above geometric features, the two-dimensional image is segmented into multiple segmentation regions. For example, starting from the corners, the image is segmented into multiple segmentation regions along the contour lines, and each region corresponds to an object or an object in the image, that is, a component in the three-dimensional model.

[0065] Step S13: Determine the spatial position of the segmentation region based on the position of the segmentation region in the two-dimensional images of different perspectives.

[0066] Step S14: Determine the first component segmentation result based on the segmentation region and the spatial position of the segmentation region.

[0067] It should be noted that when the objects in the three-dimensional model are projected onto the two-dimensional image by the camera, the relative position relationship between the objects will be maintained. Therefore, the spatial position relationship of the components in the three-dimensional model corresponding to the segmentation region can be inferred based on the position relationship of the segmentation region in the two-dimensional images of different perspectives.

[0068] Optionally, determine the geometric features in each two-dimensional image, determine the similarity of the geometric features of each segmentation region in different two-dimensional images through feature descriptors and / or matching algorithms, and based on this similarity, determine the feature points of the same three-dimensional model component corresponding to the segmentation region in the two-dimensional images of different perspectives. Analyze the relative position relationship of the feature points corresponding to each three-dimensional model component in the images of different perspectives, and merge the segmentation regions corresponding to the same three-dimensional model component in the two-dimensional images of different perspectives according to the relative position relationship.

[0069] Exemplarily, in the two-dimensional images A and B from different perspectives, both contain rectangular window components. The geometric features such as the edges, textures, and corner points of the window components in the two-dimensional images A and B are extracted through the SAM model, and the geometric features of the window components in the two two-dimensional images are respectively converted into vector forms through feature descriptors. The similarity of the geometric features of the window components in the two-dimensional images A and B is determined by calculating the Euclidean distance between these two vectors. If the similarity is greater than the similarity threshold, it is determined that the window components in the two-dimensional images A and B correspond to the same window component in the three-dimensional model, and according to the relative positions of the components in the two-dimensional image, the position of the window component in the three-dimensional model is determined to obtain the segmentation result of the window component.

[0070] Optionally, after receiving the two-dimensional image as input, the SAM model can also receive prompt information such as points, boxes, and texts input by the user to specify the target to be segmented. Specifically, the SAM model first uses the image encoder to convert the original two-dimensional image into a series of image embeddings for generating and evaluating the segmentation mask. At the same time, the prompt encoder encodes the prompt information provided by the user, such as points, boxes, texts, etc., into prompt vectors. The mask decoder combines the image embeddings and the prompt vectors, analyzes the geometric features of the components in the image through deep learning algorithms, and combines the positional relationships of the components in different two-dimensional images, and uses the self-attention mechanism and cross-attention mechanism of the Transformer (transformer model architecture) to fuse the image content and the user's intention, so as to determine the accurate boundaries of each component. Finally, the SAM model generates a segmentation mask, which is used to represent the probability that each pixel in the image belongs to the foreground or the background. Among them, the foreground pixels usually correspond to the components in the image, while the background pixels correspond to the parts outside the components. Therefore, by performing binary processing on the segmentation mask, such as setting the foreground pixels to 1 and the background pixels to 0, a binary image containing only the components can be obtained. Further, by processing the binary image such as contour extraction, each component in the image is recognized and extracted.

[0071] Optionally, a two-dimensional image dataset containing the target components is collected in advance, and the components on the two-dimensional images are labeled. The SAM model is trained using the labeled two-dimensional image dataset so that the SAM model learns the geometric features and segmentation rules of the target components. The trained SAM model is applied to the image to be segmented, and a segmentation mask is generated according to the prompt provided by the user. Post-processing such as morphological processing and boundary smoothing is performed on the segmentation result to improve the segmentation accuracy and visual effect.

[0072] Optionally, according to the grayscale histogram or color histogram of the two-dimensional image, determine the corresponding grayscale value threshold or color value threshold, and segment the two-dimensional image based on the grayscale value threshold or color value threshold. Exemplarily, according to the grayscale histogram or color histogram of the image, select a suitable threshold, compare the grayscale value or color value of each pixel in the image with the threshold, and classify the pixels greater than the threshold as the foreground and the pixels less than the threshold as the background.

[0073] In practical applications, an appropriate segmentation method can be selected according to specific requirements and image characteristics. For the component segmentation task in complex scenarios, the deep learning segmentation method combined with the SAM model can more effectively utilize the geometric features and positional relationships of two-dimensional images and more accurately identify and segment the components in the image.

[0074] Step S20, based on the image data of the three-dimensional model and the first component segmentation result, determine the spatial geometric features of the component in the three-dimensional model through the NeRF model to obtain the second component segmentation result.

[0075] It can be understood that the first component segmentation result is obtained through two-dimensional image segmentation, mainly focusing on the surface features in the image, such as the boundaries, textures, and positions of the components. These surface features are crucial for understanding and identifying the components in the three-dimensional model and can provide rich information about the shape and appearance of the components. However, two-dimensional image segmentation has limitations in mining the three-dimensional information of components. Since a two-dimensional image is only a projection of the three-dimensional space, it cannot directly provide accurate information about three-dimensional attributes such as the size and depth of the components. This means that although the first component segmentation result can well reflect the features of the components on the two-dimensional plane, in the three-dimensional space, the accuracy of this information may be insufficient. Therefore, through the NeRF model, based on the image data of the three-dimensional model and the first component segmentation result, further analyze the spatial geometric features of the components in the three-dimensional model to improve the accuracy of component segmentation.

[0076] It should be noted that NeRF, short for Neural Radiance Fields, is used for 3D scene reconstruction and rendering. In this embodiment, the NeRF model can utilize multi-view 2D image information and the first component segmentation result to reconstruct components in the 3D scene and determine the geometric features of each component in 3D space. The spatial geometric features refer to features such as the position, shape, size, and orientation of components in 3D space. For example, by inputting 2D images from multiple viewpoints into the NeRF model, the NeRF model will reconstruct components such as walls, doors, and windows in the 3D scene, and then combine the first component segmentation result to determine the position, size, and shape of these components in the 3D model. The second component segmentation result is a 3D dataset or model including points, lines, surfaces, or volume pixels, such as a 3D line drawing that can show the internal component structure, and is used to define the spatial position and boundary of components in the 3D model.

[0077] Exemplarily, the NeRF model is trained using a pre-annotated dataset. Different types of 3D model image data and corresponding first component segmentation results are obtained in advance. The image data is preprocessed, such as denoising, enhancing contrast, adjusting the size, etc., to improve the training effect. At the same time, the first component segmentation result is adjusted and optimized to ensure its match with the image data. The processed image data is used as the input of the NeRF model, and the NeRF model will utilize this image data to learn the geometric features of components in 3D space. Through iterative training, the NeRF model will gradually learn how to extract 3D spatial geometric features from the image data. During the training process, the parameters of the model, including the learning rate, batch size, regularization parameter, etc., are continuously adjusted, and at the same time, gradient descent algorithms, momentum, Adam (Adaptive Moment Estimation) optimizers, etc. are used to accelerate the training process and improve the accuracy and efficiency of the segmentation result. For example, by calculating metrics such as intersection over union, precision, and recall, to determine the error between the second component segmentation result and the real 3D model components, so as to evaluate the accuracy of the NeRF model. If the error is greater than the preset error threshold, the network structure of the NeRF model is adjusted, such as increasing or decreasing the number of layers, changing the activation function, etc., to improve the model performance.

[0078] Optionally, step S20 includes steps S21 to S22:

[0079] Step S21, based on the image data of the 3D model, determine the spatial geometric features of the component in the 3D model.

[0080] Exemplarily, load the trained NeRF model, convert the image data obtained from multiple perspectives of the 3D model into a format that can be correctly read and processed by the NeRF model, and then input the prepared image data of the 3D model into the NeRF model as the input data of the model.

[0081] Optionally, based on the input image data of the 3D model, according to the complexity of the 3D model and the distribution of components, the NeRF model selects appropriate perspectives for rendering, sets rendering parameters such as resolution, lighting conditions, camera parameters, etc., and uses the volume rendering ability of the NeRF model to generate rendered images from different perspectives, and determines the spatial geometric features of the components by comparing the rendered images from different perspectives.

[0082] Optionally, extract feature points such as corner points and edges from the image data through the NeRF model, and determine the spatial position and orientation of the components by matching the feature points from different perspectives. Specifically, use a feature point detection algorithm or a deep learning algorithm to detect feature points such as corner points and edges in the rendered image of each perspective, and match the feature points from different perspectives through a feature point matching algorithm such as FLANN (Fast Library for Approximate Nearest Neighbors), to establish the correspondence between the feature points, and calculate the spatial position and orientation of the components according to the matched feature points. For example, after obtaining the matched feature point pairs, calculate the coordinates of the matched points in 3D space according to the projection relationship of the matched feature points in multiple perspectives, restore the 3D structure of the component, and then determine the spatial position of the component by calculating the centroid or other geometric centers of the component, or analyze the feature vectors such as normal vectors and main directions in the 3D structure of the component to determine the orientation and other spatial geometric features of the component.

[0083] Step S22, determine the second component segmentation result based on the first component segmentation result and the spatial geometric features of the component in the 3D model.

[0084] Fuse the first component segmentation result and the spatial geometric features of the component in the 3D model, and use the spatial geometric features output by the NeRF model to further optimize and adjust the first component segmentation result to obtain a more accurate second component segmentation result. For example, train a deep learning model to optimize the segmentation result. This model can receive the first component segmentation result and the spatial geometric features of the component as inputs and output a more accurate second component segmentation result. Or incorporate this deep learning model into the NeRF model, and the NeRF model finally outputs the second component segmentation result.

[0085] Exemplarily, according to the positional relationship of each component in the first component segmentation result, compare it with the spatial geometric features output by the NeRF model to determine the segmentation region and parameters that need to be optimized. Select optimization algorithms such as support vector machines, random forests, and convolutional neural networks to adjust the first component segmentation result according to the spatial geometric features of the components, and iterate the optimization process to make the second component segmentation result conform to the spatial geometric features of each component in the actual 3D model.

[0086] Optionally, after obtaining the second component segmentation result, post-process the second component segmentation result, including removing noise, filling holes, and merging small regions, etc., to obtain a more accurate segmentation result.

[0087] Exemplarily, in the second component segmentation result, there may be some small and irregular regions generated due to errors or interferences in the image acquisition and processing process. These regions are regarded as noise. The step of removing noise is to identify and delete these small and irregular regions to reduce misjudgment and interference in the segmentation result. For the possible holes or missing regions in the second component segmentation result, fill these missing regions through interpolation methods, etc., to make the segmentation result more complete and continuous.

[0088] Through the above steps, the segmentation result of the first component can be fused with the spatial geometric features of the component in the 3D model, and the fused information can be used for segmentation optimization to improve the accuracy and precision of the segmentation result.

[0089] Step S30: Map the second component segmentation result to the 3D model, segment the 3D model to obtain the component, and determine the attribute information of the component through the semantic recognition model.

[0090] Align the three-dimensional coordinates of the second component segmentation result with the original 3D model, accurately locate the second component segmentation result at the corresponding position in the 3D model, segment the 3D model to obtain each component in the 3D model, and further analyze other attribute information of the component through the semantic recognition model according to the size, shape, texture, and position features of the component.

[0091] Optionally, the attribute information includes first attribute information. Step S30 includes steps S31 to S33:

[0092] Step S31: Based on the component and the second component segmentation result, determine the initial attribute features of the component.

[0093] It should be noted that the initial features of the component refer to the attribute features of the component that can be directly obtained from the image data of the 3D model and / or the second component segmentation result, such as the shape and size features, texture features, color features, and position features of the component.

[0094] Step S32: Input the initial attribute features into the semantic recognition model.

[0095] Take the initial attribute features of the components that can be directly obtained from the image data of the 3D model and / or the segmentation results of the second components as input and pass them to the semantic recognition model. Among them, the semantic recognition model is a trained machine learning model or deep learning network that can analyze and reason about the input initial attribute features to identify the semantic information of the components.

[0096] Step S33: Obtain the first attribute information output by the semantic recognition model based on the initial attribute features. The first attribute information includes the type information and function information of the component.

[0097] The semantic recognition model analyzes and reasons based on the input initial attribute features and then outputs the first attribute information. The first attribute information includes the type information and function information of the component. Among them, the type information refers to the category or type of the component, such as doors, windows, columns, beams, etc. These information help to classify the components into specific building categories. The function information refers to the function or use of the component, such as load-bearing, sound insulation, ventilation, decoration, etc.

[0098] Optionally, the attribute information further includes the second attribute information. After step S33, the data automatic conversion and fusion method for the 3D model components further includes steps S34 - S35:

[0099] Step S34: Input the first attribute information and the image data of the 3D model into the semantic recognition model.

[0100] Step S35: Determine the second attribute information of the component through the semantic recognition model. The second attribute information includes the type information, function information, material information, and manufacturing information of the component.

[0101] To more comprehensively understand the attribute information of the component, input the first attribute information previously obtained from the initial attribute features and the image data of the 3D model into the semantic recognition model together. Combining the first attribute information and the overall 3D model, the semantic recognition model analyzes and reasons based on the input first attribute information and the image data of the 3D model, and can more accurately identify the type and function of the component, and further infer the material and manufacturing information of the component. Among them, the second attribute information is an extension based on the first attribute information, and not only includes the type information and function information of the component, but also includes the material information and manufacturing information. Among them, the material information represents the type of material used for the component, such as wood, metal, plastic, glass, etc., and the manufacturing information represents the information in aspects such as the manufacturing process, connection method, and processing accuracy of the component.

[0102] Optionally, collect the BIM data sets of standard components, which include the 3D models, image data, and corresponding detailed attribute information of the components. Among them, other attribute information to be recognized can be adjusted according to the actual application requirements, such as the physical attribute information, usage and maintenance information, etc. Preprocess the collected data, including data cleaning, noise removal, normalization, etc., to ensure the quality and consistency of the data. At the same time, label the data and assign the correct attribute information labels to each component. Then, according to the task requirements and data characteristics, select a suitable machine learning model or deep learning network as the semantic recognition model. For example, a convolutional neural network can be used to process the image data, and a recurrent neural network or a graph neural network can be used to process the geometric and topological information of the components. Next, design the specific structure of the semantic recognition model, including the input layer, hidden layer, output layer, etc. The input layer is used to receive the initial attribute features, the first attribute information, and the image data of the 3D model; set the neurons and layers of the hidden layer to capture the complex features of the data; the output layer is used to output the detailed attribute information of the component. Input the preprocessed standard component data set into the semantic recognition model for training. During the training process, continuously adjust the parameters of the semantic recognition model to minimize the error between the prediction result and the true label, and obtain the trained semantic recognition model. In actual applications, input the initial attribute features and image data of the 3D model components to be recognized into the trained semantic recognition model, and the semantic recognition model will perform reasoning based on the input initial attribute features and image data and output the detailed attribute information of the components.

[0103] Step S40: Generate the standard data of the component based on the component and the attribute information.

[0104] After obtaining the segmented components and the attribute information of the components, integrate this information into a unified data structure according to the preset data standards and formats. This data structure can be a database table, an XML (eXtensible Markup Language) file, or other data containers in a specific format, and then save it as a data format that conforms to specific specifications or standards for easy data exchange, sharing, and application.

[0105] Optionally, step S40 includes steps S41 - S42:

[0106] Step S41: Obtain the standard component data attribute table.

[0107] Step S42: Based on the standard component data attribute table, associate and save the attribute information with the component to generate the standard data of the component.

[0108] It should be noted that the standard component data attribute table is a table or data structure that defines all the attribute information and its format of standard components, serving as the basis for subsequent data integration and standardization. The standard component data attribute table includes information such as attribute names, data types, value ranges, etc., which can be used to verify and format the input attribute information.

[0109] Exemplarily, according to open standards of industrial standards or other BIM-related standards, define a set of standard attributes for components, including: basic attributes such as component identifiers, names, types, etc., which are used to identify and distinguish components; geometric attributes such as dimensions (length, width, height, etc.), coordinates, shapes, etc., which describe the geometric characteristics of components; physical attributes such as materials, weights, strengths, etc., which describe the physical properties of components; functional attributes such as uses (such as load-bearing walls, soundproof walls, etc.), performances (such as insulation performance, fire resistance performance, etc.), etc., which describe the functions and performances of components. Create a three-dimensional model database, establish indexes and links to associate relevant component and attribute information, and then export the information in the database as a standard-compliant data file according to BIM standards (such as IFC standards). Through the above steps, it is possible to accurately associate the identified attribute information to the corresponding components and generate a data file that complies with BIM standards, providing strong support for the informatization management and intelligent application of construction projects.

[0110] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 2 , before step S22, the data automatic conversion and fusion method for three-dimensional model components further includes steps S23 to S25:

[0111] Step S23, determine the area of the segmentation region in the first component segmentation result.

[0112] It should be noted that there may be some regions with too small area or inaccurate segmentation in the first segmentation result, which are caused by noise, edge effects or limitations of the segmentation algorithm itself. For the segmentation regions with too small area, if they are highly similar to adjacent regions in terms of color and / or texture, then they belong to the same component. Merging these similar regions into one segmentation region can improve the continuity and integrity of the segmentation result and reduce segmentation fragments.

[0113] Exemplarily, in a three-dimensional model, a segmentation region may represent a wall or a window. If the area of a certain segmentation region is very small, it may indicate inaccurate segmentation or the presence of noise. Traverse all the segmentation regions in the first component segmentation result and calculate the area of each region.

[0114] Step S24. If the area of the segmentation region is smaller than a preset area threshold, then based on the color and / or texture between the segmentation region and the adjacent region, determine the similarity between the segmentation region and the adjacent region.

[0115] Compare the area of each segmentation region with a preset area threshold to obtain a list of segmentation regions whose areas are all smaller than the preset area threshold. For each small-area segmentation region, determine its adjacent region, where the adjacent region is another segmentation region that is spatially adjacent to the current segmentation region. For each small-area segmentation region and its adjacent region, extract color and / or texture features.

[0116] Among them, the color feature can represent the color distribution of the region through statistical methods such as color histograms and color moments. The texture feature can extract the texture information of the region through algorithms such as local binary patterns and gray-level co-occurrence matrices. Calculate the color similarity or texture similarity between the segmentation region and the adjacent region through methods such as Euclidean distance and cosine similarity, or use weighted average or other combination methods to combine the color similarity and texture similarity into a comprehensive similarity value.

[0117] Step S25. If the similarity between the segmentation region and the adjacent region is greater than a preset similarity threshold, then merge the segmentation region and the adjacent region to obtain a new segmentation result of the first component.

[0118] If the color similarity and / or texture similarity between the segmentation region and the adjacent region is greater than or equal to the similarity threshold, it is considered that they belong to the same component, and the segmentation region and the adjacent region are merged.

[0119] In this embodiment, by calculating the area of each segmentation region in the segmentation result of the first component, identify regions with too small area or inaccurate segmentation. And by evaluating the similarity of these small-area segmentation regions and adjacent regions in terms of color and / or texture, determine whether they belong to the same component. If it is determined that they belong to the same component, then merge them to enhance the continuity and integrity of the segmentation result.

[0120] Based on the above embodiments of the present application, in the third embodiment of the present application, for the same or similar content as the above embodiments, reference can be made to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 3 , after step S30, the data automatic conversion and fusion method of the three-dimensional model component further includes steps S50 to S70:

[0121] Step S50. Obtain the attribute correction information input by the user.

[0122] It should be noted that the attribute correction information is the correction or supplementary information provided by the user for the errors or inaccurate information in the attribute information output by the model. The attribute correction information is the feedback of the user on the attribute information output by the model, aiming to improve the accuracy and integrity of the attribute information.

[0123] Exemplarily, a user interaction interface is created to allow the user to input attribute correction information through means such as the interface, keyboard, voice, etc. These information include various forms such as text, pictures, voice, etc.

[0124] Step S60, input the attribute correction information and the attribute information into the semantic recognition model again.

[0125] Step S70, obtain the new attribute information output by the semantic recognition model.

[0126] The attribute correction information input by the user and the original attribute information are input into the semantic recognition model together for a new round of recognition and processing. The semantic recognition model can re-parse and recognize attributes according to the new input information and output more accurate and complete attribute information.

[0127] Optionally, the semantic recognition model can directly output the new attribute information for the user to view or use. Or output the original attribute information and the newly generated modified attribute information for the user to compare and view. For the attribute information in the image or 3D model, the new recognition results are displayed through visualization means such as annotation and highlighting.

[0128] In this embodiment, the user can provide corrections to the attribute information output by the model through various means such as interface input, keyboard input, or voice input. The attribute correction information input by these users is input into the semantic recognition model together with the original attribute information for a new round of recognition and processing. The semantic recognition model can re-parse and recognize the attributes of the components according to these new input information and output more accurate and complete attribute information. The above steps enable the semantic recognition model to better adapt to different usage scenarios and user requirements and improve the accuracy of the output of the semantic recognition model.

[0129] Exemplarily, to help understand the implementation process of a data automatic conversion and fusion method for 3D model components obtained by combining this embodiment with the above Embodiment 1, please refer to Figure 4 , Figure 4 A brief flow schematic diagram of a data automatic conversion and fusion method for 3D model components is provided. Specifically:

[0130] Input a 3D model, perform quality inspection and preprocessing on the input 3D model to ensure the accuracy and availability of the model data. The preprocessed model is converted into image data to prepare for the subsequent steps. Select a specific perspective and transform the image data to generate 2D views of the multi-perspective 3D model. The generated 2D views will be used for subsequent segmentation and recognition. Use the SAM model to segment the 2D views to obtain the first component segmentation result. Then use the NeRF model to determine the spatial geometric features of the components. Combine the first component segmentation result and the spatial geometric features of the components to obtain a more accurate second component segmentation result. Map the second component segmentation result back to the 3D model to obtain recognizable standard 3D model components. Extract the initial attribute information from the segmented components, use the semantic recognition model to analyze the initial attribute information to obtain the first attribute information, and then combine the first attribute information with the image data of the 3D model for re-analysis to obtain the second attribute information. Correct and map the second attribute information to conform to the attributes of the BIM standard data structure, and combine the standard component data attribute table to associate the attribute information with the components to generate the standard data of the 3D model components. Among them, the segmented standard 3D model components can be recombined as a new 3D model for re-segmentation processing. The attribute information output by the semantic recognition model can also be repeatedly input into the semantic recognition model for repeated analysis to improve the accuracy of attribute information recognition.

[0131] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on a method for automatically converting and fusing data of 3D model components in this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0132] This application also provides a device for automatically converting and fusing data of 3D model components. The device for automatically converting and fusing data of 3D model components includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for automatically converting and fusing data of 3D model components in the first embodiment above.

[0133] The following refers to Figure 5 , which shows a schematic structural diagram of a device for automatically converting and fusing data of 3D model components suitable for implementing the embodiments of this application. The device for automatically converting and fusing data of 3D model components in the embodiments of this application may include, but is not limited to, mobile terminals such as laptop computers and tablet computers (PAD, Portable Application Description) and fixed terminals such as desktop computers. Figure 5The data automatic conversion and fusion device of a 3D model component shown is merely an example, and should not impose any limitations on the functions and application scope of the embodiments of this application.

[0134] As Figure 5 shown, the data automatic conversion and fusion device of a 3D model component may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM, Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM, Random Access Memory) 1004. In the random access memory 1004, various programs and data required for the operation of the data automatic conversion and fusion device of a 3D model component are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, etc.; an output device 1008 including, for example, a liquid crystal display (LCD, Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data automatic conversion and fusion device of a 3D model component to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a data automatic conversion and fusion device of a 3D model component with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be alternatively implemented or had.

[0135] Specifically, according to the embodiments disclosed in this application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in this application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above functions defined in the method of the embodiments disclosed in this application are executed.

[0136] The data automatic conversion and fusion device for three-dimensional model components provided by this application adopts the data automatic conversion and fusion method for three-dimensional model components in the above-mentioned embodiment, and can solve the technical problem of low accuracy in the standard data conversion of three-dimensional model components. Compared with the prior art, the beneficial effects of the data automatic conversion and fusion device for three-dimensional model components provided by this application are the same as those of the data automatic conversion and fusion method for three-dimensional model components provided by the above-mentioned embodiment, and other technical features in this data automatic conversion and fusion device for three-dimensional model components are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0137] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0138] As mentioned above, only the specific implementation manners of this application are described, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0139] This application provides a computer-readable storage medium with computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the data automatic conversion and fusion method for three-dimensional model components in the above-mentioned embodiment.

[0140] The computer-readable storage medium provided by the present application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination of the above.

[0141] The above computer-readable storage medium can be included in the data automatic conversion and fusion device of the three-dimensional model component; or it can exist independently and not be assembled into the data automatic conversion and fusion device of the three-dimensional model component.

[0142] The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by the data automatic conversion and fusion device of the three-dimensional model component, the data automatic conversion and fusion device of the three-dimensional model component can write computer program code for performing the operations of the present application in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++, and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, or executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0144] The modules described in the embodiments of the present application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0145] The readable storage medium provided by the present application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned data automatic conversion and fusion method of three-dimensional model components, and can solve the technical problem of low accuracy in the standard data conversion of three-dimensional model components. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the data automatic conversion and fusion method of three-dimensional model components provided by the above embodiments, and will not be elaborated here.

[0146] The above are only some embodiments of the present application, and do not limit the patent scope of the present application. All equivalent structural transformations made under the technical concept of the present application by using the content of the specification and drawings of the present application, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for automatic conversion and fusion of data of a three-dimensional model component, characterized in that: The method for automatic conversion and fusion of data of the three-dimensional model component comprises: Based on the image data of the three-dimensional model, extracting a two-dimensional image of the three-dimensional model through the SAM model, and segmenting the two-dimensional image to obtain a first component segmentation result; The NeRF model selects a suitable viewing angle for rendering according to the input image data of the three-dimensional model, the complexity of the three-dimensional model and the distribution of components, sets rendering parameters, generates rendering images at different viewing angles using the volume rendering capability of the NeRF model, and determines the spatial geometric features of the components in the three-dimensional model by comparing the rendering images at different viewing angles; Determining a second component segmentation result based on the first component segmentation result and the spatial geometric features of the component in the three-dimensional model; Mapping the second component segmentation result to the three-dimensional model, segmenting the three-dimensional model to obtain the component, and determining the attribute information of the component through a semantic recognition model; Based on the component and the attribute information, standard data of the component is generated.

2. The method for automatic data conversion and fusion of a three-dimensional model component according to claim 1, characterized in that: The step of extracting a two-dimensional image of the three-dimensional model based on the image data of the three-dimensional model through the SAM model and segmenting the two-dimensional image to obtain a first component segmentation result includes: Extracting the image data of the three-dimensional model frame by frame to obtain the two-dimensional images at different viewing angles; Based on the geometric features of the two-dimensional image, segment the two-dimensional image into at least one segmented area, where the segmented area corresponds to a component in the three-dimensional model; Determining a spatial position of the segmented region based on a position of the segmented region in the two-dimensional image at different viewing angles; The first component segmentation result is determined based on the segmented area and the spatial position of the segmented area.

3. The method for automatic data conversion and fusion of a three-dimensional model component according to claim 1, characterized in that: Before the step of determining the second component segmentation result based on the first component segmentation result and the spatial geometric features of the component in the three-dimensional model, the step further includes: Determining the area of ​​the segmented region in the first component segmentation result; If there is an area of ​​the segmented region that is smaller than a preset area threshold, determining the similarity between the segmented region and the adjacent region based on the color and / or texture between the segmented region and the adjacent region; If the similarity between the segmented region and the adjacent region is greater than a preset similarity threshold, the segmented region and the adjacent region are merged to obtain a new first component segmentation result.

4. The method for automatic conversion and fusion of data of a three-dimensional model component according to claim 1, characterized in that: The attribute information includes first attribute information, and the step of determining the attribute information of the component by using a semantic recognition model includes: Determining initial attribute features of the component based on the segmentation results of the component and the second component; Inputting the initial attribute features into the semantic recognition model; The first attribute information output by the semantic recognition model based on the initial attribute feature is obtained, where the first attribute information includes type information and function information of the component.

5. The method for automatic conversion and fusion of data of a three-dimensional model component according to claim 4, characterized in that: The attribute information includes second attribute information, and after the step of obtaining the first attribute information output by the semantic recognition model based on the initial attribute feature, the step further includes: Inputting the first attribute information and the image data of the three-dimensional model into the semantic recognition model; The second attribute information of the component is determined by the semantic recognition model, where the second attribute information includes type information, function information, material information and manufacturing information of the component.

6. The method for automatic conversion and fusion of data of a three-dimensional model component according to claim 1, characterized in that: After the step of determining the attribute information of the component by the semantic recognition model, the method further includes: Get the attribute modification information entered by the user; inputting the attribute correction information and the attribute information into the semantic recognition model again; The new attribute information output by the semantic recognition model is obtained.

7. The method for automatic conversion and fusion of data of a three-dimensional model component according to claim 1, characterized in that: The step of generating standard data of the component based on the component and the attribute information comprises: Get the standard component data attribute table; Based on the standard component data attribute table, the attribute information is associated with the component and saved to generate standard data of the component.

Images