Point cloud generation method and system for deep learning semantic segmentation of building prefabricated components

Abstract

This invention discloses a point cloud generation method and system for deep learning semantic segmentation of prefabricated building components. The method mainly includes: creating structured text for generating prefabricated building component rules; automatically generating standardized prefabricated building component models based on the Revit secondary development environment; and generating point cloud data using computer graphics methods. This invention obtains structured text for prefabricated building components by dividing the document into three parts: prefabricated component category, corresponding dimensions, and geometric features. This improves the readability of building codes and makes them easier for readers to understand. By automatically generating prefabricated building component models that meet building code requirements, it greatly improves work efficiency. This invention can efficiently and automatically generate standardized point cloud data for prefabricated building component models, serving the dimensional and quality inspection of prefabricated building components.

Description

Technical Field

[0001] This invention relates to a technical solution for quality and dimensional inspection of prefabricated building components, and in particular provides a method and system for generating point cloud data for machine semantic segmentation deep learning, aiming to solve the problem of missing point cloud data for prefabricated building components. Background Technology

[0002] Point cloud data is a collection of vectors in a three-dimensional coordinate system. Its main characteristics are high precision, high resolution, and high dimensionality of geometric information, allowing for a visual representation of the shape, surface, and texture of objects in space. In the architectural field, point cloud data is used in architectural design, building monitoring, construction, architectural heritage preservation, and interior design. In building monitoring and construction, regularly acquiring point cloud data allows for the monitoring of changes and damage to buildings, enabling timely maintenance and repair, ensuring the safety and stability of the structures. It also provides construction workers with more accurate dimensional information about building quality, helping them better control and adjust construction equipment during the construction process, improving efficiency and safety.

[0003] Semantic segmentation based on point clouds refers to the semantic segmentation of scenes using point cloud data, used to identify and classify different objects or regions within the point cloud. Machine deep learning algorithms in this area automatically learn the features and semantic information of different objects in point cloud data using deep learning models, and then perform semantic segmentation based on the learned features and information. This field typically uses deep learning-based models and algorithms, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Fully Connected Neural Networks (FCNs), to process point cloud data. These models can extract, learn, and predict features from point cloud data, ultimately achieving semantic segmentation. Furthermore, to achieve better semantic segmentation results, advanced deep learning techniques, such as attention mechanisms, multi-scale feature learning, and transfer learning, can be combined to improve model accuracy. Some deep learning-based methods, such as PointNet, PointNet++, and VoxNet, have been applied to point cloud semantic segmentation tasks and have achieved good results. These methods effectively achieve semantic segmentation of point clouds by processing and learning point cloud data, providing an important foundation for 3D scene understanding and recognition.

[0004] Datasets are fundamental for training and testing machine learning models, determining their performance and generalization ability. Due to the difficulty of point cloud data acquisition and the complexity of data processing, there is currently a lack of point cloud datasets to validate experimental methods for machine deep learning. The difficulty of point cloud data acquisition lies in the fact that it typically requires expensive laser scanners or photogrammetry equipment, and necessitates professional operation, resulting in high costs. The complexity of point cloud data processing stems from the fact that it usually contains a large number of points, requiring complex processing and analysis, as well as specialized software and technical support.

[0005] In existing technologies, point cloud generation for prefabricated building components typically relies on automated conversion using a fixed point cloud density set in the model. While this method achieves a degree of automation, it fails to consider the placement and parameter settings of the scanner in a real-world environment, resulting in point cloud data that cannot accurately simulate actual scanning conditions. When applied to complex architectural scenarios, such methods often lead to data gaps or insufficient accuracy, thus affecting tasks such as quality inspection and dimensional verification. Therefore, existing technologies suffer from the problem of lacking effective simulation of real-world scanning conditions when generating point cloud data, which is precisely the key technical challenge that this invention aims to solve. Summary of the Invention

[0006] Purpose of the invention: This invention proposes a point cloud generation method and system for deep learning semantic segmentation of prefabricated building components, which solves the problems of missing point cloud data for prefabricated building components and the lack of effective simulation of real scanning conditions in existing automated methods.

[0007] Technical Solution: To achieve the above-mentioned objectives, in a first aspect, this invention proposes a point cloud generation method for deep learning semantic segmentation of prefabricated building components, comprising the following steps:

[0008] The building code requirements in PDF format are converted into an unstructured Word document that is easy to edit, and then a completeness review is conducted. Based on the Word document that has been reviewed and found to be error-free, the document content is divided into three parts: prefabricated component category, corresponding dimensions, and geometric features, resulting in a structured text of building prefabricated component rules.

[0009] Based on the structured prefabricated building component rule text, an adaptive family file is created in Revit software, a secondary development environment configuration is built, and a 3D model of the prefabricated building component in STL format is automatically generated based on the configured secondary development environment.

[0010] The automatically generated STL format prefabricated component model is moved to the origin of the world coordinate system. The world coordinate system is then converted to the scanner coordinate system through coordinate system transformation. Based on a consistent coordinate system setting, the three-dimensional coordinates are projected onto a two-dimensional plane, and the depth values ​​in the z-Buffer algorithm are used for filtering to calculate the visible triangular meshes in the STL format prefabricated component model at any scanner position.

[0011] A series of potential scanner positions are generated based on the scene conditions. The visible triangular mesh of the prefabricated component at each scanner position is calculated one by one. The combination of scanner positions that makes the visibility of the prefabricated component reach a specified threshold is selected.

[0012] For each scanning position in the selected combination of scanner positions, point cloud data of the corresponding visible triangular mesh is simulated and generated based on the scanner characteristics and the relative positional relationship between the prefabricated component and the scanner, thereby generating point cloud data of the entire prefabricated component.

[0013] Preferably, a 3D model of prefabricated building components in STL format is automatically generated based on a configured secondary development environment, including:

[0014] Based on the secondary development configuration environment and according to the structured building prefabricated component rule text, the overall length, width and height are determined for each type of prefabricated component and its corresponding size, and the edges of the three-dimensional model are determined.

[0015] Based on the edges of the 3D model, the geometric features are represented by hollow or solid extrusions according to the geometric features in the structured prefabricated building component rule text, and the 3D model of the prefabricated component in RVT format is obtained.

[0016] Convert the RVT format 3D model into an STL format 3D model composed of triangular meshes.

[0017] As a preferred method, the world coordinate system is converted to the scanner coordinate system through coordinate system transformation. The transformation formula is as follows:

[0018] v scani =M ScanTrans ×v worldi

[0019] In the formula, v worldi Let v be the coordinates of the vertex of the i-th triangular mesh on the prefabricated component model in the world coordinate system; scani M represents the coordinates in the scanner's coordinate system after coordinate transformation; Scantrans The transformation matrix is ​​obtained by performing matrix multiplication operations on the rotation matrix and the flip matrix in sequence.

[0020] Preferably, based on a consistent coordinate system setting, the visible triangular meshes in the STL format prefabricated component model at any scanner position are calculated by projecting the three-dimensional coordinates onto a two-dimensional plane and using depth values ​​filtered in the z-Buffer algorithm, including:

[0021] Define a two-dimensional projection plane located between the scanner and the prefabricated component model. This plane is perpendicular to the z-axis of the scanner coordinate system. Determine the projection matrix based on the parameters of the projection plane. Based on the projection matrix, convert the three-dimensional vertex coordinates of the triangular mesh into two-dimensional projection coordinates.

[0022] Each pixel covered by a 3D triangular mesh projected onto a 2D projection plane has its corresponding depth value, called the z-Buffer value. Based on the 2D projection plane, each pixel on the 2D projection plane is assigned an initial z-Buffer value to form an initial depth buffer.

[0023] Based on the initial depth buffer, the z-Buffer value of each 3D triangular mesh on the prefabricated component model is calculated and stored at each pixel point covered by the projection onto the 2D plane. For each pixel point on the projection plane, if the calculated z-Buffer value corresponding to the current triangular mesh is less than the previously recorded z-Buffer value, the z-Buffer value at that pixel point is updated to the calculated value; otherwise, it is retained.

[0024] The two-dimensional visible pixels are determined based on the final z-Buffer value of each pixel on the projection plane. By performing two inverse matrix transformations, the coordinates of the two-dimensional visible pixels retained on the projection plane are restored to three-dimensional point cloud coordinates, thereby filtering out the visible triangular meshes on the three-dimensional model.

[0025] Preferably, the projection matrix is ​​determined based on the parameters of the projection plane, and the calculation method is as follows:

[0026]

[0027] Among them, M ProjTrans The projection matrix is ​​represented; FoV represents the field of view that the scanner can observe through the two-dimensional projection plane; aspect is the ratio of the width to the length of the projection plane; near and far are the two closest and farthest planes that the scanner can detect.

[0028] Based on the projection matrix, the 3D vertex coordinates of the triangular mesh are converted into 2D projected coordinates, represented as:

[0029] V 2Di =M ProjTrans ×v scani

[0030] Among them, v scaniLet V be the vertex coordinates of the i-th triangular mesh in the prefabricated component model as seen from the scanner's perspective. After transformation by the projection matrix, the two-dimensional projected coordinates of this vertex are represented as V. 2Di .

[0031] Preferably, a series of potential scanner positions are generated based on the scene conditions. The visible triangular mesh of the prefabricated component at each scanner position is calculated, and a combination of scanner positions that makes the visibility of the prefabricated component reach a specified threshold is selected, including:

[0032] Based on the spatial constraints of the scene, multiple potential scanning positions are generated and the position of the triangular mesh that can be observed from the perspective of each potential scanner is calculated. The combination of scanner positions that achieves 95% coverage of the prefabricated components is determined by a heuristic algorithm.

[0033] Preferably, point cloud data of the corresponding visible triangular meshes are simulated and generated based on the characteristics of the scanner and the relative positional relationship between the prefabricated components and the scanner, including:

[0034] For each scan point in the selected combination of scanner positions, a two-dimensional plane parallel to the prefabricated component is created. The two-dimensional scan point on the created plane is calculated by generating the relative positional relationship between the scanner and the plane, as well as the parameters set by the scanner.

[0035] For the triangular mesh visible under the current scanner, the three-dimensional point cloud coordinates are simulated one by one by calculating the intersection points of the straight line formed by the scanner and the two-dimensional scanning points with the triangular mesh.

[0036] For each combination, the 3D point cloud coordinates of the corresponding visible triangular mesh are calculated for the scanner position, and finally the point cloud data of the prefabricated building component model is generated.

[0037] Secondly, a point cloud generation system for deep learning semantic segmentation of prefabricated building components includes:

[0038] The document preprocessing module is used to convert the building code requirements in PDF format into an unstructured Word document that is easy to edit, and to perform an integrity review. Based on the reviewed and error-free Word document, the document content is divided into three parts: prefabricated component category, corresponding dimensions, and geometric features, to obtain a structured prefabricated building component rule text.

[0039] The 3D model generation module is used to create adaptive family files in Revit software based on structured prefabricated building component rule text, build a secondary development environment configuration, and automatically generate 3D models of prefabricated building components in STL format based on the configured secondary development environment.

[0040] The triangular mesh detection module is used to move the automatically generated STL format prefabricated component model to the origin of the world coordinate system, and convert the world coordinate system to the scanner coordinate system through coordinate system transformation. Based on a consistent coordinate system setting, the module calculates the visible triangular meshes in the STL format prefabricated component model at any scanner position by projecting the three-dimensional coordinates onto the two-dimensional plane and using the depth value filtering in the z-Buffer algorithm.

[0041] The scanning position combination module is used to generate a series of potential scanner positions according to the scene conditions, calculate the visible triangular mesh of the prefabricated component under each scanner position, and select the scanner position combination that makes the visibility of the prefabricated component reach a specified threshold.

[0042] The point cloud data generation module is used to simulate and generate the point cloud data of the corresponding visible triangular mesh for each prefabricated component in the selected scanner position combination, based on the characteristics of the scanner and the relative positional relationship between the prefabricated component and the scanner, thereby generating the point cloud data of the entire prefabricated component.

[0043] Thirdly, the present invention also provides a computer device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the point cloud generation method for deep learning semantic segmentation of prefabricated building components as described in the first aspect of the present invention.

[0044] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the point cloud generation method for deep learning semantic segmentation of prefabricated building components as described in the first aspect of the present invention.

[0045] Beneficial effects:

[0046] (1) This invention proposes an innovative method for generating point cloud data of prefabricated building components in the field of semantic segmentation technology within computer machine learning. Through secondary development of Revit software, the automated generation of prefabricated building component models of different specifications is achieved. The visible areas of the models are then filtered using the z-Buffer algorithm to accurately extract their surface information. Compared to traditional point cloud data acquisition methods that rely on laser scanning equipment, this solution uses software processing starting from standard documents, generating high-quality point cloud data without the need for expensive scanning equipment, significantly reducing costs and technical barriers.

[0047] (2) The point cloud data of prefabricated building components generated by this invention can be used as a dataset in the training of deep learning semantic segmentation models. It can simulate most common structural features and geometric details to support the effective training of semantic segmentation models. Therefore, the point cloud data generated by this method can not only be used as a standard reference for model training, but also significantly reduce the cost and time of data acquisition, and improve the overall efficiency and feasibility of deep learning projects.

[0048] (3) This invention obtains structured text for prefabricated building components through format conversion and technical review, improving the readability of building codes and making them easier for readers to understand. Based on the obtained structured text of prefabricated building components, a model of prefabricated building components that meets the requirements of building codes is generated automatically, greatly improving work efficiency and making up for the deficiencies of existing datasets, such as missing data and small data volume. Combined with the automatically constructed model of prefabricated building components, point cloud data is generated using graphics methods. Through coordinate system transformation, projection calculations are performed in the scanner coordinate system, and different scanning perspectives are combined to achieve complete scanning of components and effective simulation of scanning conditions. The point cloud data can be used for processing and analysis in complex building scenes, improving construction efficiency and safety. Attached Figure Description

[0049] Figure 1 A flowchart of a point cloud generation method for deep learning semantic segmentation of prefabricated building components;

[0050] Figure 2 This is a flowchart of the automatic generation of 3D models of prefabricated components through BIM secondary development;

[0051] Figure 3 This is a diagram illustrating coordinate system transformation;

[0052] Figure 4 This is a schematic diagram of the two-dimensional projection coordinates of the vertices of a 3D mesh on the projection plane;

[0053] Figure 5 This is a flowchart of filtering visible triangular meshes using the z-Buffer algorithm;

[0054] Figure 6 It is a flowchart of combining different scanning perspectives to achieve a complete scan of the component. Detailed Implementation

[0055] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0056] Precast building components refer to components or modules manufactured in a factory according to design requirements, which are then rapidly assembled on-site using mechanical equipment. Precast components can include various forms such as walls, floor slabs, beams, columns, railings, and bathroom modules. Unlike traditional on-site casting, precast building components can accelerate construction progress, improve construction quality, and save manpower; therefore, high-quality dimensional and structural testing is indispensable.

[0057] Point cloud data refers to a collection of three-dimensional spatial information obtained through 3D scanning technology. It typically consists of multiple three-dimensional points, each composed of its coordinates (x, y, z) in three-dimensional space and possible other attributes (such as color, reflectivity, etc.). In recent years, the application of point cloud data in the construction industry has become increasingly widespread, mainly due to its ability to accurately capture the three-dimensional information of the real world, assisting in various stages of architectural design, construction, and management, such as Building Information Modeling (BIM), site surveying, construction monitoring, 3D visualization, safety inspection, and risk assessment. The advantages of point cloud data in applications lie in its ability to be measured by scanners in real time, its compact representation of the three-dimensional information of large-scale scenes, its clear and specific representation of the three-dimensional shape information of objects, and its lack of sensitivity to external factors such as lighting. However, because scanned 3D point cloud data is irregular, non-uniform, and disordered, how to better process point cloud data remains a major challenge.

[0058] With the continuous advancement of point cloud data processing and analysis technologies, combining point cloud data with artificial intelligence, machine learning, and big data technologies will enable future construction projects to achieve more efficient design, construction, and management. Through laser scanning and drone technology, the accuracy and efficiency of acquiring point cloud data will be further improved, thereby reducing manual intervention and human error during the quality inspection of building components. Therefore, the application of point cloud data in the construction industry is gradually deepening and showing broad prospects. With continuous technological advancements and increasing industry demands, future construction projects will be able to conduct design, construction, and management more efficiently and accurately, driving the digital transformation and sustainable development of the entire industry.

[0059] Deep learning is a subfield of machine learning and a crucial component of artificial intelligence. It primarily uses deep neural networks for model training and inference, automatically extracting features from large-scale data and learning complex patterns and representations, finding wide application across various industries. In computer vision, deep learning is applied to image classification, object detection, semantic segmentation, and image generation. Current examples of deep learning applications in computer vision include self-supervised learning, the application of the Transformer architecture with self-attention mechanisms, and the application of multimodal data. With continuous algorithm optimization, deep learning techniques in computer vision are constantly innovating.

[0060] Semantic segmentation algorithms for 3D point clouds are an important research direction in the field of computer vision. They aim to segment point clouds into subsets with different semantic labels based on the spatial geometry and shape information of the point cloud data. This technology provides strong support for understanding 3D scenes and has broad application prospects in many fields such as visual navigation and positioning, autonomous driving, augmented reality, and virtual reality. In the construction industry, due to the uniqueness of each construction project, the industry has always been labor-intensive and lacks automation and digitalization. To improve the productivity of the construction industry, the adoption of information technology is a major development trend, and the key to adopting information technology is obtaining accurate project information. To fill this gap, 3D point cloud data has become a superior method for obtaining precise geometric information of construction projects. In the construction industry, 3D point cloud data has been used to capture the geometry of construction sites, the construction progress of construction projects, and the construction quality of construction equipment. Furthermore, processing point cloud data through semantic segmentation algorithms improves the accuracy of the geometric features of buildings described by 3D point clouds, helping to make better decisions in construction projects, thereby improving construction productivity, quality, and safety.

[0061] In training deep learning semantic segmentation models, data gaps prevent the model from learning complete and accurate feature distributions, weakening its generalization ability and reducing its accuracy in practical applications. Furthermore, data gaps cause the model to focus excessively on limited and incomplete information during training, increasing the risk of overfitting and missing semantic information in certain important spatial areas, thus affecting the model's understanding of image content and deep segmentation. Currently, datasets for deep learning semantic segmentation are often lacking or limited, hindering model training. Moreover, due to the disordered, irregular, and unevenly distributed density and distribution of 3D point cloud data, semantic segmentation algorithms for 3D point clouds require training on large datasets to achieve high universality and robustness.

[0062] Based on the above introduction, this invention aims to propose a point cloud data generation method for semantic segmentation technology oriented towards deep learning in prefabricated building components. This method can generate relatively ideal point cloud data, effectively supporting the training and application of semantic segmentation algorithms. The overall flowchart of the method is shown below. Figure 1 As shown below, the specific steps are explained.

[0063] Step (1): Automatically generate prefabricated building component models based on Revit software secondary development, and use these models as the original materials for dataset training. The steps for automatically generating prefabricated building component models include creating structured prefabricated building component text and creating 3D models through Revit secondary development.

[0064] This invention proposes a method for creating structured text for generating rules of prefabricated building components, referring to... Figure 2 The steps to create text for structured prefabricated building components include the following:

[0065] S11. Based on the building code requirements in PDF format, convert it into an easily editable unstructured Word document. These building code requirements include the national standard plan "General Technical Conditions for Prefabricated Buildings, Steel Structures, Precast Components and Joints," "Technical Standard for Prefabricated Concrete Buildings" GBT 51231-2016, and the national standard "General Technical Conditions for Precast Components for Prefabricated Concrete Buildings." This step can be done using Python pdf2docx to convert the PDF document to Word format, performing the format conversion through layout analysis of the PDF document.

[0066] S12, based on an unstructured text document in Word format, performs a complete technical review of the specification document. This step involves editing and adjusting the document content to ensure its accuracy and completeness.

[0067] S13, based on the verified Word document, the document is divided into three parts: prefabricated component category, corresponding dimensions, and geometric features, resulting in a structured text for prefabricated building components. This step involves classifying and organizing the document content to guide the subsequent automated generation of prefabricated building component models. This invention uses solid slabs, hollow slabs, truss beams, and ribbed slabs related to structural performance as examples. Specifically, the categories include solid slabs, hollow slabs, truss beams, and ribbed slabs related to structural performance; the dimensions include the total length, width, and height of the prefabricated slab; and the geometry refers to the geometric features of the prefabricated slab, including the location and size of shear keys and shear cavities.

[0068] In the construction industry, particularly in the creation of prefabricated component models, this invention proposes using Revit software for secondary development to automatically generate prefabricated component models. Revit is a Building Information Modeling (BIM) software widely used in architectural design. Its core advantage lies in its powerful BIM functions, integrating architectural floor plans, elevations, sections, and other views to achieve a complete 3D model presentation. Compared to traditional manual modeling methods, this invention aims to automate the generation of prefabricated component models through secondary development.

[0069] The steps for automatically generating prefabricated building component models based on structured prefabricated building component text and through secondary development of Revit software include:

[0070] When performing secondary development based on the text of rules for generating prefabricated structural building components (S1A), the development environment must first be configured. In this example, the secondary development of Revit software mainly relies on the RevitPythonShell plugin to set up the Python language environment.

[0071] The InteractivePythonShell module involved in this process is the code writing page in RevitPythonShell, which ensures that the code runs in the appropriate environment and avoids runtime errors caused by environmental differences.

[0072] Regarding the environment configuration required for secondary development, it is also necessary to import the third-party libraries needed in secondary development, such as the Revit.DB library and the Revit.UI library, which provide basic functions and methods to serve subsequent code writing;

[0073] S1B, based on a secondary development configuration environment, and according to the structured prefabricated building component text, first determines the overall length, width and height for each type and size of prefabricated component, forming a three-dimensional solid model;

[0074] S1C, based on structured prefabricated building component text, uses empty and solid extruded bodies to represent geometric features and obtains a three-dimensional solid model of prefabricated components in RVT format;

[0075] S1D ultimately converts the RVT format into a 3D model in STL (StereoLithography) format with mesh features.

[0076] STL format files offer significant advantages in generating point clouds for images, including data structure simplicity, model compatibility and scalability, and diversity of sampling methods. Specifically, an STL file represents the geometry of a 3D surface as triangles. The surface is subdivided or decomposed into a series of small triangular faces, each described by a vertical direction and three points representing the triangle's vertices. Furthermore, unlike other 3D file formats, STL files do not represent or store color, texture, or other attributes; they describe the surface geometry of a 3D object solely by storing discretized triangular mesh information of the model's surface. Because STL files consist of only triangular meshes and do not involve complex data structures, and there are no requirements on the storage order of the triangular meshes, this file type simplifies the process of computer reading, writing, and storing information. Moreover, using the STL format, with appropriate sampling techniques and parameter settings, high-quality point cloud data can be generated, preserving more detail and accuracy. Users can flexibly control the density and accuracy of the point cloud by adjusting sampling parameters such as sampling resolution, field of view (FOV), and icosahedral subdivision number according to actual needs.

[0077] Step (2): Based on the automatically generated prefabricated building component model, the component model is moved to the scanner coordinate system through coordinate system transformation.

[0078] Reference Figure 3 In this example, the prefabricated components are as follows: Figure 3 Taking a prefabricated slab as an example, the 3D model of the slab in its initial state is changed using graphical linguistics, transforming its random placement in the world coordinate system to the scanner coordinate system, thus constructing a consistent coordinate system setting. Specifically, the line connecting the z-axis of the scanner coordinate system to the origin of both the scanner coordinate system and the origin of the world coordinate system (i.e.,...) Figure 3 vectors in Parallel, the x and y axes of the scanner coordinate system are considered to be perpendicular to the z axis.

[0079] The specific conversion process is described below:

[0080] First, move the plate model to the origin of the world coordinate system. Then, based on the world coordinate system with the plate model at the origin, use the transformation matrix M... ScanTrans The world coordinate system can be converted to the scanner coordinate system, which makes it easier to observe the precast slab model in the scanner coordinate system, as shown in formula (1).

[0081]

[0082] Among them, v i Let be the coordinates of the vertices of the triangular mesh in the prefabricated model. If the coordinates of a vertex of a mesh in the prefabricated model in the world coordinate system are... After coordinate system transformation, the coordinates in the scanner coordinate system are: Transformation matrix M ScanTrans This matrix, obtained by sequentially multiplying the rotation and flip matrices, enables the transformation from the world coordinate system to the scanner coordinate system. This process ensures that the model can be correctly aligned and observed from the scanner's perspective.

[0083] Step (3): Based on the prefabricated building component model from the scanner's perspective, construct a virtual projection plane and perform projection calculations on the triangular mesh on the prefabricated component.

[0084] Based on a consistent coordinate system setup, a projection plane needs to be defined between the scanner and the prefabricated model. This plane is perpendicular to the z-axis of the scanner's coordinate system. It is important to emphasize that the angular resolution of the projection plane needs to be set according to its size. For example, if the size of the projection plane is set to 800*600, then its angular resolution should be 800*600 pixels. Based on the set projection plane, the projection matrix M of the plane is determined according to its parameters. ProjTransThe coordinate dimension transformation is shown in formula (2). The parameters involved in the formula include field of view (FoV), aspect ratio, near plane, and far plane. Here, (FoV) represents the field of view that the scanner can observe through the two-dimensional projection plane; aspect ratio is the ratio of the width to the length of the projection plane, which is taken as 800*600 in this example; near plane and far plane are the two closest and farthest planes that the scanner can detect.

[0085]

[0086] Based on projection matrix M ProjTrans The three-dimensional vertex coordinates can be converted into two-dimensional projected coordinates using formula (3).

[0087]

[0088] in, The vertex coordinates of a certain mesh in the prefabricated component model from the scanner's perspective are represented by the two-dimensional projected coordinates of that vertex after transformation by the projection matrix. Projection transformation from 3D coordinates to 2D coordinates, such as Figure 4 As shown.

[0089] Step (iv): Use the z-Buffer algorithm to filter out the triangular meshes visible to the prefabricated component at any position of the scanner. The processing procedure is as follows: Figure 5 As shown.

[0090] It should be noted that in step (iii), each pixel covered by the 3D triangular mesh projected onto the 2D projection plane has its corresponding depth value (also known as the z-Buffer value). That is, the z-Buffer value is the distance of all triangular meshes on the 3D model from the 2D projection plane. Based on the z-Buffer value of each mesh on the prefabricated model, this part uses the z-Buffer algorithm to filter out the triangular meshes visible to the scanner at any position.

[0091] Based on the 2D projection plane, each pixel on the 2D projection plane is first assigned an initial z-Buffer value, and the initial color is set to blue. This step is to initialize the depth buffer. Typically, the initial z-Buffer value is set to infinity, representing the farthest possible distance, and a new value is assigned during subsequent updates.

[0092] Then, based on the initialized depth buffer, the visible two-dimensional projection coordinates on the projection plane are filtered out. The specific steps include:

[0093] S41, calculate and store the z-Buffer value of each pixel corresponding to the projection of each 3D triangular mesh on the prefabricated slab model onto the 2D plane. Here, z-Bufferp is used. Pi Represents a certain pixel The z-Buffer value;

[0094] S42, for all pixels on the projection plane, visible meshes are selected based on their z-Buffer values. At a single pixel, if the z-Buffer value calculated for the corresponding 3D mesh is less than the z-Buffer value recorded for the current pixel, the z-Buffer value at that pixel is updated; otherwise, it is retained. This ensures that when the 2D projections of different meshes overlap, only the z-Buffer value calculated for the current pixel by the mesh closest to the scanner is retained. For example, if meshes A and B partially overlap pixels on the projection plane, and given that the pixel z-Buffer value (z-Buffer1) calculated from mesh A is smaller than the z-Buffer value (z-Buffer2) calculated from triangular mesh B, the z-Buffer value calculated from mesh A is taken as the z-Buffer value (z-Bufferp) for that pixel. Pi =z-Buffer1), such as Figure 5 As shown in (a) in the figure.

[0095] Through the above operations, the z-Buffer value at each pixel is the depth value of the mesh closest to the projection plane. This mesh will be preserved and considered a visible mesh from the current scanner's perspective. Alternatively, it can be understood that for each pixel on the projection plane, meshes with larger or farther depth values ​​are not visible from the scanner's perspective.

[0096] S43, Simultaneously, all pixels with z-Buffer values ​​of finite values ​​are considered two-dimensional visible pixels, and the color of the current pixel is updated to red. The same method is used to represent the effect of applying the z-Buffer algorithm to the entire projection plane. Figure 5 (b) in the diagram shows the effect of applying the z-Buffer algorithm.

[0097] S44, based on the visible two-dimensional pixels in the scanner coordinate system, uses two inverse matrix transformations to restore the two-dimensional visible pixels retained on the projection plane to three-dimensional point cloud coordinates, such as... Figure 5 As shown in (c); if a 3D point cloud coordinate point falls on a visible mesh, then the mesh is considered visible, thus filtering out visible meshes on the 3D model, such as... Figure 5 As shown in (d) in the figure.

[0098] Step (5): Based on all possible scanner placement locations, use a heuristic algorithm to combine different scanner perspectives to achieve a scanning coverage of the precast slab model that reaches a specified threshold (e.g., 95%). For each scanner combination that reaches the threshold, simulate the point cloud of the visible triangular mesh from its perspective, and finally generate the point cloud data of the precast component.

[0099] Based on the 3D mesh that can be detected from any point in the scanner viewpoint as calculated in step (iv), by combining scanner viewpoints from different locations, a comprehensive scan of the precast slab model is achieved, generating a simulated point cloud. The specific steps are as follows:

[0100] S51. Based on the spatial constraints of the scene, multiple potential scanning positions are generated and the mesh position that can be observed from the perspective of each potential scanner is calculated. Finally, a heuristic algorithm is used to determine the scanner combination that enables the prefabricated component to achieve a coverage rate of 95%. This step can simulate the real scanning situation of multiple points and multiple perspectives in a real scene. Figure 6 All hollow points in (a) represent potential scan locations generated based on spatial constraints, while red points represent scanner combination locations selected by the heuristic algorithm.

[0101] S52, for each scan point in the selected scanner position combination, create a two-dimensional plane parallel to the prefabricated plate, and calculate the two-dimensional scan point Si(X) on the created plane by generating the relative positional relationship between the scanner and the plane, and the parameters set by the scanner. Si ,Y Si ).

[0102] S53, for the triangular mesh visible from the current scanner's perspective, the three-dimensional point cloud coordinates si(x) are simulated one by one by calculating the intersection points of the straight line formed by the scanner and the two-dimensional scanning points with the triangular mesh. si ,y si ,z si ),like Figure 6 As shown in (b) above. Based on the two-dimensional coordinates obtained from the generated mesh projection, the three-dimensional coordinates of the visible mesh under the current scanner view can be simulated by the intersection of the straight line and the cluster of straight lines passing through the scanner and the scanning point.

[0103] Following the steps above, calculate the 3D point cloud coordinates of the corresponding visible triangular mesh for each scanner position in each combination, and finally complete the point cloud data generation of the prefabricated building component model, such as... Figure 6 As shown in (c) in the figure.

[0104] Based on the same technical concept as the method embodiments, the present invention also provides a point cloud generation system for deep learning semantic segmentation of prefabricated building components, comprising:

[0105] The document preprocessing module is used to convert the building code requirements in PDF format into an unstructured Word document that is easy to edit, and to perform an integrity review. Based on the reviewed and error-free Word document, the document content is divided into three parts: prefabricated component category, corresponding dimensions, and geometric features, to obtain a structured prefabricated building component rule text.

[0106] The 3D model generation module is used to create adaptive family files in Revit software based on structured prefabricated building component rule text, build a secondary development environment configuration, and automatically generate 3D models of prefabricated building components in STL format based on the configured secondary development environment.

[0107] The triangular mesh detection module is used to move the automatically generated STL format prefabricated component model to the origin of the world coordinate system, and convert the world coordinate system to the scanner coordinate system through coordinate system transformation. Based on a consistent coordinate system setting, the module calculates the visible triangular meshes in the STL format prefabricated component model at any scanner position by projecting the three-dimensional coordinates onto the two-dimensional plane and using the depth value filtering in the z-Buffer algorithm.

[0108] The scanning position combination module is used to generate a series of potential scanner positions according to the scene conditions, calculate the visible triangular mesh of the prefabricated component under each scanner position, and select the scanner position combination that makes the visibility of the prefabricated component reach a specified threshold.

[0109] The point cloud data generation module is used to simulate and generate the point cloud data of the corresponding visible triangular mesh for each scanning position of the selected scanner position combination, based on the scanner characteristics and the relative positional relationship between the prefabricated component and the scanner, thereby generating the point cloud data of the entire prefabricated component.

[0110] It should be understood that the point cloud generation system for deep learning semantic segmentation of prefabricated building components in the embodiments of the present invention can realize all the technical solutions in the above method embodiments. The functions of each functional module can be specifically implemented according to the methods in the above method embodiments. The specific implementation process can be referred to the relevant descriptions in the above embodiments, which will not be repeated here.

[0111] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the point cloud generation method for deep learning semantic segmentation of prefabricated building components as described above.

[0112] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the point cloud generation method for deep learning semantic segmentation of prefabricated building components as described above.

[0113] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus (systems), computer devices, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] This invention is described with reference to a flowchart of a method according to embodiments of the invention. It should be understood that each step in the flowchart and combinations thereof can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 A device for a function specified in one or more processes.

[0115] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 The function specified in one or more processes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 Steps of a specified function in one or more processes.

Claims

1. A point cloud generation method for deep learning semantic segmentation of prefabricated building components, characterized in that, Includes the following steps: The building code requirements in PDF format are converted into an unstructured Word document that is easy to edit, and then a completeness review is conducted. Based on the Word document that has been reviewed and found to be error-free, the document content is divided into three parts: prefabricated component category, corresponding dimensions, and geometric features, resulting in a structured text of building prefabricated component rules. Based on the structured prefabricated building component rule text, an adaptive family file is created in Revit software, a secondary development environment configuration is built, and a 3D model of the prefabricated building component in STL format is automatically generated based on the configured secondary development environment. The automatically generated STL format prefabricated component model is moved to the origin of the world coordinate system. The world coordinate system is then converted to the scanner coordinate system through coordinate system transformation. Based on a consistent coordinate system setting, the three-dimensional coordinates are projected onto a two-dimensional plane, and the depth values ​​in the z-Buffer algorithm are used for filtering to calculate the visible triangular meshes in the STL format prefabricated component model at any scanner position. A series of potential scanner positions are generated based on the scene conditions. The visible triangular mesh of the prefabricated component at each scanner position is calculated one by one. The combination of scanner positions that makes the visibility of the prefabricated component reach a specified threshold is selected. For each scanning position in the selected combination of scanner positions, point cloud data of the corresponding visible triangular mesh is simulated and generated based on the scanner characteristics and the relative positional relationship between the prefabricated component and the scanner, thereby generating point cloud data of the entire prefabricated component.

2. The method according to claim 1, characterized in that, The system automatically generates 3D models of prefabricated building components in STL format based on a configured secondary development environment, including: Based on the secondary development configuration environment and according to the structured building prefabricated component rule text, the overall length, width and height are determined for each type of prefabricated component and its corresponding size, and the edges of the three-dimensional model are determined. Based on the edges of the 3D model, the geometric features are represented by hollow or solid extrusions according to the geometric features in the structured prefabricated building component rule text, and the 3D model of the prefabricated component in RVT format is obtained. Convert the RVT format 3D model into an STL format 3D model composed of triangular meshes.

3. The method according to claim 1, characterized in that, The world coordinate system is converted to the scanner coordinate system using a coordinate system transformation. The transformation formula is as follows: In the formula, Let be the coordinates of the vertex of the i-th triangular mesh in the world coordinate system on the prefabricated component model; M represents the coordinates in the scanner's coordinate system after coordinate transformation; ScanTrans The transformation matrix is ​​obtained by performing matrix multiplication operations on the rotation matrix and the flip matrix in sequence.

4. The method according to claim 1, characterized in that, Based on a consistent coordinate system setting, by projecting three-dimensional coordinates onto a two-dimensional plane and using depth value filtering in the z-Buffer algorithm, the visible triangular meshes in the STL format prefabricated component model at any scanner position are calculated, including: Define a two-dimensional projection plane located between the scanner and the prefabricated component model. This plane is perpendicular to the z-axis of the scanner coordinate system. Determine the projection matrix based on the parameters of the projection plane. Based on the projection matrix, convert the three-dimensional vertex coordinates of the triangular mesh into two-dimensional projection coordinates. Each pixel covered by a 3D triangular mesh projected onto a 2D projection plane has its corresponding depth value, called the z-Buffer value. Based on the 2D projection plane, each pixel on the 2D projection plane is assigned an initial z-Buffer value to form an initial depth buffer. Based on the initial depth buffer, the z-Buffer value of each 3D triangular mesh on the prefabricated component model is calculated and stored at each pixel point covered by the projection onto the 2D plane. For each pixel point on the projection plane, if the calculated z-Buffer value corresponding to the current triangular mesh is less than the previously recorded z-Buffer value, the z-Buffer value at that pixel point is updated to the calculated value; otherwise, it is retained. The two-dimensional visible pixels are determined based on the final z-Buffer value of each pixel on the projection plane. By performing two inverse matrix transformations, the coordinates of the two-dimensional visible pixels retained on the projection plane are restored to three-dimensional point cloud coordinates, thereby filtering out the visible triangular meshes on the three-dimensional model.

5. The method according to claim 4, characterized in that, The projection matrix is ​​determined based on the parameters of the projection plane, and the calculation method is as follows: Among them, M ProjTrans The projection matrix is ​​represented; FoV represents the field of view that the scanner can observe through the two-dimensional projection plane; aspect is the ratio of the width to the length of the projection plane; near and far are the two closest and farthest planes that the scanner can detect. Based on the projection matrix, the 3D vertex coordinates of the triangular mesh are converted into 2D projected coordinates, represented as: in, Let be the vertex coordinates of the i-th triangular mesh in the prefabricated component model as seen from the scanner's perspective. After transformation by the projection matrix, the two-dimensional projected coordinates of this vertex are expressed as:

6. The method according to claim 1, characterized in that, Based on the scene conditions, a series of potential scanner positions are generated. The visible triangular mesh of the prefabricated component at each scanner position is calculated. The combination of scanner positions that ensures the visibility of the prefabricated component reaches a specified threshold is selected, including: Based on the spatial constraints of the scene, multiple potential scanning positions are generated and the position of the triangular mesh that can be observed from the perspective of each potential scanner is calculated. The combination of scanner positions that achieves 95% coverage of the prefabricated components is determined by a heuristic algorithm.

7. The method according to claim 1, characterized in that, Based on the scanner's characteristics and the relative positional relationship between the prefabricated components and the scanner, point cloud data of the corresponding visible triangular meshes are simulated and generated, including: For each scan point in the selected combination of scanner positions, a two-dimensional plane parallel to the prefabricated component is created. The two-dimensional scan point on the created plane is calculated by generating the relative positional relationship between the scanner and the plane, as well as the parameters set by the scanner. For the triangular mesh visible under the current scanner, the three-dimensional point cloud coordinates are simulated one by one by calculating the intersection points of the straight line formed by the scanner and the two-dimensional scanning points with the triangular mesh. For each combination, the 3D point cloud coordinates of the corresponding visible triangular mesh are calculated for the scanner position, and finally the point cloud data of the prefabricated building component model is generated.

8. A point cloud generation system for deep learning semantic segmentation of prefabricated building components, characterized in that, include: The document preprocessing module is used to convert the building code requirements in PDF format into an unstructured Word document that is easy to edit, and to perform an integrity review. Based on the reviewed and error-free Word document, the document content is divided into three parts: prefabricated component category, corresponding dimensions, and geometric features, to obtain a structured prefabricated building component rule text. The 3D model generation module is used to create adaptive family files in Revit software based on structured prefabricated building component rule text, build a secondary development environment configuration, and automatically generate 3D models of prefabricated building components in STL format based on the configured secondary development environment. The triangular mesh detection module is used to move the automatically generated STL format prefabricated component model to the origin of the world coordinate system, and convert the world coordinate system to the scanner coordinate system through coordinate system transformation. Based on a consistent coordinate system setting, the module calculates the visible triangular meshes in the STL format prefabricated component model at any scanner position by projecting the three-dimensional coordinates onto the two-dimensional plane and using the depth value filtering in the z-Buffer algorithm. The scanning position combination module is used to generate a series of potential scanner positions according to the scene conditions, calculate the visible triangular mesh of the prefabricated component under each scanner position, and select the scanner position combination that makes the visibility of the prefabricated component reach a specified threshold. The point cloud data generation module is used to simulate and generate the point cloud data of the corresponding visible triangular mesh for each scanning position of the selected scanner position combination, based on the scanner characteristics and the relative positional relationship between the prefabricated component and the scanner, thereby generating the point cloud data of the entire prefabricated component.

9. A computer device, characterized in that, include: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the point cloud generation method for deep learning semantic segmentation of prefabricated building components as claimed in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the point cloud generation method for deep learning semantic segmentation of prefabricated building components as described in any one of claims 1-7.

Images