Point cloud generation method and system for deep learning semantic segmentation of building prefabricated parts

Abstract

The invention discloses a point cloud generation method and system for deep learning semantic segmentation of a building prefabricated component. The method mainly comprises the steps of creating a structured text of a building prefabricated part generation rule, automatically generating a standardized building prefabricated part model based on a Revit secondary development environment, and generating point cloud data by using a graphics method. According to the method, the document is divided into three parts, namely the prefabricated part category, the corresponding size and the geometric feature, so that the structured building prefabricated part text is obtained, the readability of the building specification is improved, a reader can understand the text conveniently, the building prefabricated part model meeting the requirements of the building specification is generated in an automatic mode, and the working efficiency is greatly improved. The method can efficiently realize automatic and standardized generation of the point cloud data of the building prefabricated part model, and serves for size and quality detection of the building prefabricated part.

Description

Technical Field

[0001] The present invention relates to a technical solution for quality and size detection of prefabricated building components, and in particular provides a method and system for generating point cloud data for machine semantic segmentation and deep learning, aiming to solve the problem of missing point cloud data of prefabricated building components. Background Art

[0002] Point cloud data is a collection of vectors in a three-dimensional coordinate system. The data's key characteristics are high precision, high resolution, and high dimensionality of geometric information, enabling intuitive representation of the shape, surface, and texture of objects in space. Point cloud data is used in the architectural field for architectural design, building monitoring, construction, cultural heritage preservation, and interior design. In building monitoring and construction, regularly acquiring point cloud data of buildings allows monitoring changes and damage, enabling timely maintenance and repairs to ensure their safety and stability. It also provides construction personnel with more accurate information on building mass and dimensions, helping them better control and adjust equipment during the construction process, thereby improving efficiency and safety.

[0003] Point cloud-based semantic segmentation refers to the use of point cloud data to perform semantic segmentation of scenes, identifying and classifying different objects or regions within the point cloud. Deep learning algorithms are used in this area by automatically learning the features and semantic information of different objects in point cloud data through deep learning models, and then performing semantic segmentation based on these learned features and information. Deep learning-based models and algorithms, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and fully connected neural networks (FCNs), are commonly used in this field to process point cloud data. These models enable feature extraction, learning, and prediction of point cloud data, ultimately achieving semantic segmentation. Furthermore, to achieve better semantic segmentation results, advanced deep learning techniques, such as attention mechanisms, multi-scale feature learning, and transfer learning, can be combined to improve model accuracy. Deep learning-based methods, such as PointNet, PointNet++, and VoxNet, have been applied to point cloud semantic segmentation tasks and achieved promising results. These methods effectively achieve semantic segmentation of point clouds by processing and learning point cloud data, providing an important foundation for 3D scene understanding and recognition.

[0004] Datasets are the foundation for training and testing machine learning models, determining their performance and generalization capabilities. Due to the difficulty of collecting point cloud data and the complexity of data processing, there is currently a lack of point cloud datasets validating machine deep learning experimental methods. The difficulty of collecting point cloud data is reflected in the fact that it typically requires the use of expensive laser scanners or photogrammeters, which require professional operation and are relatively costly. The complexity of point cloud data processing is reflected in the fact that point cloud data typically contains a large number of points, requiring complex processing and analysis, as well as specialized software and technical support.

[0005] In existing technologies, point cloud generation for prefabricated building components typically relies on automated conversion using a fixed point cloud density set by the model. While this approach achieves a certain degree of automation, it fails to account for the placement and parameter settings of the scanner in the real environment, resulting in the generated point cloud data failing to accurately simulate actual scanning conditions. When applied to complex architectural scenes, such methods often result in missing data or insufficient accuracy, which in turn impacts tasks such as quality inspection and dimensional verification. Therefore, existing technologies suffer from a lack of effective simulation of real-world scanning conditions when generating point cloud data, a key technical challenge that this invention aims to address. Summary of the Invention

[0006] Purpose of the invention: The present invention proposes a point cloud generation method and system for deep learning semantic segmentation of prefabricated building components, which solves the problems of missing point cloud data of prefabricated building components and the lack of effective simulation of real scanning conditions in existing automation methods.

[0007] Technical solution: To achieve the above-mentioned purpose, in the first aspect, the present invention proposes a point cloud generation method for deep learning semantic segmentation of building prefabricated components, comprising the following steps:

[0008] Convert building code requirements in PDF format into an easy-to-edit, unstructured Word document and conduct a completeness review. Based on the Word document that has been reviewed, divide the document content into three parts: prefabricated component category, corresponding dimensions, and geometric characteristics, thus obtaining a structured text of building prefabricated component rules.

[0009] Based on the structured rule text of building prefabricated components, an adaptive family file is created in Revit software, a secondary development environment configuration is constructed, and based on the configured secondary development environment, a 3D model of the building prefabricated components in STL format is automatically generated;

[0010] The automatically generated prefabricated component model in STL format is moved to the origin of the world coordinate system. The world coordinate system is converted to the scanner coordinate system through coordinate system transformation. Based on the consistent coordinate system setting, the three-dimensional coordinates are projected onto the two-dimensional plane and the depth value in the z-buffer algorithm is used to filter the visible triangulated mesh in the prefabricated component model in STL format at any scanner position.

[0011] Generate a series of potential scanner positions based on the scene conditions, calculate the visible triangulated meshes of the prefabricated components at each scanner position, and select the scanner position combination that makes the visibility rate of the prefabricated components reach the specified threshold;

[0012] For each scanning position of the selected scanner position combination, point cloud data of the corresponding visible triangulated mesh is simulated and generated according to the scanner characteristics and the relative position relationship between the prefabricated component and the scanner, thereby generating point cloud data of the entire prefabricated component.

[0013] As a preferred embodiment, the configured secondary development environment automatically generates a 3D model of the prefabricated building components in stl format, including:

[0014] Based on the secondary development configuration environment and the structured building prefabricated component rule text, the overall length, width and height are determined for each prefabricated component category and corresponding size, and the edges of the three-dimensional model are determined;

[0015] Based on the edge of the three-dimensional model, according to the geometric features in the structured building prefabricated component rule text, the geometric features are expressed with a hollow or solid extrusion body to obtain the three-dimensional model of the prefabricated component in RVT format;

[0016] Convert the 3D model in rvt format into a 3D model in stl format consisting of triangulated meshes.

[0017] Preferably, the world coordinate system is converted to the scanner coordinate system through coordinate system conversion, and the conversion formula is as follows:

[0018] v scani =M ScanTrans ×v worldi

[0019] Where, v worldi v is the coordinate of the vertex of the i-th triangular mesh on the prefabricated component model in the world coordinate system; scani is the coordinate in the scanner coordinate system after coordinate system conversion; M Scantrans is the transformation matrix, which is obtained by performing matrix multiplication operations on the rotation matrix and the flip matrix in sequence.

[0020] As a preference, based on a consistent coordinate system setting, by projecting the three-dimensional coordinates onto a two-dimensional plane and using the depth value screening in the z-Buffer algorithm, the triangulated mesh visible in the precast component model in stl format at any scanner position is calculated, including:

[0021] A two-dimensional projection plane is defined between the scanner and the prefabricated component model. The plane is perpendicular to the z-axis of the scanner coordinate system. A projection matrix is ​​determined based on the parameters of the projection plane. The three-dimensional vertex coordinates of the triangulated mesh are converted into two-dimensional projection coordinates based on the projection matrix.

[0022] Each pixel covered by the 3D triangulated mesh projected onto the 2D projection plane has its corresponding depth value, called the z-Buffer value. Based on the 2D projection plane, each pixel on the 2D projection plane is assigned an initial z-Buffer value to form an initialized depth buffer.

[0023] Based on the initialized depth buffer, the z-buffer value of each pixel corresponding to each 3D triangular mesh on the prefabricated component model projected onto the 2D plane is calculated and stored. For each pixel on the projection plane, if the calculated z-buffer value corresponding to the current triangular mesh is less than the previously recorded z-buffer value, the z-buffer value at the pixel is updated to the calculated value; otherwise, it is retained.

[0024] The two-dimensional visible pixels are determined based on the final z-Buffer value of each pixel on the projection plane. Through two inverse matrix transformations, the coordinates of the two-dimensional visible pixels retained on the projection plane are restored to three-dimensional point cloud coordinates, thereby screening out the visible triangulated meshes on the three-dimensional model.

[0025] Preferably, the projection matrix is ​​determined according to the parameters of the projection plane, and the calculation method is as follows:

[0026]

[0027] Among them, M ProjTrans represents the projection matrix; FoV represents the field of view that the scanner can observe through the two-dimensional projection plane; aspect is the ratio of the width to the length of the projection plane; near and far are the closest and farthest two planes that the scanner can detect;

[0028] The three-dimensional vertex coordinates of the triangular mesh are converted into two-dimensional projection coordinates based on the projection matrix, which is expressed as:

[0029] V 2Di =M ProjTrans ×v scani

[0030] Among them, v scaniis the vertex coordinate of the i-th triangular mesh of the prefabricated component model under the scanner’s viewing angle. The two-dimensional projection coordinate of the vertex after the projection matrix transformation is expressed as V 2Di .

[0031] Preferably, a series of potential scanner positions are generated according to the scene conditions, and the visible triangulated meshes of the prefabricated components at each scanner position are calculated one by one. The combination of scanner positions that makes the visibility rate of the prefabricated components reach a specified threshold is selected, including:

[0032] According to the spatial constraints of the scene, multiple potential scanning positions are generated and the triangulated mesh positions that can be observed under each potential scanner perspective are calculated. The scanner position combination that can achieve a prefabricated component coverage rate of 95% is determined through a heuristic algorithm.

[0033] Preferably, the point cloud data of the corresponding visible triangulated mesh is generated by simulating according to the characteristics of the scanner and the relative positional relationship between the prefabricated component and the scanner, including:

[0034] For each scanning point in the selected scanner position combination, a two-dimensional plane parallel to the prefabricated component is created, and the two-dimensional scanning point on the created plane is calculated by generating a relative position relationship between the scanner and the plane and the parameters set by the scanner;

[0035] For the triangulated mesh visible under the current scanner, the 3D point cloud coordinates are simulated one by one by calculating the intersection points of the triangulated mesh with the straight line formed by the scanner and the 2D scanning points;

[0036] For each scanner position in the combination, the three-dimensional point cloud coordinates of the corresponding visible triangulated mesh are calculated, and finally the point cloud data generation of the building prefabricated component model is completed.

[0037] Second, a point cloud generation system for deep learning semantic segmentation of building prefabricated components, comprising:

[0038] The document preprocessing module is used to convert the building code requirements in PDF format into an unstructured Word document for easy editing and conduct a completeness review. Based on the Word document that has been reviewed and found to be correct, the document content is divided into three parts: prefabricated component category, corresponding size and geometric characteristics, and a structured text of building prefabricated component rules is obtained;

[0039] A 3D model generation module is used to create adaptive family files in Revit software based on structured prefabricated building component rule text, build a secondary development environment configuration, and automatically generate 3D models of prefabricated building components in STL format based on the configured secondary development environment;

[0040] The triangulated mesh detection module is used to move the automatically generated STL format precast component model to the world coordinate system origin, convert the world coordinate system to the scanner coordinate system through coordinate system transformation, and calculate the triangulated mesh visible in the STL format precast component model at any scanner position by projecting the three-dimensional coordinates onto the two-dimensional plane based on the consistent coordinate system settings and using the depth value screening in the z-buffer algorithm;

[0041] The scanning position combination module is used to generate a series of potential scanner positions according to the scene conditions, calculate the visible triangulated mesh of the prefabricated components at each scanner position, and select the scanner position combination that makes the visibility rate of the prefabricated components reach a specified threshold;

[0042] The point cloud data generation module is used to simulate and generate the point cloud data of the corresponding visible triangulated mesh for each prefabricated component of the selected scanner position combination according to the scanner characteristics and the relative position relationship between the prefabricated component and the scanner, thereby generating the point cloud data of the entire prefabricated component.

[0043] In a third aspect, the present invention also provides a computer device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the point cloud generation method for deep learning semantic segmentation of prefabricated building components as described in the first aspect of the present invention are implemented.

[0044] In a fourth aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the point cloud generation method for deep learning semantic segmentation of building prefabricated components as described in the first aspect of the present invention.

[0045] Beneficial effects:

[0046] (1) The present invention proposes an innovative method for generating point cloud data of building prefabricated components in the field of semantic segmentation technology in the field of computer machine learning. Through secondary development of Revit software, the automatic generation of building prefabricated component models of different specifications is achieved, and the visible area of ​​the model is screened out by combining the z-Buffer algorithm to accurately extract its surface information. Compared with the traditional point cloud data acquisition method that relies on laser scanning equipment, this solution starts with standard documents through software processing and can generate high-quality point cloud data without expensive scanning equipment, significantly reducing costs and technical barriers.

[0047] (2) The point cloud data of prefabricated building components generated by this method can be used as a dataset for training deep learning semantic segmentation models. It can simulate most common structural features and geometric details to support the effective training of semantic segmentation models. Therefore, the point cloud data generated by this method can not only serve as a standard reference for model training, but also significantly reduce the cost and time of data acquisition, improving the overall efficiency and feasibility of deep learning projects.

[0048] (3) The present invention obtains a structured text of prefabricated building components by performing format conversion and technical review, thereby improving the readability of the building specifications and making them easier for readers to understand. Based on the obtained structured text of prefabricated building components, a prefabricated building component model that meets the requirements of the building specifications is generated in an automated manner, which greatly improves work efficiency and makes up for the defects of the existing data set being missing and the small amount of data. In combination with the automatically constructed prefabricated building component model, point cloud data is generated using graphics methods. Through coordinate system conversion, projection calculation is performed in the scanner coordinate system, and different scanning angles are combined to achieve a complete scan of the component and an effective simulation of the scanning conditions. The point cloud data can be used for processing and analysis in complex building scenes to improve construction efficiency and safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Flowchart of the point cloud generation method for deep learning semantic segmentation of building prefabricated components;

[0050] Figure 2 It is a BIM secondary development to automatically generate a 3D model flow chart for prefabricated components;

[0051] Figure 3 It is a schematic diagram of coordinate system transformation;

[0052] Figure 4 It is a schematic diagram of the two-dimensional projection coordinates of the vertices of the three-dimensional mesh on the projection plane;

[0053] Figure 5 This is a flowchart of filtering out visible triangle meshes through the z-Buffer algorithm;

[0054] Figure 6 It is a flow chart of combining different scanning perspectives to achieve a complete scan of a component. DETAILED DESCRIPTION

[0055] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0056] Prefabricated components refer to a construction method in which components or modules are manufactured in a factory according to design specifications and then quickly assembled on-site using mechanical equipment. Prefabricated components can include walls, floor slabs, beams and columns, guardrails, bathroom modules, and other forms. Unlike traditional on-site casting, prefabricated components can accelerate construction progress, improve quality, and save manpower. Therefore, high-quality dimensional and quality inspection is essential.

[0057] Point cloud data refers to a collection of three-dimensional spatial information obtained through 3D scanning technology. It is typically composed of multiple 3D points, each of which is composed of its coordinates (x, y, z) in 3D space and possibly other attributes (such as color and reflectivity). In recent years, the application of point cloud data in the construction industry has become increasingly common, primarily due to its ability to accurately capture real-world 3D information, assisting various stages of building design, construction, and management, such as Building Information Modeling (BIM), existing surveys, construction monitoring, 3D visualization, safety inspections, and risk assessments. The advantages of point cloud data in its applications lie in its ability to be measured by scanners in real time, its compact representation of 3D information in large-scale scenes, its clear and specific representation of the 3D shape of objects, and its lack of sensitivity to external factors such as lighting. However, due to the irregular, non-uniform, and disordered nature of scanned 3D point cloud data, optimal processing of point cloud data remains a major challenge.

[0058] With the continuous advancement of point cloud data processing and analysis technology, and the integration of point cloud data with artificial intelligence, machine learning, and big data technologies, future construction projects will achieve more efficient design, construction, and management. Laser scanning and drone technology will further improve the accuracy and efficiency of point cloud data acquisition, thereby reducing manual intervention and human error in the quality inspection of building components. Therefore, the application of point cloud data in the construction industry is gradually deepening and showing broad prospects. With the continuous advancement of technology and the increasing demand of the industry, future construction projects will be able to carry out design, construction, and management with greater efficiency and precision, promoting the digital transformation and sustainable development of the entire industry.

[0059] Deep learning is a subfield of machine learning and a key component of artificial intelligence. It primarily uses deep neural networks for model training and inference. It can automatically extract features from large amounts of data and learn complex patterns and representations, finding widespread application across various industries. In computer vision, deep learning is used for image classification, object detection, semantic segmentation, and image generation. Current examples of deep learning applied to computer vision include self-supervised learning, the application of the Transformer architecture with self-attention mechanisms, and the use of multimodal data. With the continuous optimization of algorithms, deep learning technology in computer vision continues to innovate.

[0060] Semantic segmentation algorithms for 3D point clouds are a key research area in computer vision. They aim to segment point clouds into subsets with distinct semantic labels based on their spatial geometry and shape information. This technology provides powerful support for 3D scene understanding and holds broad application prospects in fields such as visual navigation and positioning, autonomous driving, augmented reality, and virtual reality. In the construction industry, due to the unique nature of each construction project, the construction industry has been labor-intensive and lacks automation and digitization. To improve productivity in the construction industry, the adoption of information technology is a major development trend, and the key to this adoption is obtaining accurate construction project information. To fill this gap, 3D point cloud data has emerged as a superior method for obtaining precise geometric information about construction projects. In the construction industry, 3D point cloud data has been used to capture the geometry of construction sites, the progress of construction projects, and the quality of construction equipment. Furthermore, processing point cloud data with semantic segmentation algorithms improves the accuracy of the geometric features of buildings depicted by the 3D point clouds, facilitating better decision-making in construction projects and ultimately improving productivity, quality, and safety.

[0061] In the process of training deep learning semantic segmentation models, missing data prevents the model from learning a complete and accurate feature distribution, resulting in a weakened generalization ability of the model and a subsequent decrease in accuracy in practical applications. In addition, missing data causes the model to focus too much on limited and incomplete information during training, increasing the risk of algorithm overfitting. Semantic information in certain important spaces will be omitted, which in turn affects the model's understanding of image content and deep segmentation. Currently, there are gaps and singleness in the datasets for deep learning semantic segmentation, which has hindered the training of the model. In addition, due to the disorder, irregularity, density and uneven distribution of 3D point cloud data, the semantic segmentation algorithm of 3D point clouds needs to be trained based on a large number of datasets to achieve high universality and robustness of the algorithm.

[0062] In combination with the above introduction, the present invention aims to propose a method for generating point cloud data for semantic segmentation technology for deep learning of building prefabricated components, which can generate relatively ideal point cloud data and effectively support the training and application of semantic segmentation algorithms. The overall flow chart of the method is as follows Figure 1 As shown, the specific steps are described below.

[0063] Step (1) automatically generates a building prefabricated component model based on Revit software secondary development, and uses it as the original material for data set training. The step of automatically generating the building prefabricated component model includes creating a structured building prefabricated component text and creating a three-dimensional model through Revit secondary development.

[0064] The present invention proposes a method for creating a structured text of building prefabricated component generation rules, referring to Figure 2 , the steps to create structural building precast component text include the following steps:

[0065] S11. Convert the PDF document into an unstructured Word document for easy editing based on the building code requirements. These include the national standard "General Technical Requirements for Prefabricated Buildings, Steel Structures, Prefabricated Components and Joints," the "Technical Standard for Prefabricated Concrete Buildings" (GBT 51231-2016), and the national standard "General Technical Requirements for Prefabricated Components for Prefabricated Concrete Buildings." This step can be performed using Python pdf2docx to convert the PDF document into a Word document, performing the format conversion by analyzing the PDF document's layout.

[0066] S12, based on unstructured text documents in Word format, conducts a complete technical review of the regulatory documents. This step involves editing and adjusting the document content to ensure its accuracy and completeness;

[0067] S13, based on the Word document that has been reviewed and found to be correct, divide the document into three parts: prefabricated component category, corresponding dimensions, and geometric features, to obtain a structured building prefabricated component text. This step involves the classification and organization of document content to guide the subsequent automated generation of building prefabricated component models. In this invention, solid slabs, hollow slabs, truss beam slabs, and ribbed slabs with relevant structural properties are used as examples. Specifically, the categories include solid slabs, hollow slabs, truss beam slabs, and ribbed slabs with relevant structural properties; the dimensions include the total length, width, and height of the prefabricated slab; and the geometric shape, i.e., the geometric features of the prefabricated slab, includes the location and size of the shear keys and shear cavities.

[0068] In the construction industry, particularly when it comes to creating prefabricated component models, the present invention proposes using Revit software for secondary development to automatically generate prefabricated component models. Revit software is a building information modeling (BIM) software widely used in the field of architectural design. Its core advantage lies in its powerful BIM functionality, which integrates building plans, elevations, sections, and other views to achieve a complete presentation of the three-dimensional model. Compared to previous manual modeling methods, the present invention aims to achieve the automated generation of prefabricated component models through secondary development.

[0069] Based on the structured prefabricated building component text, the steps for automatically generating a prefabricated building component model through secondary development of Revit software include:

[0070] S1A, when performing secondary development based on the text of the rules for generating structured building prefabricated components, you first need to configure the development environment. In this example, the secondary development of Revit software mainly relies on the RevitPythonShell plug-in to implement the Python language environment.

[0071] The InteractivePythonShell module involved in this process is the code writing page in RevitPythonShell, which ensures that the code runs in a suitable environment and avoids runtime errors caused by environmental differences.

[0072] Regarding the environment configuration required for secondary development, it is also necessary to import third-party libraries required for secondary development, such as the Revit.DB library and the Revit.UI library, which provide basic functions and methods to serve subsequent code writing;

[0073] S1B, based on the secondary development configuration environment and the structured building prefabricated component text, first determines the overall length, width, and height for each prefabricated component type and size to form a three-dimensional solid model;

[0074] S1C, based on the structured building prefabricated component text, uses empty and solid extrusions to represent geometric features and obtains a three-dimensional solid model of the prefabricated component in RVT format;

[0075] S1D, finally converts the rvt format into a three-dimensional model in the stl (StereoLithography) format with mesh features.

[0076] STL files offer significant advantages for generating point clouds from images, including a simple data structure, model compatibility and scalability, and a variety of sampling methods. Specifically, STL files represent the geometry of a 3D surface using triangles. The surface is subdivided or decomposed into a series of small triangular facets, each described by vertical directions and three points representing the vertices of the triangles. Furthermore, unlike other 3D file formats, STL files do not represent or store color, texture, or other attributes. Instead, they describe the surface geometry of a 3D object by storing only the discretized triangular mesh of the model surface. Because STL files consist only of triangular meshes and do not involve complex data structures, nor do they require the order in which the triangular meshes are stored, this file type simplifies computer reading, writing, and storing information. Furthermore, using the STL format, appropriate sampling techniques and parameter settings can generate high-quality point cloud data that preserves greater detail and accuracy. Users can flexibly control the density and accuracy of the point cloud by adjusting sampling parameters such as sampling resolution, field of view, and the number of icosahedral partitions, depending on their needs.

[0077] Step (2): Based on the automatically generated building prefabricated component model, the component model is moved to the scanner coordinate system through coordinate system conversion.

[0078] Reference Figure 3 In this example, the prefabricated components are as follows Figure 3 Taking the prefabricated board in the figure as an example, we change the 3D model of the board in the initial state through graphic linguistics, transform its state of random placement in the world coordinate system into the scanner coordinate system, and build a consistent coordinate system setting. Among them, the z-axis of the scanner coordinate system and the line connecting the origin of the scanner coordinate system and the origin of the world coordinate system (i.e. Figure 3 Vectors in ) is parallel, and the x and y axes of the scanner coordinate system are considered to be perpendicular to the z axis.

[0079] The specific conversion process is described as follows:

[0080] First, the plate model is moved to the origin of the world coordinate system, and then based on the world coordinate system with the plate model at the origin, the transformation matrix M is used to transform the plate model to the world coordinate system. ScanTrans The world coordinate system can be converted into the scanner coordinate system to facilitate observation of the precast panel model in the scanner coordinate system, see formula (1).

[0081]

[0082] Among them, v i is the vertex coordinate of the triangular mesh of the prefabricated model. If the coordinates of the vertex of a mesh on the prefabricated model in the world coordinate system are Then the coordinates in the scanner coordinate system after coordinate system conversion are Transformation matrix M ScanTrans The matrix is ​​obtained by performing matrix multiplication of the rotation matrix and the flip matrix, which can realize the transformation from the world coordinate system to the scanner coordinate system. This process ensures that the model can be correctly aligned and viewed from the perspective of the scanner.

[0083] Step (3): Based on the building prefabricated component model under the scanner's viewing angle, a virtual projection plane is constructed and projection calculation is performed on the triangular mesh on the prefabricated component.

[0084] Based on the consistent coordinate system setting, it is necessary to define a projection plane located between the scanner and the precast panel model. This plane is perpendicular to the z-axis of the scanner coordinate system. It should be emphasized that the angular resolution of the projection plane needs to be set according to the size of the projection plane. For example, if the size of the projection plane is set to 800*600, the angular resolution of the plane should be 800*600 pixels. Based on the set projection plane, the projection matrix M of the plane is determined according to the parameters of the projection plane. ProjTrans, used for coordinate dimensionality conversion, see formula (2). The parameters involved in the formula include field of view (FoV), aspect ratio (aspect), near plane (near), and far plane (far). Among them, (FoV) represents the field of view that the scanner can observe through the two-dimensional projection plane; aspect ratio (aspect) is the ratio of the width to the length of the projection plane, taking 800*600 as an example in this example; near plane (near) and far plane (far) are the closest and farthest two planes that the scanner can detect.

[0085]

[0086] Based on the projection matrix M ProjTrans , the three-dimensional vertex coordinates can be converted into two-dimensional projection coordinates through formula (3).

[0087]

[0088] in, is the vertex coordinate of a mesh of the prefabricated component model under the scanner’s viewing angle. The two-dimensional projection coordinate of the vertex after the projection matrix transformation is expressed as The projection transformation from three-dimensional coordinates to two-dimensional coordinates is as follows Figure 4 shown.

[0089] Step (4) is to filter out the triangulated meshes visible to the prefabricated components at any position of the scanner by using the z-Buffer algorithm. The processing process is as follows: Figure 5 shown.

[0090] It should be noted that in step (3), each pixel covered by the 3D triangulated mesh projected onto the 2D projection plane has a corresponding depth value (also known as the z-Buffer value). In other words, the z-Buffer value is the distance between all triangulated meshes on the 3D model and the 2D projection plane. Based on the z-Buffer value of each mesh on the precast panel model, this section uses the z-Buffer algorithm to filter out the triangulated meshes visible at any position on the scanner.

[0091] Based on the 2D projection plane, each pixel on the 2D projection plane is first assigned an initial z-Buffer value and set to blue. This step is to initialize the depth buffer. The initial z-Buffer value is usually set to infinity to represent the farthest possible distance, and new values ​​are assigned during subsequent updates.

[0092] Then, based on the initialized depth buffer, filter out the visible two-dimensional projection coordinates on the projection plane. The specific steps include:

[0093] S41, calculate and store the z-Buffer value of each pixel point corresponding to the projection of each three-dimensional triangular mesh on the prefabricated panel model onto the two-dimensional plane. Pi Represents a pixel The z-Buffer value;

[0094] S42, for all pixels on the projection plane, filter out visible meshes according to the size of the z-Buffer value. At a single pixel, if the z-Buffer value calculated for the three-dimensional mesh is less than the z-Buffer value recorded for the current pixel, then update the z-Buffer value at the pixel; otherwise, retain it. This ensures that when the two-dimensional projections of different meshes overlap, only the z-Buffer value of the current pixel calculated by the mesh closest to the scanner is retained. For example, the projections of meshes A and B on the projection plane partially overlap pixels. Since the pixel z-Buffer value (z-Buffer1) calculated from mesh A is smaller than the z-Buffer value (z-Buffer2) calculated from triangular mesh B, the z-Buffer value calculated from mesh A is taken as the z-Buffer value at the pixel (z-Bufferp Pi =z-Buffer1), such as Figure 5 As shown in (a) in .

[0095] Through this operation, the z-buffer value at each pixel is the depth value of the mesh closest to the projection plane. This mesh will be retained and considered the visible mesh in the current scanner view. Alternatively, for each pixel on the projection plane, meshes with larger depth values ​​or farther away are invisible in this scanner view.

[0096] At S43, at the same time, all pixels with finite z-Buffer values ​​are two-dimensional visible pixels, and the color of the current pixel is updated to red. The same method is used to represent the effect of applying the z-Buffer algorithm to the entire projection plane. Figure 5 (b) in the figure shows the effect of applying the z-Buffer algorithm.

[0097] S44, based on the visible two-dimensional pixels in the scanner coordinate system, two inverse matrix transformations are used to restore the two-dimensional visible pixels retained on the projection plane to three-dimensional point cloud coordinates, such as Figure 5 As shown in (c) in the figure; if the coordinate point of the 3D point cloud falls on a visible mesh, the mesh is considered visible, thus filtering out the visible meshes on the 3D model, such as Figure 5 As shown in (d) in .

[0098] Step (5): Based on all possible scanner placement locations, a heuristic algorithm is used to combine different scanner perspectives so that the scanning coverage of the prefabricated panel model reaches a specified threshold (e.g., 95%). For each scanner combination that reaches the threshold, the point cloud of the visible triangulated mesh under its perspective is simulated, and finally the point cloud data of the prefabricated component is generated.

[0099] Based on the three-dimensional mesh that can be detected under the scanner's viewing angle at any position calculated in step (4), the precast panel model is fully scanned and a simulated point cloud is generated by combining the scanner's viewing angles at different positions. The specific steps are as follows:

[0100] S51, based on the spatial constraints of the scene, generates multiple potential scanning positions and calculates the mesh positions that can be observed under each potential scanner's viewing angle. Finally, a heuristic algorithm is used to determine the scanner combination that achieves a 95% coverage rate for prefabricated components. This step can simulate the actual scanning conditions of multiple points and multiple viewing angles in real scenes. Figure 6 All hollow points in (a) represent potential scanning positions generated according to spatial constraints, and the red ones are the scanner combination positions selected by the heuristic algorithm.

[0101] S52: For each scanning point in the selected scanner position combination, a two-dimensional plane parallel to the prefabricated board is created, and the two-dimensional scanning point Si (X Si ,Y Si ).

[0102] S53, for the triangulated mesh visible in the current scanner viewing angle, the three-dimensional point cloud coordinates si (x si ,y si ,z si ),like Figure 6 Based on the two-dimensional coordinates obtained by the generated mesh projection, the intersection of the straight line passing through the scanner and the scanning point and the line cluster can simulate the three-dimensional coordinates of the visible mesh under the current scanner perspective.

[0103] According to the above steps, the three-dimensional point cloud coordinates of the corresponding visible triangulated mesh are calculated for the scanner position in each combination, and finally the point cloud data generation of the building prefabricated component model is completed, such as Figure 6 As shown in (c) in .

[0104] Based on the same technical concept as the method embodiment, the present invention also provides a point cloud generation system for deep learning semantic segmentation of building prefabricated components, comprising:

[0105] The document preprocessing module is used to convert the building code requirements in PDF format into an unstructured Word document for easy editing and conduct a completeness review. Based on the Word document that has been reviewed and found to be correct, the document content is divided into three parts: prefabricated component category, corresponding size and geometric characteristics, and a structured text of building prefabricated component rules is obtained;

[0106] A 3D model generation module is used to create adaptive family files in Revit software based on structured prefabricated building component rule text, build a secondary development environment configuration, and automatically generate 3D models of prefabricated building components in STL format based on the configured secondary development environment;

[0107] The triangulated mesh detection module is used to move the automatically generated STL format precast component model to the world coordinate system origin, convert the world coordinate system to the scanner coordinate system through coordinate system transformation, and calculate the triangulated mesh visible in the STL format precast component model at any scanner position by projecting the three-dimensional coordinates onto the two-dimensional plane based on the consistent coordinate system settings and using the depth value screening in the z-buffer algorithm;

[0108] The scanning position combination module is used to generate a series of potential scanner positions according to the scene conditions, calculate the visible triangulated mesh of the prefabricated components at each scanner position, and select the scanner position combination that makes the visibility rate of the prefabricated components reach a specified threshold;

[0109] The point cloud data generation module is used to simulate and generate the point cloud data of the corresponding visible triangulated mesh for each scanning position of the selected scanner position combination according to the scanner characteristics and the relative position relationship between the prefabricated component and the scanner, thereby generating the point cloud data of the entire prefabricated component.

[0110] It should be understood that the point cloud generation system for deep learning semantic segmentation of prefabricated building components in the embodiment of the present invention can implement all the technical solutions in the above-mentioned method embodiment, and the functions of its various functional modules can be specifically implemented according to the method in the above-mentioned method embodiment. The specific implementation process can refer to the relevant description in the above-mentioned embodiment, and will not be repeated here.

[0111] The present invention also provides a computer device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the point cloud generation method for deep learning semantic segmentation of building prefabricated components as described above are implemented.

[0112] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the point cloud generation method for deep learning semantic segmentation of building prefabricated components as described above.

[0113] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus (systems), computer devices, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] The present invention is described with reference to flowcharts of methods according to embodiments of the present invention. It should be understood that each process in the flowcharts and combinations of processes in the flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts. Figure 1 A device that specifies functions in a process or multiple processes.

[0115] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A function specified in a process or multiple processes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 The steps of a specified function in a process or multiple processes.

Claims

1. A point cloud generation method for deep learning semantic segmentation of building prefabricated components, characterized in that: The following steps are involved: Convert the building code requirements in PDF format into an unstructured word document that is easy to edit, and conduct a completeness review. Based on the word document that has been reviewed correctly, divide the document content into three parts: prefabricated component category, corresponding size, and geometric characteristics, and obtain a structured building prefabricated component rule text; Based on the structured rule text of building prefabricated components, an adaptive family file is established in the Revit software, a secondary development environment configuration is constructed, and a three-dimensional model of the building prefabricated components in the stl format is automatically generated based on the configured secondary development environment; The automatically generated prefabricated component model in stl format is moved to the origin of the world coordinate system, and the world coordinate system is converted into the scanner coordinate system through coordinate system conversion. Based on the consistent coordinate system setting, the three-dimensional coordinates are projected onto the two-dimensional plane, and the depth value screening in the z-Buffer algorithm is used to calculate the visible triangulated mesh in the prefabricated component model in stl format at any scanner position; Generate a series of potential scanner positions according to the scene conditions, calculate the visible triangulated meshes of the prefabricated components under the scanner positions one by one, and select the scanner position combination that makes the visibility rate of the prefabricated components reach the specified threshold; For each scanning position of the selected scanner position combination, point cloud data of the corresponding visible triangulated mesh is simulated and generated according to the scanner characteristics and the relative position relationship between the prefabricated component and the scanner, thereby generating point cloud data of the entire prefabricated component.

2. The method according to claim 1, characterized in that The secondary development environment based on the configuration automatically generates 3D models of prefabricated building components in STL format, including: Based on the secondary development configuration environment, according to the structured building prefabricated component rule text, for each prefabricated component category and corresponding size, determine the overall length, width and height, and determine the edge of the three-dimensional model; Based on the edge of the three-dimensional model, according to the geometric features in the structured building prefabricated component rule text, the geometric features are expressed by hollow or solid stretching bodies to obtain the three-dimensional model of the prefabricated component in rvt format; Convert the 3D model in rvt format into a 3D model in stl format consisting of triangulated meshes.

3. The method according to claim 1, characterized in that The world coordinate system is converted to the scanner coordinate system through coordinate system conversion. The conversion formula is as follows: In the formula, is the coordinate of the vertex of the i-th triangular mesh on the prefabricated component model in the world coordinate system; is the coordinate in the scanner coordinate system after coordinate system conversion; M ScanTrans is the transformation matrix, which is obtained by performing matrix multiplication operations on the rotation matrix and the flip matrix in sequence.

4. The method according to claim 1, characterized in that: Based on the consistent coordinate system setting, by projecting the 3D coordinates onto the 2D plane and using the depth value screening in the z-Buffer algorithm, the triangulated mesh visible in the precast component model in stl format at any scanner position is calculated, including: A two-dimensional projection plane located between the scanner and the prefabricated component model is defined, the plane is perpendicular to the z-axis of the scanner coordinate system, a projection matrix is ​​determined according to the parameters of the projection plane, and the three-dimensional vertex coordinates of the triangular mesh are converted into two-dimensional projection coordinates based on the projection matrix; Each pixel covered by the three-dimensional triangular mesh projected onto the two-dimensional projection plane has its corresponding depth value, called the z-Buffer value. Based on the two-dimensional projection plane, an initial z-Buffer value is assigned to each pixel on the two-dimensional projection plane to form an initialized depth buffer. Based on the initialized depth buffer, the z-Buffer value of each pixel point corresponding to each 3D triangular mesh on the prefabricated component model projected onto the 2D plane is measured and stored. For each pixel point on the projection plane, if the z-Buffer value corresponding to the current triangular mesh is less than the previously recorded z-Buffer value, the z-Buffer value at the pixel point is updated to the calculated value; otherwise, it is retained; The two-dimensional visible pixels are determined according to the final z-Buffer value of each pixel on the projection plane. Through two inverse matrix transformations, the coordinates of the two-dimensional visible pixels retained on the projection plane are restored to three-dimensional point cloud coordinates, thereby screening out the visible triangular mesh on the three-dimensional model.

5. The method according to claim 4, characterized in that The projection matrix is ​​determined according to the parameters of the projection plane. The calculation method is as follows: Among them, M ProjTrans Represents the projection matrix; FoV represents the field of view that the scanner can observe through the two-dimensional projection plane; aspect is the ratio of the width and length of the projection plane; near and far are the nearest and farthest two planes that the scanner can detect; The three-dimensional vertex coordinates of the triangular mesh are converted into two-dimensional projection coordinates based on the projection matrix, which is expressed as: in, is the vertex coordinate of the i-th triangular mesh of the prefabricated component model under the scanner’s viewing angle. The two-dimensional projection coordinate of the vertex after the projection matrix transformation is expressed as 6. The method according to claim 1, characterized in that Generate a series of potential scanner positions according to the scene conditions, calculate the visible triangulated meshes of the prefabricated components under the scanner positions one by one, and select the scanner position combination that makes the visibility rate of the prefabricated components reach the specified threshold, including: According to the spatial constraints of the scene, multiple potential scanning positions are generated and the triangulated mesh positions that can be observed under each potential scanner viewing angle are calculated. The scanner position combination that makes the prefabricated component coverage rate reach 95% is determined through a heuristic algorithm.

7. The method according to claim 1, characterized in that According to the characteristics of the scanner and the relative position relationship between the prefabricated components and the scanner, the point cloud data of the corresponding visible triangulated mesh is simulated and generated, including: For each scanning point in the selected scanner position combination, a two-dimensional plane parallel to the prefabricated component is created, and the two-dimensional scanning point on the created plane is calculated by generating a relative position relationship between the scanner and the plane and parameters set by the scanner; For the triangulated mesh visible under the current scanner, the three-dimensional point cloud coordinates are simulated one by one by calculating the intersection points of the triangulated mesh with the straight line formed by the scanner and the two-dimensional scanning points; For each scanner position in the combination, the three-dimensional point cloud coordinates of the corresponding visible triangulated mesh are calculated, and finally the point cloud data generation of the building prefabricated component model is completed.

8. A point cloud generation system for deep learning semantic segmentation of building prefabricated components, characterized in that: include: The document preprocessing module is used to convert the building code requirements in PDF format into an unstructured word format document that is easy to edit and conduct a completeness review. Based on the word document that has been reviewed correctly, the document content is divided into three parts: prefabricated component category, corresponding size and geometric characteristics, and a structured building prefabricated component rule text is obtained; The 3D model generation module is used to establish an adaptive family file in Revit software based on the structured rule text of building prefabricated components, build a secondary development environment configuration, and automatically generate a 3D model of the building prefabricated components in the stl format based on the configured secondary development environment; The triangulated mesh detection module is used to move the automatically generated prefabricated component model in stl format to the origin of the world coordinate system, convert the world coordinate system into the scanner coordinate system through coordinate system conversion, and calculate the visible triangulated mesh in the prefabricated component model in stl format at any scanner position by projecting the three-dimensional coordinates onto the two-dimensional plane and filtering by the depth value in the z-Buffer algorithm based on the consistent coordinate system setting; A scanning position combination module is used to generate a series of potential scanner positions according to the scene conditions, calculate the visible triangular meshes of the prefabricated components under the scanner positions one by one, and select the scanner position combination that makes the visibility rate of the prefabricated components reach a specified threshold; The point cloud data generation module is used to simulate and generate the point cloud data of the corresponding visible triangular mesh for each scanning position of the selected scanner position combination according to the scanner characteristics and the relative position relationship between the prefabricated component and the scanner, thereby generating the point cloud data of the entire prefabricated component.

9. A computer device, characterized in that: include: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the point cloud generation method for deep learning semantic segmentation of building prefabricated components are implemented as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the point cloud generation method for deep learning semantic segmentation of building prefabricated components are implemented as described in any one of claims 1 to 7.

Images