Bridge component point cloud intelligent recognition and extraction method for prefabricated beam field complex environment

Abstract

The application discloses a bridge component point cloud intelligent identification and extraction method for a prefabricated beam field complex environment, and the method comprises the following steps: in the co-evolution initialization stage, an initial data set is constructed through a high-fidelity synthetic point cloud data generation process, and a memory-enhanced reasoning network is trained to generate the first batch of adversarial data to form an initial training pool; then, the co-evolution cycle stage is entered, the adversarial environment simulator dynamically generates targeted adversarial data according to the performance of the memory-enhanced reasoning network under complex engineering challenges such as component boundary blur and point cloud loss to enhance the model robustness, and the performance is evaluated on the validation set to calculate the reward and optimize the simulator strategy, and the cycle iteration is continued until the network performance converges; finally, the trained network is used for accurate identification and extraction of the real scanned point cloud. The application realizes high-precision and high-robustness automatic identification and extraction of bridge component point cloud data from complex environments such as dense stacking and background interference.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent construction technology, and in particular relates to a method for intelligent identification and extraction of point clouds of bridge components in the complex environment of precast beam yards. Background Technology

[0002] With the rapid development of prefabricated construction technology in my country's bridge engineering, prefabricated assembly structures have been widely used in bridge engineering due to their advantages such as high industrialization level and short construction cycle. Accurate identification and extraction of various bridge components in prefabrication yards are of great significance for improving hoisting efficiency, realizing digital inventory management, and building a quality traceability system. Traditional manual inventory methods rely on visual inspection and manual recording, which suffers from low efficiency, high omission rate, and insufficient spatial positioning accuracy. This makes it difficult to achieve automatic three-dimensional pose analysis of bridge components in complex environments, thus hindering the improvement of intelligent management in bridge engineering.

[0003] In recent years, 3D laser scanning technology has provided a new approach for the full-domain digitization of bridge engineering, enabling the rapid acquisition of large-scale point cloud data through non-contact measurement, laying the foundation for the automated identification and extraction of bridge components. However, in the process of using point cloud data to identify components, the complexity of the actual engineering environment, the disorder of the data, and its susceptibility to interference pose significant challenges to segmentation accuracy: bridge components are often stacked in multiple layers, with severe mutual occlusion; numerous interfering objects such as surrounding construction equipment, temporary supports, and workers lead to missing point clouds and significant noise; the components have diverse geometric shapes and similar boundary features, and traditional point cloud segmentation methods based on geometric features are prone to missegmentation and boundary breaks due to insufficient sensitivity to local features under complex occlusion and stacking conditions, making it difficult to achieve complete extraction of components.

[0004] While deep learning-based point cloud segmentation methods (such as PointNet++) have shown potential for automated processing, they still have significant shortcomings in bridge component identification and extraction tasks. These methods heavily rely on large amounts of high-quality labeled data, but labeling real bridge point clouds is costly, time-consuming, and limited by on-site conditions, making it difficult to cover all complex working conditions. Traditional synthetic data generation methods are mostly random and static processes, unable to strengthen the model by addressing its weaknesses in challenging scenarios such as occlusion and singular poses, resulting in insufficient model generalization ability.

[0005] Therefore, existing methods struggle to achieve stable and accurate extraction of various components in the complex environment of real-world bridge engineering, hindering subsequent applications such as dimensional inspection, digital management, automated hoisting, and digital twin modeling. To address these issues, this invention proposes an intelligent point cloud recognition and extraction method for bridge components in the complex environment of precast beam yards. By dynamically generating targeted training samples and using memory-enhanced inference, it achieves highly robust extraction of key bridge components from the complex environment of precast beam yards, providing a technical foundation for intelligent management throughout the entire lifecycle of bridge engineering projects. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent identification and extraction method for bridge component point clouds in complex environments such as precast beam yards, which solves the problems of existing technologies being highly dependent on real labeled data and lacking robustness in segmentation under complex scenarios such as occlusion and stacking, and achieves high-precision, high-generalization, automated semantic segmentation of bridge component point clouds.

[0007] The technical solution adopted in this invention is a method for intelligent identification and extraction of point clouds of bridge components in complex environments of precast beam yards, comprising the following steps:

[0008] S1. Construct an initial synthetic point cloud dataset through a high-fidelity synthetic point cloud data generation process; train a memory-enhanced inference network using the initial synthetic point cloud dataset to obtain an initial model; construct an initial state based on the prediction results of the initial model and the real labels, input it into the policy network of the adversarial environment simulator to output action vectors, and generate the first batch of adversarial data; merge the initial synthetic point cloud dataset and the adversarial data to form the first training pool.

[0009] S2, Repeat the following steps until the performance of the memory-enhanced inference network converges: The adversarial environment simulator constructs a state based on the current performance of the memory-enhanced inference network and outputs action vectors to generate a new round of adversarial data; The newly generated adversarial data is added to the cumulative training pool to form an updated cumulative training pool, and the memory-enhanced inference network is retrained using the cumulative training pool; The performance of the updated network is evaluated on a fixed validation set reserved from the newly generated data in this round, and the reward is calculated; The policy network of the adversarial environment simulator is updated using the reward.

[0010] S3 uses a trained memory-enhanced inference network to segment real-world scanned point cloud data.

[0011] Furthermore, the generation of the high-fidelity synthesized point cloud data includes:

[0012] A parameterized bridge component BIM model generation module and scene object library are constructed. Based on the key parameters of the bridge component cross-section, a three-dimensional geometric model with semantic tags is dynamically generated through programmatic scripts. Programmatic scene building and physical simulation are performed to automatically generate a scene model containing bridge components and obstacles. The parameters and spatial position of the virtual laser scanner are set, and a synthetic point cloud is generated through light projection. The synthetic point cloud is processed and samples are generated, noise is injected, and it is saved as a point cloud data file with semantic tags.

[0013] Furthermore, the key parameters of the bridge component cross-section include geometric dimensional parameters, internal structural parameters, and orientation parameters;

[0014] The programmed script reads parameters through a parameter input interface and automatically completes 3D modeling;

[0015] The parameters of the virtual laser scanner include horizontal field of view, vertical field of view, angular resolution, and ranging error; the scanner position is defined by three-dimensional spatial coordinates, and point clouds are generated through multi-view collaborative generation.

[0016] Furthermore, the forward propagation and reasoning of the memory-enhanced inference network includes: processing the input point cloud data through the SetAbstraction layer to obtain semantic features; calculating the similarity weight between each local feature vector extracted from the semantic features and the prototype vector in the external memory matrix; reading information from the memory bank based on the similarity weight to obtain memory-enhanced features; and concatenating the memory-enhanced features with the original features and then upsampling and classifying them to output semantic labels.

[0017] Furthermore, the similarity weight is calculated using dot product attention and the softmax function. Specifically, it is calculated by dividing the dot product of the local feature vector and the transpose of the memory prototype vector by the square root of the feature dimension, and then normalizing it using the softmax function.

[0018] Furthermore, in S1, the training of the memory-enhanced inference network uses a boundary reinforcement joint loss function, which includes a weighted cross-entropy loss, a Lovász-Softmax loss, and a boundary penalty term, with the total loss being a weighted sum of the loss terms;

[0019] The weights of the weighted cross-entropy loss range from [0.5, 1.5], the weights of the Lovász-Softmax loss range from [1.0, 2.0], and the weights of the boundary penalty term range from [0.1, 0.5].

[0020] Furthermore, in S2, the state of the adversarial environment simulator is constructed as a multi-dimensional feature map, including three channels: a channel for identifying incorrect prediction points, a channel for identifying low-confidence regions, and a channel for the normalized distance from the prediction point to the class boundary.

[0021] The action vectors output by the adversarial environment simulator are encoded scene parameters, including the pose adjustment parameters of bridge components, the pose adjustment parameters of occluders, and the position adjustment parameters of the virtual scanner.

[0022] Furthermore, the action vector is used to adjust the scene parameters and then re-process the high-fidelity synthesized point cloud data to generate a new round of labeled adversarial data.

[0023] Furthermore, in S2, the calculation of the reward specifically involves:

[0024] Save a snapshot of the current parameters of the memory-enhanced inference network ; Using newly generated adversarial data Train and update the network; compute the trained and updated network on a fixed validation set. Average crossover ratio The average intersection-union ratio of the parameter snapshot on the same validation set The difference ; Calculate the average information entropy of the prediction results of the parameter snapshot on the validation set. ; Calculate the reward value according to the formula:

[0025]

[0026] in, This is the reward value; The hyperparameters for balancing the weights; Indicates that the model uses a parameter snapshot For the validation set The predicted probability distribution.

[0027] Furthermore, the hyperparameters in the weighted sum are initially set to 1.0 and adjusted according to the performance of the validation set: if the generated data is invalid, the hyperparameters are lowered; if the model learns slowly, the hyperparameters are raised.

[0028] The beneficial effects of this invention are:

[0029] 1. This invention effectively reduces the cost and threshold of data preparation. Only a small initial dataset needs to be constructed, and subsequent massive training data are automatically generated by the Adversarial Environment Simulator (AES). This overcomes the bottlenecks of difficult real point cloud data collection and extremely high annotation costs, and provides a practical data solution for the application of deep learning in vertical fields such as bridge component recognition and extraction.

[0030] 2. This invention significantly improves the quality and efficiency of training data and the final performance of the model. Through the unique reward mechanism of AES, it can intelligently evaluate and accurately focus on the weakest link of the model, dynamically generate high-value challenge samples for complex scenarios such as boundary blurring caused by dense stacking and point cloud fragmentation caused by severe occlusion, avoid computational waste on a large number of simple or irrelevant samples, thereby improving data utilization and training efficiency, and enabling the model to maintain high segmentation accuracy under occlusion and stacking conditions.

[0031] 3. This invention significantly enhances the robustness of the model in complex scenarios. By using the external memory bank of the Memory-Enhanced Inference Network (MARN) to store prototype features of multiple scenarios, it can actively perform feature retrieval and completion when faced with point cloud missing due to densely stacked intersection regions or severe occlusion that have not been seen during training, rather than relying solely on the local receptive field. This significantly improves the accuracy and reliability of bridge component extraction in extremely complex environments.

[0032] 4. This invention achieves full automation and intelligence from data generation to model training. By integrating the two into a closed-loop system through a co-evolutionary framework, the system can automatically evaluate the model status, generate solutions, and complete model evolution without the need for repeated manual intervention in data generation strategies or adjustments to the network structure. This significantly reduces the operational difficulty and maintenance cost of the technology application. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart illustrating the method of the present invention.

[0035] Figure 2 This is a schematic diagram of the generated point cloud dataset of bridge components (taking a concrete box girder as an example).

[0036] Figure 3 This is a visual diagram of the identification results of bridge components (taking a concrete box girder as an example). Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] like Figure 1 As shown in the embodiment, a method for intelligent identification and extraction of bridge component point clouds in the complex environment of a precast beam yard is provided. This method includes three core modules: high-fidelity synthetic point cloud data generation (step S1), design and training of a memory-enhanced inference network (step S2), and construction, operation, and optimization of an adversarial environment simulator (step S3). These three modules work together to form a complete automated training system. The automated training and inference application process of this system mainly includes the following three stages:

[0039] (1) Co-evolution initialization: First, execute the high-fidelity synthetic point cloud data generation process described in step S1 to manually construct a small-scale, representative initial synthetic point cloud dataset. The initial set is much smaller than the final training set and is only used for system startup; then, step S2 is executed, using... The MARN algorithm is trained independently until convergence to obtain an initial model with basic segmentation capabilities. The final MARN model weights and memory M are saved for subsequent inference on real scanned point clouds. Based on the initial model... The predicted results are compared with the true labels to construct the first round of states. The input is then fed into a randomly initialized AES policy network, which outputs the first action vector. During the system startup phase, this action can be viewed as an exploratory perturbation of the initial scene parameters, based on the simulation environment. The included parameter adjustment instructions generate the first batch of adversarial data. ; Initial dataset and The pool is merged to form the training pool for the first round of iterations. Complete system initialization.

[0040] (2) Co-evolutionary Iteration: The system enters a cyclic iteration. In each training round t, step S3 is executed first: AES is based on the current MARN model on the data. The state constructed by the representation on the surface Generate Actions This will generate a new round of adversarial data. Subsequently, step S2 is executed to process the newly generated data. Add to the cumulative training pool (which is a dataset that expands over time, containing data from the initial dataset). The system uses all data generated up to the current round and employs this cumulative training pool to train and update the MARN model's parameters. Next, step S32 is executed to evaluate the current MARN performance and calculate the reward on a fixed validation set reserved from the newly generated data in this round. Finally, execute step S33 and utilize the reward. Update the policy network of AES. If MARN performance has converged (e.g., after multiple consecutive rounds). Increase below the set threshold If the training is terminated, then the training will be stopped; otherwise, the training will be stopped. The cycle continues.

[0041] (3) Model inference application: After training, the final MARN model can be used to identify and extract bridge components from real scanned, unlabeled precast beam yard point cloud data. Its inference process is the forward propagation process defined in step S2, including: performing standardization preprocessing such as downsampling and denoising on the input point cloud; inputting the preprocessed point cloud into the deployed MARN inference model; MARN performs forward propagation, extracts features and queries its external memory M for feature completion and enhancement; the network outputs the probability that each point belongs to "bridge component" (category "1") or "background" (category "0"); assigns semantic labels to each point, generates the final segmentation result map, and realizes the automatic identification and localization of bridge component instances.

[0042] To realize the above-mentioned automated co-evolutionary training and inference application process, the specific implementation methods of the three core steps will be explained below.

[0043] Step S1: High-fidelity synthetic point cloud data generation: This step is fundamental to achieving high-performance deep learning. A small-scale initial synthetic point cloud dataset is manually constructed, and subsequent large-scale training data is dynamically generated using an adversarial environment simulator (AES). This process, based on BIM parametric modeling and physical simulation technology, simulates the physical characteristics and complex environment of a real laser scanner, generating point cloud data with accurate semantic labels in batches. Specifically, it includes the following steps:

[0044] S11: Construct a parametric bridge component BIM model generation module and scene object library: Through programmatic scripts, dynamically generate a three-dimensional geometric model that conforms to design specifications based on the key parameters of the bridge component cross-section. The size and position of internal prestressed ducts and other structures are controlled by parameters. At the same time, three-dimensional models of various dynamic obstacles such as gantry cranes, beam transport vehicles, support frames and construction personnel are prefabricated and semantic labels are predefined. Bridge components are labeled as "1" and the background of the rest is "0".

[0045] To achieve the flexibility and automation of the parametric modeling described above, the script provides a standardized parameter input interface, supporting the reading of key cross-sectional parameters of components from external configuration files (such as JSON and YAML formats), databases, or graphical user interfaces (GUIs). These parameters are passed in as variables and mainly cover the following categories: geometric dimension parameters, internal structural parameters, and pose parameters, including the initial position of the component in the global coordinate system. , , The script encapsulates the logic for dynamically constructing a 3D geometric model based on the aforementioned parameter variables, such as length L, height H, and width W, through extrusion and lofting algorithms to generate the main shape of the component. This process requires no manual intervention from the modeling software interface and is entirely automated by the script.

[0046] S12: Procedural Scene Construction and Physics Simulation: Based on training requirements, procedural scene construction is completed using a policy or AES-driven approach, specifically including:

[0047] The main control script calls the BIM model library and scene object library to automatically generate bridge component and obstacle models according to the input dimensions, positions, and attitude parameters, completes stacking and layout, retains component semantic tags, and exports them as OBJ intermediate format; it then loads the scene file into the Open3D simulation environment and sets the virtual 3D laser scanner parameters, including the horizontal field of view (FOV). h Vertical field of view (FOV) v The angular resolution Δθ and ranging error are defined; simultaneously, the scanner's position in the simulated environment is defined, determined by the following formula:

[0048]

[0049] in, Indicates the spatial location of the scanner. , and Each virtual scanner is assigned its three-dimensional spatial coordinates. Using ray casting, rays are emitted from the positions of each virtual scanner. The nearest intersection point is calculated using an algorithm that intersects the rays with a triangular mesh, generating a synthetic point cloud containing coordinates, category, and other information. This represents the set of position coordinates for all scanners. Together, they form an optimized scanning solution for a specific scenario, capturing data collaboratively from multiple perspectives to generate comprehensive high-fidelity point cloud data.

[0050] S13: Data Processing and Sample Generation: Gaussian noise is injected to simulate surface roughness, and moving noise points such as workers and tools are randomly generated; the final point cloud data is saved in .txt format, containing the coordinates (X, Y, Z), color values ​​(R, G, B), and semantic label fields of the point cloud data. The coordinates (X, Y, Z) of each point represent the three-dimensional position of the point in the global coordinate system, which is the actual spatial position of the point on the surface of the bridge component calculated by the ray casting algorithm; among them, the color values ​​(R, G, B) are used to enhance point cloud feature extraction, and the semantic label field is used for training supervision; a preliminary initial synthetic point cloud sample set is generated. This includes data from multiple independent scenarios, each scenario containing several bridge components. For example... Figure 2 The image shows a schematic diagram of a high-fidelity bridge component synthetic point cloud dataset generated by the method of the present invention, taking a concrete box girder as an example.

[0051] Step S2: Design and training of the Memory-Enhanced Reasoning Network (MARN): This network performs point cloud semantic segmentation using a feedforward approach. Its training process is standard supervised learning, with the core being the introduction of a differentiable external memory query mechanism to achieve the storage and reuse of historical scene features. Specifically, it includes the following processes:

[0052] S21: Forward Propagation and Reasoning

[0053] Input a batch of point cloud data of size N×C. The high-level semantic features are obtained by sequentially performing downsampling and local feature extraction through multiple Set Abstraction (SA) layers. Where N represents the total number of 3D spatial points contained in the input single batch of point cloud data, and C represents the number of feature channels for each point. In the general implementation, C is 6, which means that it includes the 3D coordinates (X, Y, Z) and color information (R, G, B) of the point. The corresponding semantic label data (size N×1) is used as a separate supervision target and does not participate in the forward propagation of the network. It is only used for subsequent loss calculation and parameter update. The Set Abstraction layer, whose structure and function refer to the corresponding layer in the PointNet++ network, is mainly used to downsample the input point cloud and extract multi-level local semantic features.

[0054] A trainable external memory matrix M with dimensions K×d is introduced; where K is the preset total number of prototype vectors, representing the number of typical scene feature patterns that the memory can store; and d is the feature dimension of each prototype vector, which is the same as the local feature vector extracted by the network. The dimensions are the same; matrix M is initialized using the Xavier uniform distribution method, i.e., from the distribution... Mid-sampling is used to ensure training stability; each row in the matrix (j=1,2,...,K) represents a prototype vector used to encode and store local features of a typical type of bridge component.

[0055] Perform memory enhancement operations on Each local feature vector ,calculate With each prototype vector in the external trainable memory matrix M Similarity weight :

[0056]

[0057] in, This represents a scaling factor used to scale values ​​when calculating dot product attention, stabilizing the gradient and preventing the inner product result from becoming too large. Gradient vanishing; The function's purpose is to transform the similarity between features and memory prototypes into weight assignments, and its output value is... Characterizes the current local features The degree of relevance to the j-th prototype in memory; the higher the weight, the more important the prototype is in providing information to complete or enhance the current feature; T represents the relevance of the memory prototype vector. Perform a transpose operation.

[0058] Information is retrieved from the memory based on similarity weights:

[0059]

[0060] in, Indicates attention weights The memory-enhanced feature retrieved for the i-th feature vector is obtained by weighted summation of all prototype vectors in the memory matrix.

[0061] The enhanced features are obtained by concatenating the read memory features with the original features:

[0062]

[0063] in, This represents the final output enhanced feature vector, derived from the original local features. Its corresponding memory characteristics It is assembled along the feature dimension.

[0064] The enhanced features are upsampled to the scale of the input point cloud by the Feature Propagation (FP) layer, and finally the semantic labels of each point are predicted by a fully connected classifier.

[0065] S22: Loss Calculation and Parameter Update:

[0066] Calculate the joint loss function of boundary reinforcement based on network prediction results and real labels. :

[0067]

[0068] in, For weighted cross-entropy loss, For Lovász-Softmax loss, For boundary penalty terms; The weights for the weighted cross-entropy loss are [0.5, 1.5]. The values ​​are the weights for the Lovász-Softmax loss, ranging from [1.0, 2.0]. The weight value for the boundary penalty term ranges from [0.1, 0.5]. =1.0, =1.5, =0.2 is the baseline configuration. This set of parameters can be used as the initial values ​​for model training, and subsequent optimization should be guided by the performance on the validation set.

[0069] The loss is calculated using the gradient descent algorithm. The gradients of the MARN backbone network parameters and the memory matrix M are calculated, and end-to-end parameter optimization is performed. The memory matrix M is updated only during the training phase, using the Adam optimizer (with the learning rate synchronized with the network parameters), and the prototype vector is adjusted through gradient backpropagation. During the inference phase, M remains frozen to ensure the stability of the segmentation process. This mechanism enables the model to dynamically learn the point cloud features of bridge components, providing a consistent foundation for subsequent applications such as size detection. Figure 3 The image shows a schematic diagram of the MARN model, trained as described above, identifying bridge components from real precast beam yard point cloud data.

[0070] Step 3: Construction and Operation of the Adversarial Environment Simulator (AES): In this invention, the Adversarial Environment Simulator (AES) serves as an intelligent agent system. Its core function is to evaluate the weaknesses of the current segmentation model and dynamically generate challenging samples for targeted attacks to improve the model's robustness. The operation of AES is iteratively performed alternately with the training process of MARN (segmentation model). Each iteration includes four core stages: state acquisition, action generation and environment execution, reward calculation, and policy network update. The specific implementation method is as follows:

[0071] S31: State Acquisition and Action Generation:

[0072] In the t-th iteration, AES receives data from the MARN network based on the data generated in the previous iteration. The obtained prediction results are compared with the true labels to construct the state. This state is a multidimensional feature map of size N×3, where N is the data from the previous round. The number of point clouds in the dataset. This feature map is composed of three channels concatenated along the feature dimension, collectively representing the prediction weaknesses of the MARN model on the current data:

[0073] (a) Error prediction point identification channel: Compare the prediction results with the true labels. Error prediction points are identified as 1, and correct points are identified as 0.

[0074] (b) Low-confidence region identification channel: based on MARN prediction entropy Calculate the low-confidence region; where, This represents the predicted probability value of the MARN model for the current point cloud point belonging to the i-th category; i represents the index category, i=1 represents the "bridge component" category, i=2 represents the "background" category, and the entropy above a set threshold is marked as 1, and the rest as 0. This channel is used to identify areas where the model decision is ambiguous and the confidence level is low;

[0075] (c) Normalized distance channel from predicted point to class boundary: Calculate the normalized distance from each predicted point to the nearest boundary of its predicted class region, within the range [0,1]. This channel quantifies how close each point is to the classification boundary; the smaller the distance, the more likely the point is to be in a confusing boundary zone.

[0076] AES policy network by state The network takes actions as input and outputs them. The probability distribution, whose probability space is related to the scene parameter vector. The value space is consistent. By sampling from this distribution, the specific action vector to be finally executed is obtained. = This vector The scene parameters required for the next batch of training data are encoded, and their specific composition is as follows: the first 6 The elements are The pose adjustment parameters for each bridge component (each bridge component includes 3 translations and 3 rotations); the following 6M elements are the pose adjustment parameters for M occluders; the final 3 The elements are Adjust the position parameters of the virtual scanner.

[0077] Simulation environment call action Based on the encoded parameter adjustment instructions, the system automatically updates the poses of bridge components, obstructions, and the position of the virtual scanner. Subsequently, based on these updated parameters, the system automatically re-executes the high-fidelity synthetic point cloud data generation process described in step S1, including procedural scene construction, physical simulation, noise injection, and sample generation, outputting a new batch of synthetic data. For use in MARN forward propagation.

[0078] S32: Reward Calculation and Performance Evaluation

[0079] According to AES action Generate a new batch of training data Additionally, a small portion (e.g., 10%-20%) of the data is retained from this generation process and used as a fixed validation set. This data is used for reward calculation in this round. This approach ensures the consistency of data distribution between the evaluation and training environments, thereby guaranteeing the fairness of reward signal evaluation. The reward calculation process is as follows:

[0080] First, save a snapshot of the current parameters of the MARN model. Subsequently, using Perform a training update on the model; then, compute validation sets on the models before and after training, respectively. The average crossover ratio mIoU, the difference of which is denoted as Simultaneously, the average information entropy of the model's predictions on the validation set before training is calculated. This value measures the overall uncertainty of the model when faced with new data; ultimately, AES received a reward in this round. The calculation formula is determined by both performance improvement and prediction uncertainty:

[0081]

[0082] Among them, the difference The model is using newly generated data After training and updating, on a fixed validation set The average crossover ratio obtained above Compared with the snapshot of model parameters before the update Average crossover ratio obtained on the same validation set The difference between the two values ​​quantifies the net improvement in model segmentation performance resulting from the introduction of new data in this round; p represents the probability distribution of the MARN model's class predictions for point cloud data, and in the entropy calculation H(p), p refers to the parameter snapshot used by the model. For the validation set The predicted probability distribution; To balance the two weights, the hyperparameter is initially set to 1.0. This parameter needs to be adjusted based on the validation set performance: if the data generated by AES is abnormally distributed or invalid, and does not improve the model, it should be lowered. If the model learns slowly on certain challenging points (such as extreme occlusion), the speed can be increased appropriately. This is to encourage AES to generate more of these difficult samples.

[0083] S33: Policy Optimization and Network Update: AES employs the Proximal Policy Optimization (PPO) algorithm, with Adam as its core optimizer. Adam leverages Adam's adaptive learning rate to address the sparse reward and gradient sparsity issues in policy gradient training, thereby promoting stability and efficient convergence during the training process.

[0084] The key hyperparameter settings prioritize training consistency and are configured as follows: the initial learning rate is set to 3 × 10⁻. 4 It can be dynamically fine-tuned based on the fluctuations in the reward signal; the policy clip range is 0.2 (range [0.1, 0.3]), used to strictly constrain the policy update magnitude; the value function loss coefficient ( The value is 0.5 (range [0.5, 1.0]), used to balance the weights of value learning and policy optimization; the entropy reward coefficient ( The value is 0.01, used to encourage full exploration of the scene parameter space in the early stages of training; discount factor The value range is [0.9, 0.99], with an initial value of 0.95. During actual training, it can be dynamically fine-tuned based on the growth trend of the accumulated reward over multiple iterations: if the reward growth is too slow, it can be appropriately increased. This encourages agents to make longer-term plans; if oscillations or instability occur during training, the intensity should be appropriately reduced. .

[0085] In addition, the accompanying settings of gradient clipping (clip value=0.5), batch size (128 or 256), and the number of update rounds per sampling (epoch = 4) together constitute a complete and reproducible hyperparameter scheme, which coordinates multiple dimensions to ensure the training consistency and overall stability of AES in the process of co-evolution with the segmentation network.

[0086] Based on the collected state, action, and reward sequences Update strategy network parameters parameters Its objective function is to maximize the expected cumulative reward:

[0087]

[0088] in, This represents the objective function for maximizing the expected cumulative reward; E represents the mathematical expectation. Discount factor; Discount factor t raised to the power of t is used to calculate the present value of the reward for the next t steps at the current moment; This represents the reward obtained at time step t.

[0089] This invention addresses the technical bottlenecks in the segmentation and extraction of bridge components, proposing a systematic and innovative solution. Its core advantages are reflected in the following four aspects:

[0090] (1) To address the issues of high data preparation costs and difficult annotation, existing methods rely on manual collection and annotation, or use randomly synthesized data with distributions deviating from reality, resulting in high costs and low efficiency. This invention achieves automated and adaptive optimization of data generation by constructing an adversarial environment simulator (AES) based on reinforcement learning (PPO). AES dynamically generates challenging scenario parameters that maximize model performance gains by analyzing the current state of the model (such as prediction error and uncertainty regions); its reward function simultaneously considers short-term performance improvement and long-term exploration incentives, driving AES to generate high-value samples. This method effectively reduces the dependence on manually labeled data, realizes the automatic generation of massive amounts of high-quality training data, and significantly improves data utilization efficiency and model training feasibility.

[0091] (2) To address the problem of low-quality training samples and difficulty in strengthening model weaknesses, traditional synthesis methods mostly use static random generation, which cannot optimize samples to target model defects. This invention achieves precise targeting of weak links in the model through the intelligent reward mechanism of AES. AES is performance-oriented and dynamically adjusts scene parameters through reinforcement learning strategies. Its reward function ensures that the generated data directly targets performance bottlenecks and encourages exploration of high-uncertainty regions. This method significantly improves the relevance and effectiveness of training data, avoids resource waste on simple samples, and thus improves the model's segmentation accuracy and convergence speed.

[0092] (3) To address the issue of insufficient robustness of models in complex scenarios such as densely stacked intersection regions or severe occlusion, existing network structures struggle to perform effective inference when point clouds are severely missing. This invention introduces a differentiable external memory matrix into the Memory-Enhanced Inference Network (MARN), and uses an attention mechanism to dynamically query and fuse historical scene feature prototypes, achieving cross-sample information completion and inference enhancement. This mechanism enables the model to actively retrieve and utilize prior knowledge when facing complex situations such as occlusion, significantly improving the accuracy and stability of bridge component identification and extraction in extreme scenarios.

[0093] (4) To address the issues of high technical application thresholds and reliance on manual intervention, traditional methods require repeated manual adjustments to data generation strategies and network structures, resulting in low efficiency and poor reproducibility. This invention constructs a closed-loop system for the co-evolution of AES and MARN, achieving full automation from data generation to model training. The reward signal of AES directly originates from the performance feedback of MARN, forming a self-driven optimization loop; the accompanying automatic hyperparameter setting strategy further ensures system stability. This design significantly reduces the need for manual intervention, shortens the model development cycle, and improves the system's practicality and deployability.

[0094] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0095] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for intelligent identification and extraction of point clouds of bridge components in complex environments of precast beam yards, characterized in that, Includes the following steps: S1. Construct an initial synthetic point cloud dataset through a high-fidelity synthetic point cloud data generation process; use the initial synthetic point cloud dataset to train a memory-enhanced inference network to obtain an initial model. The initial state is constructed based on the prediction results of the initial model and the real labels. It is then input into the policy network of the adversarial environment simulator to output action vectors, generating the first batch of adversarial data. The initial synthetic point cloud dataset and the adversarial data are merged to form the first training pool. S2, Repeat the following steps until the performance of the memory-enhanced inference network converges: The adversarial environment simulator constructs a state based on the current performance of the memory-enhanced inference network and outputs action vectors to generate a new round of adversarial data; The newly generated adversarial data is added to the cumulative training pool to form an updated cumulative training pool, and the memory-enhanced inference network is retrained using the cumulative training pool; The performance of the updated network is evaluated on a fixed validation set reserved from the newly generated data in this round, and the reward is calculated; The policy network of the adversarial environment simulator is updated using the reward. S3 uses a trained memory-enhanced inference network to segment real-world scanned point cloud data; In S2, the state of the adversarial environment simulator is constructed as a multi-dimensional feature map, including three channels: a channel for identifying incorrect prediction points, a channel for identifying low-confidence regions, and a channel for the normalized distance from the prediction point to the class boundary. The action vectors output by the adversarial environment simulator are encoded scene parameters, including the pose adjustment parameters of bridge components, the pose adjustment parameters of occluders, and the position adjustment parameters of the virtual scanner. The action vector is used to adjust the scene parameters and then re-process the high-fidelity synthesized point cloud data to generate a new round of labeled adversarial data. In S2, the calculation of the reward specifically involves: Save a snapshot of the current parameters of the memory-enhanced inference network ; Using newly generated adversarial data Train and update the network; Calculate the network after training and updating on a fixed validation set. Average crossover ratio The average intersection-union ratio of the parameter snapshot on the same validation set The difference ; Calculate the average information entropy of the prediction results of the parameter snapshot on the validation set. ; Calculate the reward value using the formula: ； in, As a reward value; The hyperparameters for balancing the weights; Indicates that the model uses a parameter snapshot For the validation set The predicted probability distribution.

2. The method for intelligent identification and extraction of bridge component point clouds in complex environments of precast beam yards according to claim 1, characterized in that, The generation of the high-fidelity synthetic point cloud data includes: A parameterized bridge component BIM model generation module and scene object library are constructed. Based on the key parameters of the bridge component cross-section, a three-dimensional geometric model with semantic tags is dynamically generated through programmatic scripts. Programmatic scene building and physical simulation are performed to automatically generate a scene model containing bridge components and obstacles. The parameters and spatial position of the virtual laser scanner are set, and a synthetic point cloud is generated through light projection. The synthetic point cloud is processed and samples are generated, noise is injected, and it is saved as a point cloud data file with semantic tags.

3. The method for intelligent identification and extraction of bridge component point clouds in complex environments of precast beam yards according to claim 2, characterized in that, The key parameters of the bridge component cross-section include geometric dimensions, internal structural parameters, and orientation parameters; The programmed script reads parameters through a parameter input interface and automatically completes 3D modeling; The parameters of the virtual laser scanner include horizontal field of view, vertical field of view, angular resolution, and ranging error; the scanner position is defined by three-dimensional spatial coordinates, and point clouds are generated through multi-view collaborative generation.

4. The method for intelligent identification and extraction of bridge component point clouds in complex environments of precast beam yards according to claim 1, characterized in that, The forward propagation and inference of the memory-enhanced inference network includes: inputting point cloud data and processing it through the SetAbstraction layer to obtain semantic features; calculating the similarity weight between each local feature vector extracted from the semantic features and the prototype vector in the external memory matrix; reading information from the memory bank based on the similarity weight to obtain memory-enhanced features; and concatenating the memory-enhanced features with the original features and then upsampling and classifying them to output semantic labels.

5. The intelligent identification and extraction method for point clouds of bridge components in complex environments of precast beam yards according to claim 4, characterized in that, The similarity weights are calculated using dot product attention and the softmax function. Specifically, the dot product of the local feature vector and the transpose of the memory prototype vector is divided by the square root of the feature dimension, and then normalized using the softmax function.

6. The method for intelligent identification and extraction of bridge component point clouds in complex environments of precast beam yards according to claim 1, characterized in that, In S1, the memory-enhanced inference network is trained using a boundary reinforcement joint loss function, which includes a weighted cross-entropy loss, a Lovász-Softmax loss, and a boundary penalty term, with the total loss being a weighted sum of the loss terms. The weights of the weighted cross-entropy loss range from [0.5, 1.5], the weights of the Lovász-Softmax loss range from [1.0, 2.0], and the weights of the boundary penalty term range from [0.1, 0.5].

7. The method for intelligent identification and extraction of bridge component point clouds in complex environments of precast beam yards according to claim 1, characterized in that, The hyperparameters in the weighted sum are initially set to 1.0 and adjusted based on the performance of the validation set: if the generated data is invalid, the hyperparameters are lowered; if the model learns slowly, the hyperparameters are raised.

Images