A method for identifying bridge defects and its unmanned aerial vehicle (UAV) system

Abstract

A method for bridge defect identification and its unmanned aerial vehicle (UAV) system are disclosed. The method includes: ① using an UAV to fly around the bridge and generate a bridge environment map; ② building and training an improved PointNet++ point cloud segmentation model to detect and locate key bridge components such as main beams and piers; ③ using the UAV to approach the surface of the key components and collect detailed images of the bridge; ④ building and training a DeepLabv3+ defect segmentation model to achieve defect identification and location of key bridge structural components. This invention supports complex bridge environment mapping and close-up bridge photography in satellite signal-limited environments, and can be used for automated, hierarchical acquisition of bridge image data and identification of corresponding defects.

Description

Technical Field

[0001] This invention relates to a method for identifying bridge defects and an unmanned aerial vehicle (UAV) system for bridge inspection, specifically to the development of intelligent UAV equipment and methods based on computer vision technology for routine bridge defect inspection tasks, belonging to the field of structural engineering. Background Technology

[0002] As a crucial component of transportation infrastructure, bridge structures are continuously subjected to the dual effects of environmental erosion and load impacts during long-term service. These factors significantly accelerate material degradation, leading to a shortened service life. Cracks, concrete spalling, and voids, among other defects, directly reflect the decline in the internal performance of bridge structures. The appearance of these defects indicates a decrease in the bridge's durability and even its load-bearing capacity, seriously affecting structural safety. According to industry statistics, approximately 40% of the bridges in service in my country's existing highway network have been in service for over 20 years, with 30% exhibiting obvious defects, and over 100,000 bridges defined as dangerous. Therefore, timely and accurate detection of bridge defects is crucial, playing a significant role in ensuring bridge construction quality and operational safety, and preventing and reducing the occurrence of catastrophic accidents.

[0003] Traditional bridge inspection methods rely on manual visual inspection and large-scale bridge inspection equipment, which suffers from high costs, low efficiency, and high risks. Computer vision and drone technology, with their advantages of non-contact and high automation, have become emerging methods for bridge defect detection. However, traditional drones rely on satellite signal information for positioning, but highway bridges are mostly located in remote mountainous areas where drones have difficulty receiving satellite signals and therefore cannot function properly. In addition, drones cannot get close enough to capture high-resolution images of the bridge surface, hindering the identification of minor defects such as cracks. Finally, close-up images taken by drones cannot confirm the actual location of bridge defects in three-dimensional space, which is crucial for subsequent in-depth analysis. Therefore, to meet the daily inspection needs of bridge structures, it is necessary to develop an autonomous positioning drone device and defect identification method suitable for bridge inspection environments. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a method and unmanned aerial vehicle (UAV) system for bridge defect identification, aiming to improve the feasibility of bridge defect identification based on UAV and computer vision technologies in practical applications. The specific contents include: The present invention proposes a method for identifying defects in bridge inspection, comprising the following steps: Step 1: Use an autonomous positioning UAV to fly around the bridge, generate a bridge environment map based on the autonomous positioning module of the UAV, and output the bridge point cloud P1. Step 2: Build and train an improved PointNet++ point cloud segmentation model, and based on the trained improved PointNet++ point cloud segmentation model, process the bridge point cloud P1 in real time at the edge, detect and locate key bridge components such as main beams and piers, and output the segmented bridge point cloud map. Step 3: After detecting key structural components such as main beams and piers, conduct localized detailed inspections of key bridge components. Use autonomous positioning UAV equipment to fly to the corresponding key component locations and get close to the surface of the key components to collect detailed bridge images I1. Step 4: Build and train the DeepLabv3+ defect segmentation model, use the trained DeepLabv3+ defect segmentation model to detect the image data I1, and output the corresponding defect segmentation mask to realize defect identification and localization of key components of the bridge structure.

[0005] Furthermore, the autonomous localization module in step 1 adopts a real-time localization and mapping (SLAM) algorithm that combines inertial measurement unit (IMU) and lidar data, and includes six modules in sequence: data acquisition and synchronization, feature extraction and matching, pose estimation, data fusion, map building, loop closure detection and global optimization. The data acquisition and synchronization module synchronously acquires data from LiDAR and IMU. LiDAR provides a 3D point cloud of the environment, while IMU provides high-frequency acceleration and angular velocity information. The feature extraction module extracts salient features such as corners, edges, or planar features from the 3D point cloud and matches the current features with existing maps or features from previous scans. The pose estimation module uses Iterative Closest Point (ICP) to estimate the current pose of the UAV equipment, achieving real-time positioning. The data fusion module fuses the inertial data from the IMU with the pose estimation from the LiDAR, using an extended Kalman filter to optimize the pose trajectory and reduce accumulated errors and drift. The map building module integrates the currently scanned point cloud into a global map to construct a bridge environment map. The loop closure detection and global optimization module adds a pose constraint when the UAV equipment moves to a previously visited area, indicating that the current pose should coincide with the historical pose, to eliminate pose drift generated during long-term operation and ensure the stability and accuracy of the SLAM system.

[0006] Furthermore, step 2, building and training the improved PointNet++ point cloud segmentation model, specifically includes: Step 2.1: Build an improved PointNet++ point cloud segmentation model, which consists of an input layer, a feature extraction module, a segmentation module, and an output layer; The input layer is used to input point cloud data; the feature extraction module uses two setabstraction (SA) structures as its basic structure. The SA structure selects a certain number of key points by sampling the farthest point, and then selects points within a spherical region of a certain radius around each key point as a set. Features are then extracted from the points in this set to obtain point cloud feature information across multiple layers; the segmentation module upsamples the feature information back to the original point cloud size by sequentially connecting two interpolation operations and two multilayer perceptron layers; the output layer is used to predict and output the classification probability of each point. Step 2.2: To improve point cloud segmentation accuracy, a Varifocal impairment function is introduced into the improved PointNet++ point cloud segmentation model to replace the original cross-entropy loss function. The formula is as follows:

[0007] in, The IoU-based perceptual classification score predicted by the model. The Interchange Value (IOU) between the predicted bounding box and the ground truth. and The representative coefficients are 0.75 and 2, respectively. Step 2.3: Collect a large amount of bridge point cloud data, and segment and label the collected bridge point clouds. The labeling categories include piers, main beams, bridge deck systems, and retaining walls. Introduce shape-aware point cloud data augmentation technology to enhance the bridge point clouds and their labeling information. The formula is as follows:

[0008] in S , T This represents two different bridge point cloud data. S i , T j Representing point clouds respectively S and T A randomly selected subset, where FPS represents the farthest point sampling method, can be represented by its single iteration process as follows:

[0009] in p t This represents the point selected in this iteration. Represents the calculation point x Time s The minimum distance, x It belongs to the original point cloud S , s Belongs to the current selected point cloud set K tBy using shape-aware point cloud data augmentation technology, we simulate the problems of missing and sparse points in real-world scenarios and alleviate the inter-class imbalance problem in the data, thereby forming a point cloud segmentation dataset for improving the training of the PointNet++ point cloud segmentation model. Step 2.4: Randomly divide the point cloud segmentation dataset into a training set and a test set. The training dataset is used to train the improved PointNet++ point cloud segmentation model, and the test dataset is used to evaluate the performance of the trained model; finally, the trained improved PointNet++ point cloud segmentation model is obtained.

[0010] Furthermore, step 4, building and training the DeepLabv3+ disease segmentation model, specifically includes: Step 4.1: Build and train the DeepLabv3+ disease segmentation model; Step 4.2: Collect image data of bridge structural defects and use Labelme software to annotate the collected images. The annotation objects are the pixels of bridge defects in the images, including corrosion, cracks, voids, dampness, and peeling. Augment the existing image data by randomly cutting a block of region from the image, rotating and color-dithering it, and then randomly pasting it onto a region of the original image to generate a new defect image and corresponding annotation. The original image and the generated image are merged into a dataset for the DeepLabv3+ defect segmentation model. Step 4.3: Split the disease dataset into a training set and a test set. The training dataset is used to train the DeepLabv3+ disease segmentation model, and the test dataset is used to evaluate the model performance. Step 4.4: Use the trained DeepLabv3+ defect segmentation model to detect the image data I1, output the corresponding defect segmentation mask, and realize the defect identification and location of key components of the bridge structure.

[0011] The second aspect of this invention relates to a bridge defect identification unmanned aerial vehicle (UAV) system, comprising a UAV and a ground station for controlling and receiving UAV data. The ground station includes a remote controller, a data transmission module, and a computer. The UAV system is controlled by the remote controller, which sends commands via wireless signals to control the UAV's takeoff, landing, and normal flight. The data transmission module receives visual data acquired from a monocular camera and a lidar, and outputs it to the computer via analog video signals. The computer acts as a receiver and processor for real-time point cloud and image data, enabling real-time visualization of the bridge point cloud data and further processing of the image data to identify bridge defects. The UAV is equipped with a lidar, a high-resolution monocular camera, and a three-axis camera gimbal to acquire high-definition images of the bridge. The characteristic feature is that the central control unit of the UAV is equipped with a working system, which includes: The UAV autonomous positioning module is used to generate a bridge environment map and output the bridge point cloud P1. The point cloud module, based on the trained improved PointNet++ point cloud segmentation model, processes the bridge point cloud P1 in real time at the edge, detects and locates key bridge components such as main beams and piers, and outputs the segmented bridge point cloud map. The image acquisition module is used to acquire high-definition images of the bridge. The edge computing module uses a trained DeepLabv3+ defect segmentation model to detect the collected bridge image data and outputs the corresponding defect segmentation mask to achieve defect identification and localization of key bridge structural components.

[0012] Compared with existing methods, the present invention has the following advantages: (1) The autonomous positioning UAV equipment for bridge inspection described above can achieve autonomous positioning and navigation in the environment of satellite signal rejection, avoiding the problem that the UAV cannot receive satellite information due to obstruction at the bottom of the bridge and stops working. (2) The intelligent identification method for bridge defects described above can achieve efficient and accurate identification of key bridge components at the edge, thereby completing two modes: preliminary overall inspection and detailed local inspection of the bridge. (3) The intelligent identification method for bridge defects is described above. Through two models, bridge component identification and defect identification, it accurately identifies and locates the defects of key bridge components, providing support for bridge management and maintenance. Attached Figure Description

[0013] Figure 1 This is a flowchart of a bridge defect identification method provided by an embodiment of the present invention; Figure 2 This is a structural diagram of the improved PointNet++ point cloud segmentation algorithm of the present invention; Figure 3 This is a schematic diagram of point cloud data enhancement according to an embodiment of the present invention; Figure 4 This is a schematic diagram of bridge point cloud segmentation according to an embodiment of the present invention; Figure 5 This is a schematic diagram of bridge defect identification and location according to an embodiment of the present invention; Figure 6 This is a schematic diagram of an autonomous positioning UAV equipment for bridge inspection provided in an embodiment of the present invention; Detailed Implementation

[0014] The following detailed description, with reference to the accompanying drawings, provides an autonomous positioning unmanned aerial vehicle (UAV) device and defect identification method for bridge inspection according to the present invention.

[0015] Example 1 An embodiment of the present invention provides a method for identifying defects in bridge inspection, the process of which is as follows: Figure 1 As shown, the specific steps include: Step 1: Use an autonomous positioning UAV to fly around the bridge. Generate a bridge environment map based on the autonomous positioning module of the UAV. Employ a Simultaneous Localization and Mapping (SLAM) algorithm that combines inertial measurement unit (IMU) and lidar data. The algorithm includes six modules in sequence: data acquisition and synchronization, feature extraction and matching, pose estimation, data fusion, map building, loop closure detection and global optimization. Finally, output the bridge point cloud P1. Step 2: Build and train an improved PointNet++ point cloud segmentation model for bridge point cloud P1; Step 2.1: Build an improved PointNet++ point cloud segmentation model, such as... Figure 2 As shown, the model consists of an input layer, a feature extraction module, a segmentation module, and an output layer; The input layer is used to input point cloud data; the feature extraction module uses two setabstraction (SA) structures as its basic structure. The SA structure selects a certain number of key points by sampling the farthest point, and then selects points within a spherical region of a certain radius around each key point as a set. Features are then extracted from the points in this set to obtain point cloud feature information across multiple layers; the segmentation module upsamples the feature information back to the original point cloud size by sequentially connecting two interpolation operations and two multilayer perceptron layers; the output layer is used to predict and output the classification probability of each point. Step 2.2: To improve point cloud segmentation accuracy, a Varifocal impairment function is introduced into the improved PointNet++ point cloud segmentation model to replace the original cross-entropy loss function. The formula is as follows:

[0016] in, The IoU-based perceptual classification score predicted by the model. The Interchange Value (IOU) between the predicted bounding box and the ground truth. and The representative coefficients are 0.75 and 2, respectively. Step 2.3: Collect a large amount of bridge point cloud data, and segment and label the collected bridge point clouds. The labeling categories include piers, main beams, bridge deck systems, and retaining walls; for example... Figure 3 As shown, a shape-aware point cloud data augmentation technique is introduced to enhance the bridge point cloud and its annotation information. The formula is as follows:

[0017] in S , T This represents two different bridge point cloud data. S i , T j Representing point clouds respectively S and T A randomly selected subset, where FPS represents the farthest point sampling method, can be represented by its single iteration process as follows:

[0018] in p t This represents the point selected in this iteration. Represents the calculation point x Time s The minimum distance, x It belongs to the original point cloud S , s Belongs to the current selected point cloud set K t By using shape-aware point cloud data augmentation technology, we simulate the problems of missing and sparse points in real-world scenarios and alleviate the inter-class imbalance problem in the data, thereby forming a point cloud segmentation dataset for improving the training of the PointNet++ point cloud segmentation model. Step 2.4: Randomly divide the point cloud segmentation dataset into a training set and a test set. The training dataset is used to train the improved PointNet++ point cloud segmentation model, and the test dataset is used to evaluate the performance of the trained model, as shown in Table 1; thus, the trained improved PointNet++ point cloud segmentation model is obtained. Table 1. Point cloud segmentation accuracy for bridges

[0019] Step 2.5: Based on the trained improved PointNet++ point cloud segmentation model, process the bridge point cloud P1 in real time at the edge, detect and locate key bridge components such as main beams and piers, and output the segmented bridge point cloud map, such as... Figure 4 As shown; Step 3: After detecting key structural components such as main beams and piers, conduct localized detailed inspections of key bridge components. Use autonomous positioning UAV equipment to fly to the corresponding key component locations and get close to the surface of the key components to collect detailed bridge images I1. Step 4: Build and train the DeepLabv3+ defect segmentation model to achieve defect identification and localization of key components of the bridge structure.

[0020] Step 4.1: Build and train the DeepLabv3+ disease segmentation model; Step 4.2: Collect image data of bridge structural defects and use Labelme software to annotate the collected images. The annotation objects are the pixels of bridge defects in the images, including corrosion, cracks, voids, dampness, and peeling. Augment the existing image data by randomly cutting a block of region from the image, rotating and color-dithering it, and then randomly pasting it onto a region of the original image to generate a new defect image and corresponding annotation. The original image and the generated image are merged into a dataset for the DeepLabv3+ defect segmentation model. Step 4.3: Split the disease dataset into a training set and a test set. The training dataset is used to train the DeepLabv3+ disease segmentation model, and the test dataset is used to evaluate the model performance. Step 4.4: Use the trained DeepLabv3+ defect segmentation model to detect the image data I1, and output the corresponding defect segmentation mask to achieve defect identification and localization of key components of the bridge structure, such as... Figure 5 As shown.

[0021] Example 2 like Figure 6 This embodiment relates to a bridge defect identification drone system, including a drone and a ground station for controlling and receiving drone data. The ground station includes a remote controller, a data transmission module, and a computer. The remote controller controls the drone system by sending commands via wireless signals to control the drone's takeoff, landing, and normal flight. The data transmission module receives visual data collected from a monocular camera and LiDAR, and outputs it to the computer via analog video signals. The computer acts as a receiver and processor for real-time point cloud and image data, enabling real-time visualization of the bridge point cloud data and further processing of the image data to identify bridge defects. The drone is equipped with a LiDAR, a high-resolution monocular camera, and a three-axis camera gimbal to acquire high-definition images of the bridge. The central control unit of the aforementioned drone is equipped with a working system, which includes: The UAV autonomous positioning module is used to generate a bridge environment map and output the bridge point cloud P1. The point cloud module, based on the trained improved PointNet++ point cloud segmentation model, processes the bridge point cloud P1 in real time at the edge, detects and locates key bridge components such as main beams and piers, and outputs the segmented bridge point cloud map. The image acquisition module is used to acquire high-definition images of the bridge. The edge computing module uses a trained DeepLabv3+ defect segmentation model to detect the collected bridge image data and outputs the corresponding defect segmentation mask to achieve defect identification and localization of key bridge structural components.

[0022] The improved PointNet++ point cloud segmentation model consists of an input layer, a feature extraction module, a segmentation module, and an output layer. The input layer is used to input point cloud data; the feature extraction module uses two ISAs as its basic structure. The ISA structure selects a certain number of key points by sampling the farthest point, and then selects points within a spherical region of a certain radius around each key point as a set. Then, feature extraction is performed on the points in this set to obtain point cloud feature information between multiple layers; the segmentation module upsamples the feature information back to the original point cloud size by sequentially connecting two interpolation operations and two multilayer perceptron layers; the output layer is used to predict and output the classification probability of each point. A variable loss function is introduced to replace the original cross-entropy loss function, and its formula is as follows:

[0023] in, The IoU-based perceptual classification score predicted by the model. The Interchange Value (IOU) between the predicted bounding box and the ground truth. and The representative coefficients are 0.75 and 2, respectively.

[0024] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for identifying defects in bridges, characterized in that, Includes the following steps: Step 1: Use an autonomous positioning UAV to fly around the bridge, generate a bridge environment map based on the autonomous positioning module of the UAV, and output the bridge point cloud P1. Step 2: Build and train an improved PointNet++ point cloud segmentation model, and based on the trained improved PointNet++ point cloud segmentation model, process the bridge point cloud P1 in real time at the edge, detect and locate key bridge components such as the main beam and piers, and output the segmented bridge point cloud map. The construction and training of the improved PointNet++ point cloud segmentation model specifically includes: Step 2.1: Build an improved PointNet++ point cloud segmentation model, which consists of an input layer, a feature extraction module, a segmentation module, and an output layer; The input layer is used to input point cloud data; the feature extraction module uses two setabstraction (SA) structures as its basic structure. The SA structure selects a certain number of key points by sampling the farthest point, and then selects points within a spherical region of a certain radius around each key point as a set. Features are then extracted from the points in this set to obtain point cloud feature information across multiple layers; the segmentation module upsamples the feature information back to the original point cloud size by sequentially connecting two interpolation operations and two multilayer perceptron layers; the output layer is used to predict and output the classification probability of each point. Step 2.2: To improve point cloud segmentation accuracy, a Varifocal impairment function is introduced into the improved PointNet++ point cloud segmentation model to replace the original cross-entropy loss function. The formula is as follows: in, The IoU-based perceptual classification score predicted by the model. The Interchange Value (IOU) between the predicted bounding box and the ground truth. and The representative coefficients are 0.75 and 2, respectively. Step 2.3: Collect a large amount of bridge point cloud data, and segment and label the collected bridge point clouds. The labeling categories include piers, main beams, bridge deck systems, and retaining walls. Introduce shape-aware point cloud data augmentation technology to enhance the bridge point clouds and their labeling information. The formula is as follows: Where S and T represent two different bridge point cloud data, S i T j Representing a randomly selected subset of point clouds S and T respectively, FPS represents the farthest point sampling method, and its single iteration process is expressed as follows: Where p t This represents the point selected in this iteration. This represents the minimum distance from point x to point s, where x belongs to the original point cloud S and s belongs to the currently selected point cloud set K. t By using shape-aware point cloud data augmentation technology, we simulate the missing and sparsity problems of point clouds in real-world scenarios and alleviate the inter-class imbalance problem in the data, thereby forming a point cloud segmentation dataset for improving the training of the PointNet++ point cloud segmentation model. Step 2.4: Randomly divide the point cloud segmentation dataset into a training set and a test set. The training dataset is used to train the improved PointNet++ point cloud segmentation model, and the test dataset is used to evaluate the performance of the trained model; finally, the trained improved PointNet++ point cloud segmentation model is obtained. Step 3: After detecting the key structural components of the main beam and piers, carry out local fine-grained inspection of the key components of the bridge. Use an autonomous positioning UAV to fly to the corresponding key component location and get close to the surface of the key component to collect detailed images of the bridge I1. Step 4: Build and train the DeepLabv3+ defect segmentation model, use the trained DeepLabv3+ defect segmentation model to detect the image data I1, and output the corresponding defect segmentation mask to realize defect identification and localization of key components of the bridge structure.

2. The method for identifying defects in bridge inspection as described in claim 1, characterized in that, The autonomous localization module in step 1 adopts a real-time localization and mapping (SLAM) algorithm that combines inertial measurement unit (IMU) and lidar data, and includes six modules in sequence: data acquisition and synchronization, feature extraction and matching, pose estimation, data fusion, map building, loop closure detection and global optimization. The data acquisition and synchronization module synchronously acquires data from LiDAR and IMU. LiDAR provides a 3D point cloud of the environment, while IMU provides high-frequency acceleration and angular velocity information. The feature extraction module extracts salient features such as corners, edges, or planar features from the 3D point cloud and matches the current features with existing maps or features from previous scans. The pose estimation module uses Iterative Closest Point (ICP) to estimate the current pose of the UAV equipment, achieving real-time positioning. The data fusion module fuses the inertial data from the IMU with the pose estimation from the LiDAR, using an extended Kalman filter to optimize the pose trajectory and reduce accumulated errors and drift. The map building module integrates the currently scanned point cloud into a global map to construct a bridge environment map. The loop closure detection and global optimization module adds a pose constraint when the UAV equipment moves to a previously visited area, indicating that the current pose should coincide with the historical pose, to eliminate pose drift generated during long-term operation and ensure the stability and accuracy of the SLAM system.

3. The method for identifying defects in bridge inspection as described in claim 1, characterized in that, Step 4, which involves building and training the DeepLabv3+ disease segmentation model, specifically includes: Step 4.1: Build and train the DeepLabv3+ disease segmentation model; Step 4.2: Collect image data of bridge structural defects and use Labelme software to annotate the collected images. The annotation objects are the pixels of bridge defects in the images, including corrosion, cracks, voids, dampness, and peeling defects. Augment the existing image data by randomly cutting a block region from the image, rotating and color-dithering it, and then randomly pasting it onto a region of the original image to generate a new defect image and corresponding annotation. The original image and the generated image are merged into the DeepLabv3+ defect segmentation model dataset. Step 4.3: Split the disease dataset into a training set and a test set. The training dataset is used to train the DeepLabv3+ disease segmentation model, and the test dataset is used to evaluate the model performance. Step 4.4: Use the trained DeepLabv3+ defect segmentation model to detect the image data I1, output the corresponding defect segmentation mask, and realize the defect identification and location of key components of the bridge structure.

4. A bridge defect identification drone system, comprising a drone and a ground station for controlling and receiving drone data. The ground station includes a remote controller, a data transmission module, and a computer. The remote controller controls the drone system, sending commands via wireless signals to control takeoff, landing, and normal flight. The data transmission module receives visual data acquired from a monocular camera and LiDAR, and outputs it to the computer via analog video signals. The computer acts as a receiver and processor for real-time point cloud and image data, enabling real-time visualization of bridge point cloud data and further processing of image data to identify bridge defects. The drone is equipped with LiDAR, a high-resolution monocular camera, and a three-axis camera gimbal to acquire high-definition images of the bridge. Its features are, The central control unit of the aforementioned drone is equipped with a working system, which includes: The UAV autonomous positioning module is used to generate a bridge environment map and output the bridge point cloud P1. The point cloud module, based on a trained improved PointNet++ point cloud segmentation model, processes the bridge point cloud P1 in real time at the edge, detects and locates key bridge components such as main beams and piers, and outputs a segmented bridge point cloud map. The image acquisition module is used to acquire high-definition images of the bridge. The edge computing module uses a trained DeepLabv3+ defect segmentation model to detect the collected bridge image data and outputs the corresponding defect segmentation mask to achieve defect identification and localization of key bridge structural components.

5. The bridge defect identification UAV system as described in claim 4, characterized in that, The improved PointNet++ point cloud segmentation model consists of an input layer, a feature extraction module, a segmentation module, and an output layer. The input layer is used to input point cloud data; the feature extraction module uses two ISAs as its basic structure. The ISA structure selects a certain number of key points by sampling the farthest point, and then selects points within a spherical region of a certain radius around each key point as a set. Then, feature extraction is performed on the points in this set to obtain point cloud feature information between multiple layers; the segmentation module upsamples the feature information back to the original point cloud size by sequentially connecting two interpolation operations and two multilayer perceptron layers; the output layer is used to predict and output the classification probability of each point. A variable loss function is introduced to replace the original cross-entropy loss function, and its formula is as follows: in, The IoU-based perceptual classification score predicted by the model. The Interchange Value (IOU) between the predicted bounding box and the ground truth. and The representative coefficients are 0.75 and 2, respectively.

Images