Lidar-based train forward obstacle intrusion detection method and system

Abstract

Disclosed are a LiDAR-based train forward obstacle intrusion detection method and system, relating to the technical field of obstacle detection during train operation. The method comprises: acquiring real-time point cloud data of a LiDAR; using a pre-trained forward obstacle detection model to process the real-time point cloud data inputted by the LiDAR, and detecting obstacles in the forward direction of train operation to obtain three-dimensional bounding boxes and point clouds of the obstacles; and using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network to construct a train forward obstacle detection algorithm, and inputting a training dataset for training, so as to obtain a forward obstacle detection model. The detection method and the system consider spatial distribution patterns of point clouds in train forward operating environments, avoid repeated extraction of useless features, and increase the receptive field size without reducing the output feature mapping size, thereby obtaining high-resolution semantic features; and information interaction between high-level features and low-level features in different dimensions is achieved, thus enhancing positional semantic information of the features.

Description

A method and system for detecting forward obstacle encroachment on trains using lidar Technical Field

[0001] This invention relates to the field of train operation obstacle detection technology, specifically to a method and system for detecting forward obstacle intrusions into train limits using lidar based on domain knowledge verification. Background Technology

[0002] With the continuous advancement of intelligent and information-based digital construction in rail transit, environmental perception technology has become one of the main development directions of modern rail transit trains. Obstacle detection, as a key function of train environmental perception, can help trains detect obstacles on the track in a timely manner, ensuring the safe operation of trains.

[0003] Currently, obstacle detection technologies for rail transit trains can be categorized based on the sensors used, including those based on industrial cameras, millimeter-wave radar, frequency-modulated continuous-wave radar, and lidar. Industrial camera-based methods use sensors such as color or infrared cameras to acquire high-resolution images of the area in front of the train, offering high obstacle detection accuracy. However, this method is limited by two-dimensional images and struggles to provide three-dimensional information such as obstacle distance. Millimeter-wave radar-based methods use 3D or 4D millimeter-wave radar to acquire three-dimensional point cloud information of the area in front of the train. Its emitted radio waves can penetrate rain and fog, demonstrating good performance in adverse weather conditions. However, the point cloud density in this method is too sparse, making it difficult to detect obstacles at medium to long distances. Frequency-modulated continuous-wave radar-based methods use continuous-wave radar with a transmission frequency modulated by a specific signal to acquire grayscale image information of the area in Cartesian coordinates in front of the train. This method offers long detection range and strong resistance to adverse weather conditions, but the acquired grayscale image resolution is low, making it difficult to detect obstacles. LiDAR-based methods use solid-state or semi-solid-state LiDAR to acquire high-resolution 3D point cloud information in front of the train. These methods have high point cloud density, long detection range, and are not affected by lighting conditions, showing great application potential in the field of obstacle detection for rail transit trains. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for detecting forward obstacles to trains using lidar based on domain knowledge verification, so as to solve the technical problems in the background art mentioned above.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a method for detecting forward obstacle intrusions into a train using lidar, comprising:

[0007] Acquire real-time point cloud data from LiDAR;

[0008] A pre-trained forward obstacle detection model is used to process real-time point cloud data input from LiDAR to detect obstacles in the direction of train movement, obtaining 3D bounding boxes and point clouds of the obstacles. Training the forward obstacle detection model includes: scanning the train's forward movement scene with LiDAR to acquire 3D point cloud data under different scenes, labeling the obstacle point clouds, and constructing a training dataset; constructing a train forward obstacle detection algorithm using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network, inputting the algorithm into the training dataset for training, and obtaining the forward obstacle detection model.

[0009] Furthermore, based on the L-shaped sparse residual convolution module, an L-shaped backbone feature extraction network is constructed, including: considering the elongated and narrow spatial distribution of point clouds in the railway operating environment, the L-shaped sparse residual convolution module consists of two 3D submanifold sparse convolution operations, and its calculation formula is as follows: LB = R(SMC) i,j,k＝5,3,1 / s＝1 B(SMC i,j,k＝1,3,5 / s＝1 BR(Input))+Input)

[0010] Where Input represents the input features, and SMC i,j,k / s SMC represents the sparse convolution operation on 3D submanifolds. i,j,k / s B represents the SMC-BN sequential structure, SMC i,j,k / s BR represents the SMC-BN-ReLU sequential operation, i,j,k represent the kernel size of the 3D submanifold sparse convolution operation in the x, y, and z directions, s represents the stride of the convolution kernel, and R represents the ReLU activation function.

[0011] Furthermore, an L-shaped backbone feature extraction network is constructed using an L-shaped sparse residual convolution module, the specific structure of which is as follows:

[0012] Where LB represents an L-shaped sparse residual convolution module, and SC i,j,k / s This represents a 3D regular sparse convolution operation, where i, j, k represent the kernel size in the x, y, and z directions, s represents the kernel stride, n represents the number of layers, and L0 represents the original input features.

[0013] Furthermore, based on the cross-dimensional encoding module, a cross-dimensional autoencoding network was constructed, comprising: the cross-dimensional encoding module consists of three parts: three-dimensional feature compression, two-dimensional feature upsampling, and fusion convolution;

[0014] 3D feature compression accumulates the sparse 3D features input to the backbone network along the z-axis, mapping the sparse 3D features to sparse BEV 2D features while preserving sparsity, as follows:

[0015] Where SBEV represents sparse bird's-eye view features, Input 3D Represents the sparse three-dimensional features of the input;

[0016] Two-dimensional feature upsampling involves upsampling the sparse BEV two-dimensional features, represented as:

[0017] in, Let x represent the upsampled two-dimensional feature. p The x-coordinate and y-coordinate of the original two-dimensional feature are represented by... p y represents the original two-dimensional feature's y-coordinate, and p represents the relevant index of the two-dimensional feature.

[0018] The fusion convolution first concatenates and fuses the sparse BEV 2D features and the upsampled 2D features, and then uses a 3D standard residual sparse convolution module to remove noise, which can be formally expressed as:

[0019] Among them, Output fused SMC represents the features after fusion. i,j / s denoted as 2D submanifold sparse convolution operation, i,j represent the kernel size of the 2D submanifold sparse convolution operation in the x and y directions, s represents the stride of the convolution kernel, and Contact represents concatenation.

[0020] Furthermore, an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network are used to construct a train forward obstacle detection algorithm. The training dataset is input to obtain a forward obstacle detection model, including:

[0021] Dynamically voxelize the input 3D point cloud;

[0022] The voxelized point cloud is input into an L-shaped backbone feature extraction network for feature extraction, resulting in features L1-L6.

[0023] After the L-shaped backbone feature extraction network, a cross-dimensional autoencoder network is connected to obtain features P4, P5 and P6;

[0024] The feature P4 is input into the sparse center prediction head to obtain the three-dimensional center coordinates, length, width, height and angle of each obstacle. The sparse center prediction head uses five prediction heads consisting of a 2D submanifold sparse convolution operation with a 3×3 convolution kernel size and a 2D submanifold sparse convolution operation with a 1×1 convolution kernel size to predict the (x, y) coordinates, z coordinates, (l, w, h) length, width, height, angle and heatmap of the obstacle.

[0025] Furthermore, the sparse center head representation is: P center =5×SMC i,j＝3,3 / s＝1 (SMC i,j＝1,1 / s＝1(P4)) ={(x pred ,y pred ),z pred ,(l pred ,w pred ,h pred ), angle pred heatmap pred}

[0026] Among them, SMC i,j / s Let represent a 2D submanifold sparse convolution operation, where i,j represent the kernel size of the 2D submanifold sparse convolution operation in the x and y directions, and s represent the stride of the kernel.

[0027] Secondly, the present invention provides a train forward obstacle intrusion detection system using lidar, comprising:

[0028] The acquisition module is used to acquire real-time point cloud data from the LiDAR.

[0029] The detection module is used to process real-time point cloud data input from the LiDAR using a pre-trained forward obstacle detection model, detect obstacles in the direction of train movement, and obtain the 3D bounding boxes and point clouds of the obstacles. Training the forward obstacle detection model includes: scanning the train's forward movement scene using LiDAR to obtain 3D point cloud data under different scenes, labeling the obstacle point clouds, and constructing a training dataset; constructing a train forward obstacle detection algorithm using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network, inputting it into the training dataset for training, and obtaining the forward obstacle detection model.

[0030] Thirdly, the present invention provides a non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the train forward obstacle intrusion detection method using lidar as described in the first aspect.

[0031] Fourthly, the present invention provides a computer device including a memory and a processor, the processor and the memory communicating with each other, the memory storing program instructions executable by the processor, and the processor calling the program instructions to execute the train forward obstacle intrusion detection method using lidar as described in the first aspect.

[0032] Fifthly, the present invention provides an electronic device comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the train forward obstacle intrusion detection method using lidar as described in the first aspect.

[0033] The beneficial effects of this invention are as follows: The proposed L-shaped residual sparse convolutional module fully considers the spatial distribution of point clouds in the train's forward operating environment, avoiding the repeated extraction of useless features. By focusing features on the x and z axes, the L-shaped backbone feature extraction network increases the receptive field size without sacrificing the output feature mapping size, thus obtaining high-resolution semantic features. Furthermore, the cross-dimensional autoencoder network compresses features from three-dimensional space to two-dimensional space and fuses and extracts features at different scales, achieving information interaction between high-level and shallow features of different dimensions. This enhances the positional semantic information of the features and further improves the algorithm's performance. By fusing the L-shaped backbone feature extraction network and the cross-dimensional autoencoder network, a robust train forward obstacle detection algorithm is constructed, achieving high-precision three-dimensional detection of obstacles in front of the train.

[0034] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 is a flowchart of the train forward obstacle intrusion detection method using lidar according to an embodiment of the present invention.

[0037] Figure 2 is a structural diagram of the L-shaped residual sparse convolution module according to an embodiment of the present invention.

[0038] Figure 3 is a structural diagram of the cross-dimensional encoding module according to an embodiment of the present invention.

[0039] Figure 4 is a structural diagram of the cross-dimensional autoencoder network described in an embodiment of the present invention.

[0040] Figure 5 is a structural diagram of the train forward obstacle detection algorithm according to an embodiment of the present invention.

[0041] Figure 6 is a comparison of obstacle detection between existing algorithms and the algorithm described in this invention. Detailed Implementation

[0042] Example 1

[0043] In this embodiment 1, a train forward obstacle intrusion detection system using LiDAR is first provided, including: an acquisition module for acquiring real-time point cloud data from LiDAR; and a detection module for processing the real-time point cloud data input from LiDAR using a pre-trained forward obstacle detection model to detect obstacles in the direction of train movement and obtain the 3D bounding boxes and point clouds of the obstacles. The training of the forward obstacle detection model includes: scanning the train's forward movement scene using LiDAR to acquire 3D point cloud data under different scenes, labeling the obstacle point clouds, and constructing a training dataset; constructing a train forward obstacle detection algorithm using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network, inputting the training dataset for training, and obtaining the forward obstacle detection model.

[0044] In this embodiment, the above-described system is used to implement a train forward obstacle intrusion detection method using LiDAR, including: acquiring real-time point cloud data from LiDAR; processing the real-time point cloud data input from LiDAR using a pre-trained forward obstacle detection model to detect obstacles in the direction of train movement, and obtaining the 3D bounding boxes and point clouds of the obstacles; wherein, training the forward obstacle detection model includes: scanning the train's forward movement scene using LiDAR to acquire 3D point cloud data under different scenes, labeling the obstacle point clouds, and constructing a training dataset; constructing a train forward obstacle detection algorithm using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network, inputting the training dataset for training, and obtaining the forward obstacle detection model.

[0045] Based on an L-shaped sparse residual convolution module, an L-shaped backbone feature extraction network is constructed. This includes: considering the elongated and narrow spatial distribution of point clouds in railway operating environments, the L-shaped sparse residual convolution module consists of two 3D submanifold sparse convolution operations, calculated as follows: LB = R(SMC) i,j,k＝5,3,1 / s＝1 B(SMC i,j,k＝1,3,5 / s＝1 BR(Input))+Input)

[0046] Where Input represents the input features, and SMC i,j,k / s SMC represents the sparse convolution operation on 3D submanifolds. i,j,k / s B represents the SMC-BN sequential structure, SMC i,j,k / s BR represents the SMC-BN-ReLU sequential operation, i,j,k represent the kernel size of the 3D submanifold sparse convolution operation in the x, y, and z directions, s represents the stride of the convolution kernel, and R represents the ReLU activation function.

[0047] An L-shaped backbone feature extraction network is constructed using an L-shaped sparse residual convolution module, and its specific structure is as follows:

[0048] Where LB represents an L-shaped sparse residual convolution module, and SC i,j,k / s This represents a 3D regular sparse convolution operation, where i, j, k represent the kernel size in the x, y, and z directions, s represents the kernel stride, n represents the number of layers, and L0 represents the original input features.

[0049] A cross-dimensional automatic coding network was constructed based on the cross-dimensional coding module, which consists of three parts: three-dimensional feature compression, two-dimensional feature upsampling, and fusion convolution.

[0050] 3D feature compression accumulates the sparse 3D features input to the backbone network along the z-axis, mapping the sparse 3D features to sparse BEV 2D features while preserving sparsity, as follows:

[0051] Where SBEV represents sparse bird's-eye view features, Input 3D Represents the sparse three-dimensional features of the input;

[0052] Two-dimensional feature upsampling involves upsampling the sparse BEV two-dimensional features, represented as:

[0053] in, Let x represent the upsampled two-dimensional feature. p The x-coordinate and y-coordinate of the original two-dimensional feature are represented by... p y represents the original two-dimensional feature's y-coordinate, and p represents the relevant index of the two-dimensional feature.

[0054] The fusion convolution first concatenates and fuses the sparse BEV 2D features and the upsampled 2D features, and then uses a 3D standard residual sparse convolution module to remove noise, which can be formally expressed as:

[0055] Among them, Output fused SMC represents the features after fusion. i,j / s denoted as 2D submanifold sparse convolution operation, i,j represent the kernel size of the 2D submanifold sparse convolution operation in the x and y directions, s represents the stride of the convolution kernel, and Contact represents concatenation.

[0056] A train forward obstacle detection algorithm is constructed using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network. The training dataset is input to obtain the forward obstacle detection model, which includes:

[0057] Dynamically voxelize the input 3D point cloud;

[0058] The voxelized point cloud is input into an L-shaped backbone feature extraction network for feature extraction, resulting in features L1-L6.

[0059] After the L-shaped backbone feature extraction network, a cross-dimensional autoencoder network is connected to obtain features P4, P5 and P6;

[0060] The feature P4 is input into the sparse center prediction head to obtain the three-dimensional center coordinates, length, width, height and angle of each obstacle. The sparse center prediction head uses five prediction heads consisting of a 2D submanifold sparse convolution operation with a 3×3 convolution kernel size and a 2D submanifold sparse convolution operation with a 1×1 convolution kernel size to predict the (x, y) coordinates, z coordinates, (l, w, h) length, width, height, angle and heatmap of the obstacle.

[0061] The sparse center head representation is: P center =5×SMC i,j＝3,3 / s＝1 (SMC i,j＝1,1 / s＝1 (P4)) ={(x pred ,y pred ),z pred ,(l pred ,w pred ,h pred ), angle pred heatmap pred}

[0062] Among them, SMC i,j / s Let represent a 2D submanifold sparse convolution operation, where i,j represent the kernel size of the 2D submanifold sparse convolution operation in the x and y directions, and s represent the stride of the kernel.

[0063] Example 2

[0064] This embodiment 2 provides a method for detecting obstacles ahead of a train using lidar. By employing an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network to optimize the feature extraction and fusion process, high-precision 3D measurement of obstacles ahead of the train is achieved, and the detection accuracy of obstacles at long distances is improved. The method includes the following steps:

[0065] Step 1: Use LiDAR to scan the train's forward running scene, obtain 3D point cloud data under different scenes, label obstacle point clouds, and build a training dataset.

[0066] Step 2: An L-shaped sparse residual convolution module was designed, and an L-shaped backbone feature extraction network was constructed.

[0067] Step 2 specifically includes the following steps:

[0068] Step 2.1: Considering the elongated and narrow spatial distribution of point clouds in railway operating environments, an L-shaped sparse residual convolution module is designed. This module consists of two 3D submanifold sparse convolution operations, and its calculation formula is as follows: LB = R(SMC)i,j,k＝5,3,1 / s＝1 B(SMC i,j,k＝1,3,5 / s＝1 BR(Input))+Input) (1)

[0069] Where Input represents the input features, and SMC i,j,k / s SMC represents the sparse convolution operation on 3D submanifolds. i,j,k / s B represents the SMC-BN sequential structure, SMC i,j,k / s BR represents the SMC-BN-ReLU sequential operation, i,j,k represent the kernel size of the 3D submanifold sparse convolution operation in the x, y, and z directions, s represents the stride of the convolution kernel, and R represents the ReLU activation function.

[0070] Step 2.2: Construct an L-shaped backbone feature extraction network using an L-shaped sparse residual convolution module. Its specific structure is as follows:

[0071] Where LB represents an L-shaped sparse residual convolution module, and SC i,j,k / s This represents a 3D regular sparse convolution operation, where i, j, k represent the kernel size in the x, y, and z directions, s represents the kernel stride, n represents the number of layers, and L0 represents the original input features.

[0072] Step 3 involves designing a cross-dimensional encoding module and constructing a cross-dimensional autoencoding network; specifically, it includes the following steps:

[0073] The cross-dimensional encoding module consists of three parts: three-dimensional feature compression, two-dimensional feature upsampling, and fusion convolution.

[0074] 3D feature compression accumulates the sparse 3D features input to the backbone network along the z-axis, mapping the sparse 3D features to sparse BEV 2D features while preserving sparsity. This process can be represented as:

[0075] Where SBEV represents sparse bird's-eye view features, Input 3D This represents the sparse three-dimensional features of the input.

[0076] Two-dimensional feature upsampling involves upsampling sparse BEV two-dimensional features, and the process can be represented as follows:

[0077] in, Let x represent the upsampled two-dimensional feature. p The x-coordinate and y-coordinate of the original two-dimensional feature are represented by... p y represents the original two-dimensional feature's y-coordinate, and p represents the relevant index of the two-dimensional feature.

[0078] The fusion convolution first concatenates and fuses the sparse BEV 2D features and the upsampled 2D features, and then uses a 3D standard residual sparse convolution module to remove noise, which can be formally expressed as:

[0079] Among them, Output fused SMC represents the features after fusion. i,j / s denoted as 2D submanifold sparse convolution operation, i,j represent the kernel size of the 2D submanifold sparse convolution operation in the x and y directions, s represents the stride of the convolution kernel, and Contact represents concatenation.

[0080] In summary, the 3D feature L6 output by the L-shaped backbone feature extraction network is compressed into a sparse BEV 2D feature P6. Then, the sparse BEV 2D feature P6 and the 3D feature L5 are input into the cross-dimensional encoding module to obtain the sparse BEV 2D feature P5; subsequently, the sparse BEV 2D feature P5 and the 3D feature L4 are input into the cross-dimensional encoding module to obtain the feature sparse BEV 2D feature P4. Finally, the cross-dimensional autoencoding network, composed of 3D features L6, L5, L6 and 2D features P6, P5, P4, achieves efficient feature fusion and representation.

[0081] Step 4: Construct a train forward obstacle detection algorithm using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network. Input the training dataset to obtain the forward obstacle detection model; specifically, this includes the following steps:

[0082] Step 4.1: Dynamically voxelize the input 3D point cloud;

[0083] Step 4.2: Input the voxelized point cloud into the L-shaped backbone feature extraction network for feature extraction to obtain features L1-L6;

[0084] Step 4.3: Connect the cross-dimensional autoencoder network after the L-shaped backbone feature extraction network to obtain features P4, P5 and P6;

[0085] Step 4.4: Input feature P4 into the sparse center prediction head to obtain the 3D center coordinates, length, width, height, and angle of each obstacle. The sparse center prediction head uses five prediction heads, each consisting of a 2D submanifold sparse convolution operation with a 3×3 kernel and a 1×1 kernel, to predict the (x, y) coordinates, z coordinates, (l, w, h) length, width, height, angle, and heatmap of the obstacles. The sparse center head can be represented as:

[0086] Among them, SMC i,j / sLet represent a 2D submanifold sparse convolution operation, where i,j represent the kernel size of the 2D submanifold sparse convolution operation in the x and y directions, and s represent the stride of the kernel.

[0087] Step 5: Use the trained forward obstacle detection model to process the real-time point cloud data input by the lidar, detect obstacles in the direction of the train's movement, and obtain the 3D bounding boxes and point clouds of the obstacles.

[0088] In summary, the L-shaped residual sparse convolution module proposed in this embodiment fully considers the spatial distribution of the point cloud in the train's forward operating environment, avoiding the repeated extraction of useless features. By focusing features on the x and z axes, the L-shaped backbone feature extraction network increases the receptive field size without sacrificing the output feature map size, thus obtaining high-resolution semantic features. Furthermore, the cross-dimensional autoencoder network compresses features from three-dimensional space to two-dimensional space and fuses and extracts features at different scales, achieving information interaction between high-level and shallow features of different dimensions. This enhances the positional semantic information of the features and further improves the algorithm's performance. By fusing the L-shaped backbone feature extraction network and the cross-dimensional autoencoder network, a robust train forward obstacle detection algorithm is constructed, achieving high-precision three-dimensional detection of obstacles in front of the train.

[0089] Example 3

[0090] As shown in Figure 1, this embodiment provides a method for detecting obstacles ahead of a train using lidar, including the following steps:

[0091] Step 1: Use LiDAR to scan the train's forward running scene, obtain 3D point cloud data under different scenes, label obstacle point clouds, and build a training dataset.

[0092] Step 2: An L-shaped sparse residual convolution module was designed, and an L-shaped backbone feature extraction network was constructed.

[0093] Step 2 specifically includes the following steps:

[0094] Step 2.1, Figure 2 is a structural diagram of the L-shaped residual sparse convolution module described in this embodiment of the invention. Considering the elongated and narrow spatial distribution of point clouds in railway operating environments, an L-shaped sparse residual convolution module is designed, consisting of two 3D submanifold sparse convolution operations. Its calculation formula is as follows: LB = R(SMC) i,j,k＝5,3,1 / s＝1 B(SMC i,j,k＝1,3,5 / s＝1 BR(Input))+Input) (7)

[0095] Where Input represents the input features, and SMC i,j,k / s SMC represents the sparse convolution operation on 3D submanifolds.i,j,k / s B represents the SMC-BN sequential structure, SMC i,j,k / s BR represents the SMC-BN-ReLU sequential operation, i,j,k represent the kernel size of the 3D submanifold sparse convolution operation in the x, y, and z directions, s represents the stride of the convolution kernel, and R represents the ReLU activation function.

[0096] The point cloud of a train's forward operating environment typically exhibits a long and narrow spatial distribution. This spatial distribution limitation restricts the feature extraction capability of commonly used standard residual sparse convolution modules in train forward operating environment point clouds. Therefore, this paper proposes an L-shaped sparse residual convolution module. In the L-shaped sparse residual convolution module, 3D submanifold sparse convolution operations with a 1×3×5 kernel and 5×3×1 kernel are superimposed to extract depth features from the input point cloud in an L-shape. This combination allows irregularly sized sparse convolutions to focus on feature extraction in different directions in three-dimensional space. Specifically, the 1×3×5 kernel submanifold sparse convolution operation used in this invention focuses on extracting features in the z-axis direction, while the 5×3×1 kernel submanifold sparse convolution operation focuses on extracting features in the x-axis direction. This allows the module to increase the receptive field size without sacrificing the output feature map size, thereby obtaining high-resolution semantic features.

[0097] Step 2.2: Construct an L-shaped backbone feature extraction network using an L-shaped sparse residual convolution module. Its specific structure is as follows:

[0098] Where LB represents an L-shaped sparse residual convolution module, and SC i,j,k / s This represents a 3D regular sparse convolution operation, where i, j, k represent the kernel size in the x, y, and z directions, s represents the kernel stride, n represents the number of layers, and L0 represents the original input features.

[0099] As shown in Table 1, the L-shaped backbone feature extraction network outputs a total of 6 layers, with each layer being half the size of the previous layer. If the scale of L1 is considered to be 1, then the scales of L2, L3, L4, L5, and L6 are 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32, respectively. Furthermore, the L-shaped backbone feature extraction network designed in this embodiment uses two L-shaped sparse residual convolutional modules to extract features in the first three layers and one L-shaped sparse residual convolutional module in the last three layers. This design ensures effective capture of spatial information at different scales while achieving a balance between computational efficiency and feature extraction accuracy.

[0100] Table 1

[0101] Step 3: A cross-dimensional encoding module was designed, and a cross-dimensional autoencoding network was constructed;

[0102] Step 3 specifically includes the following steps:

[0103] Step 3.1, Figure 3 is a structural diagram of the cross-dimensional coding module according to an embodiment of the present invention. The cross-dimensional coding module of the present invention consists of three parts: three-dimensional feature compression, two-dimensional feature upsampling, and fusion convolution;

[0104] Step 3.2, 3D feature compression, involves accumulating the sparse 3D features input to the backbone network along the z-axis. While preserving sparsity, this process maps the sparse 3D features to sparse BEV 2D features. This process can be represented as:

[0105] Where SBEV represents sparse bird's-eye view features, Input 3D This represents the sparse three-dimensional features of the input.

[0106] It should be noted that in this invention, the forward direction of the train is taken as the positive x-axis, the horizontal leftward direction as the positive y-axis, and the vertical upward direction as the positive z-axis. The origin (0, 0, 0) is the emission point of the vehicle-mounted lidar. During the accumulation process, all feature values ​​are accumulated along the z-axis until z = min(z 雷达扫描范围 On the plane, the width of the feature is made 1, thereby transforming the sparse three-dimensional feature into a two-dimensional feature while keeping the x and y features unchanged.

[0107] Step 3.3, Two-dimensional feature upsampling: The sparse BEV two-dimensional features are upsampled, and the process can be represented as follows:

[0108] in, Let x represent the upsampled two-dimensional feature. p The x-coordinate and y-coordinate of the original two-dimensional feature are represented by... p y represents the original two-dimensional feature's y-coordinate, and p represents the relevant index of the two-dimensional feature.

[0109] Sparse features, as a special data structure, consist of three parts: spatial shape, feature coordinates, and feature values. Taking advantage of their hierarchical composition, this invention directly doubles the feature coordinates of sparse features without altering their original features, thus achieving an upsampling effect while preserving the original features to the maximum extent.

[0110] Step 3.4, the fusion convolution first concatenates and fuses the sparse BEV 2D features and the upsampled 2D features, and then uses a 3D standard residual sparse convolution module to remove noise, which can be formally expressed as:

[0111] Among them, Output fused SMC represents the features after fusion. i,j / s denoted as 2D submanifold sparse convolution operation, i,j represent the kernel size of the 2D submanifold sparse convolution operation in the x and y directions, s represents the stride of the convolution kernel, and Contact represents concatenation.

[0112] During feature fusion, frequent interactions between feature information of different dimensions and scales often result in noise and instability in the fused features. These redundancies and interferences can adversely affect subsequent obstacle detection tasks. To address this, this invention introduces a 2D standard residual sparse convolution module, connected after the fused features, to remove noise and extract more critical semantic information.

[0113] Step 3.5, Figure 4 is a structural diagram of the cross-dimensional autoencoding network described in this embodiment of the invention. In summary, the three-dimensional feature L6 output by the L-shaped backbone feature extraction network is compressed into a sparse BEV two-dimensional feature P6. Then, the sparse BEV two-dimensional feature P6 and the three-dimensional feature L5 are input into the cross-dimensional encoding module to obtain the sparse BEV two-dimensional feature P5; subsequently, the sparse BEV two-dimensional feature P5 and the three-dimensional feature L4 are input into the cross-dimensional encoding module to obtain the sparse BEV two-dimensional feature P4. In this process, the three-dimensional features of different scales output by the backbone network are efficiently transformed into two-dimensional features, and through cross-scale and cross-dimensional feature fusion, the diversity and accuracy of feature representation are significantly enhanced. Finally, the cross-dimensional autoencoding network, composed of three-dimensional features L6, L5, L6 and two-dimensional features P6, P5, P4, adaptively utilizes multi-source feature information, improving the expressive power and robustness of the features.

[0114] Step 4: Figure 5 is a structural diagram of the train forward obstacle detection algorithm according to an embodiment of the present invention. An L-shaped backbone feature extraction network and a cross-dimensional autoencoder network are used to construct the train forward obstacle detection algorithm. The training dataset is input to obtain the forward obstacle detection model.

[0115] Step 4 specifically includes the following steps:

[0116] Step 4.1: Dynamically voxelize the input 3D point cloud;

[0117] Step 4.2: Input the voxelized point cloud into the L-shaped backbone feature extraction network for feature extraction to obtain features L1-L6;

[0118] Step 4.3: Connect the cross-dimensional autoencoder network after the L-shaped backbone feature extraction network to obtain features P4, P5 and P6;

[0119] Step 4.4: Input feature P4 into the sparse center prediction head to obtain the 3D center coordinates, length, width, height, and angle of each obstacle. The sparse center prediction head uses five prediction heads, each consisting of a 2D submanifold sparse convolution operation with a 3×3 kernel and a 1×1 kernel, to predict the (x, y) coordinates, z coordinates, (l, w, h) length, width, height, angle, and heatmap of the obstacles. The sparse center head can be represented as:

[0120] Among them, SMC i,j / s Let represent a 2D submanifold sparse convolution operation, where i,j represent the kernel size of the 2D submanifold sparse convolution operation in the x and y directions, and s represent the stride of the kernel.

[0121] Step 5: Use the trained forward obstacle detection model to process the real-time point cloud data input by the lidar, detect obstacles in the direction of the train's movement, and obtain the 3D bounding boxes and point clouds of the obstacles.

[0122] As shown in Figure 6, to verify the effectiveness and superiority of the method described in this embodiment, a section of railway point cloud (2418 point cloud data points, 1692 for training and 726 for testing) was selected for verification. It was compared with existing algorithms, specifically PointPillar, PillarNet, CenterPoint, Part-A2-Net, and VoxelNeXt. This embodiment uses the average mean average precision (mAP) and the average mean average recall (mAR) to measure the obstacle detection capability of various methods. A higher average precision indicates more accurate detection of obstacle locations, and a higher average recall indicates fewer missed obstacles. Table 2 shows the experimental results.

[0123] Table 2 Comparison of Experimental Results

[0124] As can be seen from Table 2, the mAP of this method is much greater than that of other algorithms, and the mAR of this method is also greater than that of other algorithms, thus proving that this method can achieve better obstacle detection.

[0125] The innovation of the train forward obstacle detection method using LiDAR proposed in this embodiment is mainly reflected in the following three aspects: First, an L-shaped backbone feature extraction network composed of L-shaped residual sparse convolutional modules is used to extract features of the input voxels. This backbone network can extract higher-dimensional semantic features. Second, a cross-dimensional autoencoder network is designed to realize information interaction between high-level features and shallow features of different dimensions, thereby enhancing the positional semantic information of the features. Third, by fusing the L-shaped backbone feature extraction network and the cross-dimensional autoencoder network, a powerful train forward obstacle detection algorithm is designed, which effectively improves the obstacle detection performance.

[0126] Example 4

[0127] This embodiment 4 provides a non-transitory computer-readable storage medium for storing computer instructions. When executed by a processor, the computer instructions implement the train forward obstacle intrusion detection method using lidar as described above. The method includes:

[0128] Acquire real-time point cloud data from LiDAR;

[0129] A pre-trained forward obstacle detection model is used to process real-time point cloud data input from LiDAR to detect obstacles in the direction of train movement, obtaining 3D bounding boxes and point clouds of the obstacles. Training the forward obstacle detection model includes: scanning the train's forward movement scene with LiDAR to acquire 3D point cloud data under different scenes, labeling the obstacle point clouds, and constructing a training dataset; constructing a train forward obstacle detection algorithm using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network, inputting the algorithm into the training dataset for training, and obtaining the forward obstacle detection model.

[0130] Example 5

[0131] This embodiment 5 provides a computer device, including a memory and a processor, wherein the processor and the memory communicate with each other, and the memory stores program instructions that can be executed by the processor. The processor calls the program instructions to execute the train forward obstacle intrusion detection method using lidar as described above, the method including:

[0132] Acquire real-time point cloud data from LiDAR;

[0133] A pre-trained forward obstacle detection model is used to process real-time point cloud data input from LiDAR to detect obstacles in the direction of train movement, obtaining 3D bounding boxes and point clouds of the obstacles. Training the forward obstacle detection model includes: scanning the train's forward movement scene with LiDAR to acquire 3D point cloud data under different scenes, labeling the obstacle point clouds, and constructing a training dataset; constructing a train forward obstacle detection algorithm using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network, inputting the algorithm into the training dataset for training, and obtaining the forward obstacle detection model.

[0134] Example 6

[0135] This embodiment 6 provides an electronic device, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the train forward obstacle intrusion detection method using lidar as described above, the method including:

[0136] Acquire real-time point cloud data from LiDAR;

[0137] A pre-trained forward obstacle detection model is used to process real-time point cloud data input from LiDAR to detect obstacles in the direction of train movement, obtaining 3D bounding boxes and point clouds of the obstacles. Training the forward obstacle detection model includes: scanning the train's forward movement scene with LiDAR to acquire 3D point cloud data under different scenes, labeling the obstacle point clouds, and constructing a training dataset; constructing a train forward obstacle detection algorithm using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network, inputting the algorithm into the training dataset for training, and obtaining the forward obstacle detection model.

[0138] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0140] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0141] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to perform a series of operational steps on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0142] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solutions disclosed in the present invention, various modifications or variations that can be made by those skilled in the art without creative effort should be included within the scope of protection of the present invention.

Claims

1. A train forward obstacle intrusion detection method using a laser radar, characterized by, include: Acquire real-time point cloud data from LiDAR; A pre-trained forward obstacle detection model is used to process real-time point cloud data input from LiDAR, detecting obstacles in the direction of train movement and obtaining 3D bounding boxes and point clouds of the obstacles. Training the forward obstacle detection model includes: scanning the train's forward movement scene with LiDAR to obtain 3D point cloud data under different train forward movement scenes, labeling obstacle point clouds, and constructing a training dataset; constructing a train forward obstacle detection algorithm using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network; inputting the training dataset into the train forward obstacle detection model and training it using the train forward obstacle detection algorithm to obtain a trained forward obstacle detection model.

2. The train forward obstacle intrusion detection method using a laser radar according to claim 1, wherein The aforementioned method for detecting forward obstacle encroachment on trains using lidar also includes: Based on an L-shaped sparse residual convolution module, an L-shaped backbone feature extraction network is constructed. This addresses the elongated and narrow spatial distribution of point clouds in railway operating environments. The L-shaped sparse residual convolution module consists of two 3D submanifold sparse convolution operations, and its calculation formula is as follows: LB = R(SMC i,j,k＝5,3,1 / s＝1 B(SMC i,j,k＝1,3,5 / s＝1 BR(Input) + Input) where Input denotes the input feature, SMC i,j,k / s denotes the 3D submanifold sparse convolution operation, SMC i,j,k / s B represents the SMC-BN sequential structure, SMC i,j,k / s BR denotes the SMC-BN-ReLU sequential operation, i, j, k represent the kernel size of the 3D submanifold sparse convolution operation in x, y, z directions, s represents the step of the kernel, and R represents the Relu activation function.

3. The method of train forward obstacle intrusion detection using laser radar according to claim 2, wherein, The aforementioned method for detecting forward obstacle encroachment on trains using lidar also includes: An L-shaped main feature extraction network is constructed using an L-shaped sparse residual convolution module, and a specific structure of the L-shaped main feature extraction network is as follows: wherein LB represents an L-type sparse residual convolution module, SC i,j,k / s represents a 3D regular sparse convolution operation, i, j, k represent the kernel size of the regular sparse convolution operation in the x, y, z three directions, s represents the step of the kernel, n represents the number of layers, and L0 is the original input feature.

4. The train forward obstacle detection method using a laser radar according to claim 3, characterized by, The aforementioned method for detecting forward obstacle encroachment on trains using lidar also includes: A cross-dimensional autoencoding network was constructed based on the cross-dimensional encoding module, which consists of three parts: three-dimensional feature compression, two-dimensional feature upsampling, and fusion convolution. The three-dimensional feature compression accumulates the sparse three-dimensional features input by the backbone network as a whole along the z-axis, while maintaining sparsity, maps the sparse three-dimensional features to sparse BEV two-dimensional features, and is expressed as: wherein SBEV denotes sparse bird's eye view features, Input 3D denotes input sparse 3D features; The two-dimensional feature up-sampling up-samples the sparse BEV two-dimensional feature, denoted as: wherein x denotes the up-sampled two-dimensional feature p x denotes the x-coordinate of the original two-dimensional feature p y denotes the y-coordinate of the original two-dimensional feature, and p denotes the relevant index of the two-dimensional feature. The fusion convolution first splices and fuses the sparse BEV two-dimensional feature and the up-sampled two-dimensional feature, and then uses a 3D standard residual sparse convolution module to remove noise, which is formally expressed as: where Output fused denotes the fused features, SMC i,j / s denotes the 2D submanifold sparse convolution operation, i, j denote the kernel size of the 2D submanifold sparse convolution operation in x, y directions, s denotes the step size of the kernel, and Contact denotes concatenation.

5. The train forward obstacle detection method using a laser radar according to claim 4, wherein, The train forward obstacle detection algorithm constructed using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network includes the following steps: A training dataset is input into the train forward obstacle detection model, and the algorithm is used to train the model to obtain a trained forward obstacle detection model. An algorithm for detecting forward obstacles to trains is constructed using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network. The training dataset is then input into the train forward obstacle detection model. The input 3D point cloud is dynamically voxelized; the voxelized point cloud is then input into an L-shaped backbone feature extraction network for feature extraction to obtain features L1-L6. A cross-dimensional autoencoder network is connected after the L-shaped backbone feature extraction network. The cross-dimensional autoencoder network outputs features P4, P5 and P6. The feature P4 is input into the sparse center prediction head to obtain the three-dimensional center coordinates, length, width, height and angle of each obstacle. The sparse center prediction head uses five prediction heads consisting of a 2D submanifold sparse convolution operation with a 3×3 convolution kernel size and a 2D submanifold sparse convolution operation with a 1×1 convolution kernel size to predict the (x, y) coordinates, z coordinates, (l, w, h) length, width, height, angle and heatmap of the obstacle. After training, the trained forward obstacle detection model is obtained.

6. The train forward obstacle detection method using a laser radar according to claim 5, wherein, Sparse center head is represented as: P center = 5 x SMC i,j＝3,3 / s＝1 (SMC i,j＝1,1 / s＝1 (P4)) = {(x pred ,y pred ), z pred ,(l pred ,w pred ,h pred ), angle pred , heatmap pred} where SMC i,j / s denotes 2D submanifold sparse convolution operation, i, j denote the kernel size of 2D submanifold sparse convolution operation in x, y direction, and s denotes the step of the kernel.

7. A train forward obstacle intrusion detection system using a laser radar, characterized by, include: The acquisition module is used to acquire real-time point cloud data from the LiDAR. The detection module is used to process real-time point cloud data input from LiDAR using a pre-trained forward obstacle detection model, detect obstacles in the direction of train movement, and obtain the 3D bounding boxes and point clouds of the obstacles. Training the forward obstacle detection model includes: scanning the train's forward movement scene using LiDAR to obtain 3D point cloud data under different train forward movement scenes, labeling the obstacle point clouds, and constructing a training dataset; constructing a train forward obstacle detection algorithm using an L-shaped backbone feature extraction network and a cross-dimensional autoencoder network; inputting the training dataset into the train forward obstacle detection model and training it using the train forward obstacle detection algorithm to obtain a trained forward obstacle detection model.

8. A non-transitory computer-readable storage medium, comprising: The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the train forward obstacle intrusion detection method using lidar as described in any one of claims 1-6.

9. A computer device, comprising: The system includes a memory and a processor, which communicate with each other. The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the train forward obstacle intrusion detection method using lidar as described in any one of claims 1-6.

10. An electronic device, comprising: include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the train forward obstacle intrusion detection method using lidar as described in any one of claims 1-6.

Images