Train forward obstacle invasion detection method and system using laser radar

Abstract

The invention provides a train forward obstacle intrusion detection method and system using a laser radar, and belongs to the technical field of train operation obstacle detection. Real-time point cloud data of the laser radar are acquired; processing real-time point cloud data input by the laser radar by using a pre-trained forward obstacle detection model, detecting an obstacle in the forward direction of train operation, and obtaining a three-dimensional bounding box and a point cloud of the obstacle; wherein an L-shaped trunk feature extraction network and a cross-dimension automatic coding network are used for constructing a train forward obstacle detection algorithm, a training data set is input for training, and a forward obstacle detection model is obtained. According to the method, the spatial distribution pattern of the point cloud of the forward operation environment of the train is considered, repeated extraction of useless features is avoided, the size of a receptive field is increased under the condition that the output feature mapping size is not lost, and high-resolution semantic features are obtained; information interaction between high-level features and shallow-level features in different dimensions is realized, so that position semantic information of the features is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of train running obstacle detection, and in particular to a train forward obstacle intrusion detection method and system using a laser radar. Background Art

[0002] With the continuous advancement of digital construction such as intelligent and information-based rail transit, environmental perception technology has become one of the main development directions of modern rail transit trains. Obstacle detection, as a key function of train environmental perception, can assist trains in timely detecting obstacles on the route and ensure the safety of train operation.

[0003] At present, the forward obstacle detection technology for rail transit trains can be divided into methods based on industrial cameras, millimeter wave radars, frequency modulated continuous wave radars, and laser radars according to the sensors used. The method based on industrial cameras uses sensors such as color cameras or infrared cameras to obtain high-resolution image information in front of the train, and has high obstacle detection accuracy, but it is limited to two-dimensional images and it is difficult to provide three-dimensional information such as the distance of obstacles. The method based on millimeter wave radar uses 3D or 4D millimeter wave radars to obtain three-dimensional point cloud information in front of the train. The radio waves it transmits can penetrate rain and fog, and its performance is good in bad weather, but its point cloud density is too sparse, making it difficult to detect obstacles at medium and long distances. The method based on frequency modulated continuous wave radar uses a continuous wave radar whose transmission frequency is modulated by a specific signal to obtain grayscale image information in the Cartesian coordinate system in front of the train. It has a long detection distance and strong resistance to bad weather, but the resolution of the obtained grayscale image is low, making it difficult to detect obstacles. The lidar-based method uses solid-state or semi-solid-state lidar to obtain high-resolution three-dimensional point cloud information in front of the train. It has a high point cloud density, a long detection distance and is not affected by lighting conditions. It shows great application potential in the field of forward obstacle detection for rail transit trains. Summary of the invention

[0004] The object of the present invention is to provide a method and system for detecting obstacle intrusion ahead of a train using a laser radar, so as to solve at least one technical problem existing in the above-mentioned background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a method for detecting a train forward obstacle intrusion using a laser radar, comprising:

[0007] Get real-time point cloud data from LiDAR;

[0008] A pre-trained forward obstacle detection model is used to process real-time point cloud data input by a laser radar, obstacles ahead of the train are detected, and a three-dimensional bounding box and point cloud of the obstacle are obtained; wherein, the training of the forward obstacle detection model includes: using a laser radar to scan the forward running scene of the train, obtaining three-dimensional point cloud data under different scenes, annotating obstacle point clouds, and forming a training data set; using an L-type backbone feature extraction network and a cross-dimensional automatic encoding network to construct a train forward obstacle detection algorithm, inputting the training data set for training, and obtaining a forward obstacle detection model.

[0009] Furthermore, based on the L-type sparse residual convolution module, an L-type backbone feature extraction network is constructed, including: for the slender and narrow spatial distribution of the railway operation environment point cloud, the L-type sparse residual convolution module is composed of two 3D sub-manifold sparse convolution operations, and its calculation formula is as follows:

[0010] LB=R(SMC i,j,k＝5,3,1 / s＝1 B(SMC i,j,k＝1,3,5 / s＝1 BR(Input))+Input)

[0011] Among them, Input represents the input feature, SMC i,j,k / s Represents 3D submanifold sparse convolution operation, SMC i,j,k / s B represents the SMC-BN sequential structure, SMC i,j,k / s BR represents the SMC-BN-ReLU sequential operation, i, j, k represent the convolution kernel sizes of the 3D submanifold sparse convolution operation in the x, y, and z directions, s represents the step size of the convolution kernel, and R represents the ReLU activation function.

[0012] Furthermore, an L-type sparse residual convolution module is used to construct an L-type backbone feature extraction network, and its specific structure is as follows:

[0013]

[0014] Among them, LB represents the L-type sparse residual convolution module, SC i,j,k / s It represents a 3D regular sparse convolution operation, i, j, k represent the convolution kernel size of the regular sparse convolution operation in the x, y, and z directions, s represents the step size of the convolution kernel, n represents the number of layers, and L0 is the original input feature.

[0015] Furthermore, based on the cross-dimensional encoding module, a cross-dimensional automatic encoding network is constructed, including: the cross-dimensional encoding module is composed of three parts: three-dimensional feature compression, two-dimensional feature upsampling and fusion convolution;

[0016] The three-dimensional feature compression accumulates the sparse three-dimensional features of the backbone network input along the z-axis as a whole, and maps the sparse three-dimensional features into sparse BEV two-dimensional features while maintaining sparsity, which can be expressed as:

[0017]

[0018] Among them, SBEV represents sparse bird's-eye view features, Input 3D Represents the sparse three-dimensional features of the input;

[0019] Two-dimensional feature upsampling The sparse BEV two-dimensional feature is upsampled and expressed as:

[0020]

[0021] in, represents the two-dimensional feature after upsampling, x p Represents the x-coordinate, y-coordinate of the original two-dimensional feature p represents the y coordinate of the original two-dimensional feature, and p represents the related index of the two-dimensional feature.

[0022] The fused convolution first concatenates the sparse BEV 2D features and the upsampled 2D features, and then uses a 3D standard residual sparse convolution module to remove noise, which can be formally expressed as:

[0023]

[0024] Among them, Output fused Represents the fused features, SMC i,j / s Represents a 2D submanifold sparse convolution operation, i, j represent the convolution kernel size of the 2D submanifold sparse convolution operation in the x and y directions, s represents the step size of the convolution kernel, and Contact represents splicing.

[0025] Furthermore, an L-shaped backbone feature extraction network and a cross-dimensional auto-encoding network are used to construct a train forward obstacle detection algorithm. The training data set is input to obtain a forward obstacle detection model, including:

[0026] Dynamically voxelize the input 3D point cloud;

[0027] The voxelized point cloud is input into the L-type backbone feature extraction network for feature extraction to obtain features L1-L6;

[0028] After the L-shaped backbone feature extraction network, a cross-dimensional auto-encoding network is connected to obtain features P4, P5, and P6;

[0029] The feature P4 is input into the sparse center prediction head to obtain the three-dimensional center coordinates, length, width, height and angle of each obstacle; the sparse center prediction head uses 5 prediction heads consisting of a 2D submanifold sparse convolution operation with a 3×3 convolution kernel size and a 2D submanifold sparse convolution operation with a 1×1 convolution kernel size to predict the obstacle’s (x, y) coordinates, z coordinates, (l, w, h) length, width, height, angle and heat map.

[0030] Furthermore, the sparse central head is expressed as:

[0031] P center =5×SMC i,j＝3,3 / s＝1 (SMC i,j＝1,1 / s＝1 (P4))

[0032] ={(x pred ,y pred ),z pred ,(l pred ,w pred ,h pred ),angle pred ,heatmap pred}

[0033] Among them, SMC i,j / s Represents a 2D submanifold sparse convolution operation, i, j represent the convolution kernel size of the 2D submanifold sparse convolution operation in the x and y directions, and s represents the step size of the convolution kernel.

[0034] In a second aspect, the present invention provides a train forward obstacle intrusion detection system using a laser radar, comprising:

[0035] Acquisition module, used to obtain real-time point cloud data of LiDAR;

[0036] The detection module is used to process the real-time point cloud data input by the laser radar using a pre-trained forward obstacle detection model, detect obstacles in the forward direction of the train, and obtain the three-dimensional bounding box and point cloud of the obstacle; wherein, the training of the forward obstacle detection model includes: using the laser radar to scan the forward running scene of the train, obtaining the three-dimensional point cloud data in different scenes, marking the obstacle point cloud, and forming a training data set; using the L-type backbone feature extraction network and the cross-dimensional automatic encoding network to construct the train forward obstacle detection algorithm, input the training data set for training, and obtain the forward obstacle detection model.

[0037] In a third aspect, the present invention provides a non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the method for detecting forward obstacle intrusion by a train using a laser radar as described in the first aspect is implemented.

[0038] In a fourth aspect, the present invention provides a computer device comprising a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method for detecting forward obstacle intrusion by a train using a laser radar as described in the first aspect.

[0039] In a fifth aspect, the present invention provides an electronic device comprising: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the method for detecting forward obstacle intrusion of a train using a laser radar as described in the first aspect.

[0040] Beneficial effects of the present invention: The proposed L-shaped residual sparse convolution module fully considers the spatial distribution morphology of the point cloud of the forward running environment of the train, avoids the repeated extraction of useless features, and by focusing on the features on the x-axis and z-axis, the L-shaped backbone feature extraction network increases the size of the receptive field without losing the size of the output feature map, thereby obtaining high-resolution semantic features. In addition, the cross-dimensional automatic encoding network compresses the features of the three-dimensional space into two-dimensional space, and fuses and extracts features at different scales, thereby realizing the information interaction between high-level features and shallow features of different dimensions, thereby enhancing the location semantic information of the features and further improving the performance of the algorithm. By fusing the L-shaped backbone feature extraction network and the cross-dimensional automatic encoding network, a good train forward obstacle detection algorithm is constructed, which realizes high-precision three-dimensional detection of obstacles ahead of the train.

[0041] Additional advantages of the present invention will be more clearly given in the following description or learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0043] Figure 1 This is a flow chart of a method for detecting an obstacle intrusion ahead of a train using a laser radar according to an embodiment of the present invention.

[0044] Figure 2 This is a structural diagram of the L-type residual sparse convolution module described in an embodiment of the present invention.

[0045] Figure 3It is a structural diagram of the cross-dimensional encoding module according to an embodiment of the present invention.

[0046] Figure 4 This is a structural diagram of the cross-dimensional automatic encoding network described in an embodiment of the present invention.

[0047] Figure 5 This is a structural diagram of a train forward obstacle detection algorithm according to an embodiment of the present invention.

[0048] Figure 6 The figure is a comparison chart of obstacle detection between the existing algorithm and the algorithm described in the present invention. DETAILED DESCRIPTION

[0049] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below by the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0050] It should be understood by those skilled in the art that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0051] It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have a meaning consistent with that in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless as defined herein.

[0052] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements and / or groups thereof.

[0053] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. Different embodiments or examples described in this specification and features of different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

[0054] To facilitate understanding of the present invention, the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings, and the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0055] Those skilled in the art should understand that the drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.

[0056] Example 1

[0057] In this embodiment 1, a train forward obstacle intrusion detection system using a laser radar is first provided, including: an acquisition module, used to acquire real-time point cloud data of the laser radar; a detection module, used to process the real-time point cloud data input by the laser radar using a pre-trained forward obstacle detection model, detect obstacles in the forward direction of the train, and obtain a three-dimensional bounding box and point cloud of the obstacle; wherein, training the forward obstacle detection model includes: using the laser radar to scan the forward running scene of the train, obtaining three-dimensional point cloud data in different scenes, marking the obstacle point cloud, and forming a training data set; using an L-type backbone feature extraction network and a cross-dimensional automatic encoding network to construct a train forward obstacle detection algorithm, inputting the training data set for training, and obtaining a forward obstacle detection model.

[0058] In this embodiment, the above-mentioned system is used to implement a method for detecting obstacles in front of a train using a laser radar, including: obtaining real-time point cloud data of the laser radar; using a pre-trained forward obstacle detection model to process the real-time point cloud data input by the laser radar, detecting obstacles in front of the train, and obtaining a three-dimensional bounding box and point cloud of the obstacle; wherein, training the forward obstacle detection model includes: using a laser radar to scan the forward running scene of the train, obtaining three-dimensional point cloud data in different scenes, annotating the obstacle point cloud, and forming a training data set; using an L-type backbone feature extraction network and a cross-dimensional automatic encoding network to construct a train forward obstacle detection algorithm, inputting the training data set for training, and obtaining a forward obstacle detection model.

[0059] Based on the L-type sparse residual convolution module, an L-type backbone feature extraction network is constructed, including: for the slender and narrow spatial distribution of the railway operation environment point cloud, the L-type sparse residual convolution module is composed of two 3D sub-manifold sparse convolution operations, and its calculation formula is as follows:

[0060] LB=R(SMC i,j,k＝5,3,1 / s＝1 B(SMC i,j,k＝1,3,5 / s＝1 BR(Input))+Input)

[0061] Among them, Input represents the input feature, SMC i,j,k / s Represents 3D submanifold sparse convolution operation, SMC i,j,k / sB represents the SMC-BN sequential structure, SMC i,j,k / s BR represents the SMC-BN-ReLU sequential operation, i, j, k represent the convolution kernel sizes of the 3D submanifold sparse convolution operation in the x, y, and z directions, s represents the step size of the convolution kernel, and R represents the ReLU activation function.

[0062] The L-type sparse residual convolution module is used to build an L-type backbone feature extraction network, and its specific structure is as follows:

[0063]

[0064] Among them, LB represents the L-type sparse residual convolution module, SC i,j,k / s It represents a 3D regular sparse convolution operation, i, j, k represent the convolution kernel size of the regular sparse convolution operation in the x, y, and z directions, s represents the step size of the convolution kernel, n represents the number of layers, and L0 is the original input feature.

[0065] Based on the cross-dimensional encoding module, a cross-dimensional automatic encoding network is constructed, including: the cross-dimensional encoding module consists of three parts: three-dimensional feature compression, two-dimensional feature upsampling and fusion convolution;

[0066] The three-dimensional feature compression accumulates the sparse three-dimensional features of the backbone network input along the z-axis as a whole, and maps the sparse three-dimensional features into sparse BEV two-dimensional features while maintaining sparsity, which can be expressed as:

[0067]

[0068] Among them, SBEV represents sparse bird's-eye view features, Input 3D Represents the sparse three-dimensional features of the input;

[0069] Two-dimensional feature upsampling The sparse BEV two-dimensional feature is upsampled and expressed as:

[0070]

[0071] in, represents the two-dimensional feature after upsampling, x p Represents the x-coordinate, y-coordinate of the original two-dimensional feature p represents the y coordinate of the original two-dimensional feature, and p represents the related index of the two-dimensional feature.

[0072] The fused convolution first concatenates the sparse BEV 2D features and the upsampled 2D features, and then uses a 3D standard residual sparse convolution module to remove noise, which can be formally expressed as:

[0073]

[0074] Among them, Output fused Represents the fused features, SMC i,j / s Represents a 2D submanifold sparse convolution operation, i, j represent the convolution kernel size of the 2D submanifold sparse convolution operation in the x and y directions, s represents the step size of the convolution kernel, and Contact represents splicing.

[0075] The train forward obstacle detection algorithm is constructed using the L-type backbone feature extraction network and the cross-dimensional automatic encoding network. The training data set is input to obtain the forward obstacle detection model, which includes:

[0076] Dynamically voxelize the input 3D point cloud;

[0077] The voxelized point cloud is input into the L-type backbone feature extraction network for feature extraction to obtain features L1-L6;

[0078] After the L-shaped backbone feature extraction network, a cross-dimensional auto-encoding network is connected to obtain features P4, P5, and P6;

[0079] The feature P4 is input into the sparse center prediction head to obtain the three-dimensional center coordinates, length, width, height and angle of each obstacle; the sparse center prediction head uses 5 prediction heads consisting of a 2D submanifold sparse convolution operation with a 3×3 convolution kernel size and a 2D submanifold sparse convolution operation with a 1×1 convolution kernel size to predict the obstacle’s (x, y) coordinates, z coordinates, (l, w, h) length, width, height, angle and heat map.

[0080] The sparse central head is expressed as:

[0081] P center =5×SMC i,j＝3,3 / s＝1 (SMC i,j＝1,1 / s＝1 (P4))

[0082] ={(x pred ,y pred ),z pred ,(l pred ,w pred ,h pred ),angle pred ,heatmap pred}

[0083] Among them, SMC i,j / s Represents a 2D submanifold sparse convolution operation, i, j represent the convolution kernel size of the 2D submanifold sparse convolution operation in the x and y directions, and s represents the step size of the convolution kernel.

[0084] Example 2

[0085] In this embodiment 2, a method for detecting obstacles ahead of a train using a laser radar is provided, which optimizes the feature extraction and fusion process by using an L-shaped backbone feature extraction network and a cross-dimensional automatic encoding network, thereby achieving high-precision three-dimensional measurement of obstacles ahead of the train and improving the detection accuracy of long-distance obstacles. The method includes the following steps:

[0086] Step 1: Use laser radar to scan the train forward running scene, obtain three-dimensional point cloud data in different scenes, mark the obstacle point cloud, and form a training data set;

[0087] Step 2, an L-shaped sparse residual convolution module is designed and an L-shaped backbone feature extraction network is constructed;

[0088] The step 2 specifically includes the following steps:

[0089] Step 2.1, in view of the slender and narrow spatial distribution of the railway operating environment point cloud, an L-type sparse residual convolution module is designed, which consists of two 3D sub-manifold sparse convolution operations. The calculation formula is as follows:

[0090] LB=R(SMC i,j,k＝5,3,1 / s＝1 B(SMC i,j,k＝1,3,5 / s＝1 BR(Input))+Input) (1)

[0091] Among them, Input represents the input feature, SMC i,j,k / s Represents 3D submanifold sparse convolution operation, SMC i,j,k / s B represents the SMC-BN sequential structure, SMC i,j,k / s BR represents the SMC-BN-ReLU sequential operation, i, j, k represent the convolution kernel sizes of the 3D submanifold sparse convolution operation in the x, y, and z directions, s represents the step size of the convolution kernel, and R represents the ReLU activation function.

[0092] Step 2.2, use the L-type sparse residual convolution module to build an L-type backbone feature extraction network, the specific structure of which is as follows:

[0093]

[0094] Among them, LB represents the L-type sparse residual convolution module, SC i,j,k / s It represents a 3D regular sparse convolution operation, i, j, k represent the convolution kernel size of the regular sparse convolution operation in the x, y, and z directions, s represents the step size of the convolution kernel, n represents the number of layers, and L0 is the original input feature.

[0095] Step 3, designing a cross-dimensional encoding module and constructing a cross-dimensional automatic encoding network; specifically including the following steps:

[0096] The cross-dimensional encoding module consists of three parts: three-dimensional feature compression, two-dimensional feature upsampling and fused convolution.

[0097] The three-dimensional feature compression accumulates the sparse three-dimensional features input by the backbone network along the z-axis as a whole, and maps the sparse three-dimensional features into sparse BEV two-dimensional features while maintaining sparsity. This process can be expressed as:

[0098]

[0099] Among them, SBEV represents sparse bird's-eye view features, Input 3D Represents the sparse 3D features of the input.

[0100] Two-dimensional feature upsampling The sparse BEV two-dimensional features are upsampled, and the process can be expressed as:

[0101]

[0102] in, represents the two-dimensional feature after upsampling, x p Represents the x-coordinate, y-coordinate of the original two-dimensional feature p represents the y coordinate of the original two-dimensional feature, and p represents the related index of the two-dimensional feature.

[0103] The fused convolution first concatenates the sparse BEV 2D features and the upsampled 2D features, and then uses a 3D standard residual sparse convolution module to remove noise, which can be formally expressed as:

[0104]

[0105] Among them, Output fused Represents the fused features, SMC i,j / s Represents a 2D submanifold sparse convolution operation, i, j represent the convolution kernel size of the 2D submanifold sparse convolution operation in the x and y directions, s represents the step size of the convolution kernel, and Contact represents splicing.

[0106] In summary, the three-dimensional feature L6 output by the L-type backbone feature extraction network is compressed into a sparse BEV two-dimensional feature P6. Then the sparse BEV two-dimensional feature P6 and the three-dimensional feature L5 are input into the cross-dimensional encoding module to obtain the sparse BEV two-dimensional feature P5; then the sparse BEV two-dimensional feature P5 and the three-dimensional feature L4 are input into the cross-dimensional encoding module to obtain the feature sparse BEV two-dimensional feature P4. Finally, the three-dimensional features L6, L5, L6 and the two-dimensional features P6, P5, P4 form a cross-dimensional automatic encoding network to achieve efficient feature fusion and representation.

[0107] Step 4: Use the L-type backbone feature extraction network and the cross-dimensional automatic encoding network to build a train forward obstacle detection algorithm, input the training data set, and obtain the forward obstacle detection model; specifically, the following steps are included:

[0108] Step 4.1, dynamically voxelize the input three-dimensional point cloud;

[0109] Step 4.2, input the voxelized point cloud into the L-type backbone feature extraction network for feature extraction to obtain features L1-L6;

[0110] Step 4.3, connect the cross-dimensional auto-encoding network after the L-shaped backbone feature extraction network to obtain features P4, P5 and P6;

[0111] Step 4.4, input feature P4 into the sparse center prediction head to obtain the three-dimensional center coordinates, length, width, height and angle of each obstacle. The sparse center prediction head uses 5 prediction heads consisting of a 2D submanifold sparse convolution operation with a 3×3 convolution kernel size and a 2D submanifold sparse convolution operation with a 1×1 convolution kernel size to predict the obstacle's (x, y) coordinates, z coordinates, (l, w, h) length, width, height, angle and heat map. The sparse center head can be expressed as:

[0112]

[0113] Among them, SMC i,j / s Represents a 2D submanifold sparse convolution operation, i, j represent the convolution kernel size of the 2D submanifold sparse convolution operation in the x and y directions, and s represents the step size of the convolution kernel.

[0114] Step 5: Use the trained forward obstacle detection model to process the real-time point cloud data input by the lidar, detect obstacles in the direction of the train, and obtain the three-dimensional bounding box and point cloud of the obstacles.

[0115] In summary, the L-type residual sparse convolution module proposed in this embodiment fully considers the spatial distribution morphology of the point cloud of the train's forward running environment, avoids the repeated extraction of useless features, and focuses on the features on the x-axis and z-axis. The L-type backbone feature extraction network increases the size of the receptive field without losing the size of the output feature map, and obtains high-resolution semantic features. In addition, the cross-dimensional automatic encoding network compresses the features of the three-dimensional space into two-dimensional space, and fuses and extracts features at different scales, thereby realizing the information interaction between high-level features and shallow features of different dimensions, thereby enhancing the location semantic information of the features and further improving the performance of the algorithm. By fusing the L-type backbone feature extraction network and the cross-dimensional automatic encoding network, a good train forward obstacle detection algorithm is constructed, and high-precision three-dimensional detection of obstacles ahead of the train is achieved.

[0116] Example 3

[0117] like Figures 1 to 5 As shown, this embodiment provides a method for detecting obstacles ahead of a train using a laser radar, comprising the following steps:

[0118] Step 1: Use a laser radar to scan the train forward running scene, obtain three-dimensional point cloud data in different scenes, mark the obstacle point cloud, and form a training data set;

[0119] Step 2, an L-shaped sparse residual convolution module is designed and an L-shaped backbone feature extraction network is constructed;

[0120] The step 2 specifically includes the following steps:

[0121] Step 2.1, in view of the slender and narrow spatial distribution of the railway operating environment point cloud, an L-type sparse residual convolution module is designed, which consists of two 3D sub-manifold sparse convolution operations. The calculation formula is as follows:

[0122] LB=R(SMC i,j,k＝5,3,1 / s＝1 B(SMC i,j,k＝1,3,5 / s＝1 BR(Input))+Input) (1)

[0123] Among them, Input represents the input feature, SMC i,j,k / s Represents 3D submanifold sparse convolution operation, SMC i,j,k / s B represents the SMC-BN sequential structure, SMC i,j,k / s BR represents the SMC-BN-ReLU sequential operation, i, j, k represent the convolution kernel sizes of the 3D submanifold sparse convolution operation in the x, y, and z directions, s represents the step size of the convolution kernel, and R represents the ReLU activation function.

[0124] The point cloud of the forward running environment of the train usually presents an elongated and narrow spatial distribution form. This difference in spatial distribution form leads to the limitation of the feature extraction ability of the commonly used standard residual sparse convolution module in the point cloud of the forward running environment of the train. Therefore, this paper proposes an L-type sparse residual convolution module. In the L-type sparse residual convolution module, the 3D sub-manifold sparse convolution operation with a convolution kernel size of 1×3×5 and the 3D sub-manifold sparse convolution operation with a convolution kernel size of 5×3×1 are superimposed into an L-shaped deep feature extraction of the input point cloud. This combination allows sparse convolutions of irregular sizes to focus on feature extraction in different directions in three-dimensional space. Specifically, the sub-manifold sparse convolution operation with a convolution kernel size of 1×3×5 adopted by the present invention focuses on extracting features in the z-axis direction, while the sub-manifold sparse convolution operation with a convolution kernel size of 5×3×1 focuses on extracting features in the x-axis direction. This enables the module to increase the size of the receptive field without losing the size of the output feature map, thereby obtaining high-resolution semantic features.

[0125] Step 2.2, use the L-type sparse residual convolution module to build an L-type backbone feature extraction network, the specific structure of which is as follows:

[0126]

[0127] Among them, LB represents the L-type sparse residual convolution module, SC i,j,k / s It represents a 3D regular sparse convolution operation, i, j, k represent the convolution kernel size of the regular sparse convolution operation in the x, y, and z directions, s represents the step size of the convolution kernel, n represents the number of layers, and L0 is the original input feature.

[0128] As shown in Table 1, the L-type backbone feature extraction network outputs a total of 6 layers, in which the size of each layer is reduced by two times relative to the previous layer. The scale of L1 is regarded as 1, and the scales of L2, L3, L4, L5, and L6 are 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32, respectively. In addition, the L-type backbone feature extraction network designed in this embodiment uses two L-type sparse residual convolution modules to extract features in the first three layers, and uses one L-type sparse residual convolution module to extract features in the last three layers. This design not only ensures that spatial information can be effectively captured at different scales, but also achieves a balance between computational efficiency and feature extraction accuracy.

[0129] Table 1

[0130]

[0131] Step 3: Design a cross-dimensional encoding module and construct a cross-dimensional automatic encoding network;

[0132] The step 3 specifically includes the following steps:

[0133] Step 3.1, a cross-dimensional encoding module, which consists of three parts: 3D feature compression, 2D feature upsampling and fusion convolution;

[0134] Step 3.2, 3D feature compression: The sparse 3D features input by the backbone network are accumulated along the z-axis as a whole. While maintaining sparsity, the sparse 3D features are mapped to sparse BEV 2D features. This process can be expressed as:

[0135]

[0136] Among them, SBEV represents sparse bird's-eye view features, Input 3D Represents the sparse 3D features of the input.

[0137] It should be noted that the present invention takes the forward direction of the train as the positive direction of the x-axis, the horizontal left direction of the train as the positive direction of the y-axis, the vertical upward direction of the train as the positive direction of the z-axis, and the origin (0, 0, 0) as the launch point of the vehicle-mounted laser radar. In the accumulation process, all eigenvalues ​​are accumulated along the z-axis to z=min(z 雷达扫描范围 ) plane, making the width of the feature become 1, and then converting the sparse three-dimensional features into two-dimensional features while keeping the x and y features unchanged.

[0138] Step 3.3, 2D feature upsampling The sparse BEV 2D features are upsampled, and the process can be expressed as:

[0139]

[0140] in, represents the two-dimensional feature after upsampling, x p Represents the x-coordinate, y-coordinate of the original two-dimensional feature p represents the y coordinate of the original two-dimensional feature, and p represents the related index of the two-dimensional feature.

[0141] As a special data structure, sparse features consist of three parts: spatial shape, feature coordinates, and feature values. In view of its hierarchical composition, the present invention directly doubles the feature coordinates of sparse features without changing their original features, so as to achieve the upsampling effect while retaining the original features to the maximum extent.

[0142] Step 3.4, the fused convolution first concatenates the sparse BEV 2D features and the upsampled 2D features, and then uses a 3D standard residual sparse convolution module to remove noise, which can be formally expressed as:

[0143]

[0144] Among them, Output fused Represents the fused features, SMC i,j / s Represents a 2D submanifold sparse convolution operation, i, j represent the convolution kernel size of the 2D submanifold sparse convolution operation in the x and y directions, s represents the step size of the convolution kernel, and Contact represents splicing.

[0145] In the process of feature fusion, due to the frequent interaction of feature information of different dimensions and scales, noise and unstable information often exist in the fused features. These redundant and interfering factors may have an adverse effect on the subsequent obstacle detection task. To this end, the present invention further introduces a 2D standard residual sparse convolution module, which is connected to the fused features to remove noise and extract more critical semantic information.

[0146] Step 3.5, in summary, the three-dimensional feature L6 output by the L-type backbone feature extraction network is compressed into a sparse BEV two-dimensional feature P6. Then the sparse BEV two-dimensional feature P6 and the three-dimensional feature L5 are input into the cross-dimensional encoding module to obtain the sparse BEV two-dimensional feature P5; then the sparse BEV two-dimensional feature P5 and the three-dimensional feature L4 are input into the cross-dimensional encoding module to obtain the sparse BEV two-dimensional feature P4. In this process, the three-dimensional features of different scales output by the backbone network are efficiently converted into two-dimensional features, and the diversity and accuracy of feature expression are significantly enhanced through cross-scale and dimensional feature fusion. Finally, the cross-dimensional automatic encoding network composed of three-dimensional features L6, L5, L6 and two-dimensional features P6, P5, P4 adaptively utilizes multi-source feature information and improves the expressiveness and robustness of the features.

[0147] Step 4: Use the L-type backbone feature extraction network and the cross-dimensional auto-encoding network to build a train forward obstacle detection algorithm, input the training data set, and obtain the forward obstacle detection model;

[0148] The step 4 specifically comprises the following steps:

[0149] Step 4.1, dynamically voxelize the input three-dimensional point cloud;

[0150] Step 4.2, input the voxelized point cloud into the L-type backbone feature extraction network for feature extraction to obtain features L1-L6;

[0151] Step 4.3, connect the cross-dimensional auto-encoding network after the L-shaped backbone feature extraction network to obtain features P4, P5 and P6;

[0152] Step 4.4, input feature P4 into the sparse center prediction head to obtain the three-dimensional center coordinates, length, width, height and angle of each obstacle. The sparse center prediction head uses 5 prediction heads consisting of a 2D submanifold sparse convolution operation with a 3×3 convolution kernel size and a 2D submanifold sparse convolution operation with a 1×1 convolution kernel size to predict the obstacle's (x, y) coordinates, z coordinates, (l, w, h) length, width, height, angle and heat map. The sparse center head can be expressed as:

[0153]

[0154] Among them, SMC i,j / s Represents a 2D submanifold sparse convolution operation, i, j represent the convolution kernel size of the 2D submanifold sparse convolution operation in the x and y directions, and s represents the step size of the convolution kernel.

[0155] Step 5: Use the trained forward obstacle detection model to process the real-time point cloud data input by the lidar, detect obstacles in the direction of the train, and obtain the three-dimensional bounding box and point cloud of the obstacles.

[0156] like Figure 5 As shown, in order to verify the effectiveness and superiority of the method described in this embodiment, a section of railway point cloud (2418 point cloud data, 1692 for training, and 726 for testing) was selected for verification. It is compared with existing algorithms, including PointPillar, PillarNet, CenterPoint, Part-A2-Net and VoxelNeXt. This embodiment uses the average of the mean detection precision (mAP: mean average precision) and the average of the mean recall (mAR: mean average recall) to measure the obstacle detection capabilities of various methods. The larger the average of the mean detection precision, the more accurate the location of the detected obstacle, and the larger the average of the mean recall, the fewer missed obstacles. Table 2 shows the experimental results.

[0157] Table 2 Comparison of experimental results

[0158]

[0159] It can be seen from Table 2 that the mAP of this method is much larger than the mAP of other algorithms, and the mAR of this method is also larger than the mAR of other algorithms, which proves that this method can achieve better obstacle detection.

[0160] The innovativeness of the train forward obstacle detection method using laser radar proposed in this embodiment is mainly reflected in the following three aspects: first, an L-type backbone feature extraction network composed of L-type residual sparse convolution modules is used to extract the features of the input voxels. The backbone network can extract higher-dimensional semantic features; second, a cross-dimensional automatic encoding network is designed to realize the information interaction between high-level features and shallow features of different dimensions, thereby enhancing the location semantic information of the features; third, by fusing the L-type backbone feature extraction network and the cross-dimensional automatic encoding network, a powerful train forward obstacle detection algorithm is designed, which effectively improves the obstacle detection performance.

[0161] Example 4

[0162] This embodiment 4 provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method for detecting a train obstacle intrusion using a laser radar as described above is implemented. The method includes:

[0163] Get real-time point cloud data from LiDAR;

[0164] A pre-trained forward obstacle detection model is used to process real-time point cloud data input by a laser radar, obstacles ahead of the train are detected, and a three-dimensional bounding box and point cloud of the obstacle are obtained; wherein, the training of the forward obstacle detection model includes: using a laser radar to scan the forward running scene of the train, obtaining three-dimensional point cloud data under different scenes, annotating obstacle point clouds, and forming a training data set; using an L-type backbone feature extraction network and a cross-dimensional automatic encoding network to construct a train forward obstacle detection algorithm, inputting the training data set for training, and obtaining a forward obstacle detection model.

[0165] Example 5

[0166] This embodiment 5 provides a computer device, including a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, the processor calls the program instructions to execute the above-mentioned train forward obstacle intrusion detection method using laser radar, the method comprising:

[0167] Get real-time point cloud data from LiDAR;

[0168] A pre-trained forward obstacle detection model is used to process real-time point cloud data input by a laser radar, obstacles ahead of the train are detected, and a three-dimensional bounding box and point cloud of the obstacle are obtained; wherein, the training of the forward obstacle detection model includes: using a laser radar to scan the forward running scene of the train, obtaining three-dimensional point cloud data under different scenes, annotating obstacle point clouds, and forming a training data set; using an L-type backbone feature extraction network and a cross-dimensional automatic encoding network to construct a train forward obstacle detection algorithm, inputting the training data set for training, and obtaining a forward obstacle detection model.

[0169] Example 6

[0170] This embodiment 6 provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes instructions for implementing the method for detecting a train forward obstacle intrusion using a laser radar as described above, the method comprising:

[0171] Get real-time point cloud data from LiDAR;

[0172] A pre-trained forward obstacle detection model is used to process real-time point cloud data input by a laser radar, obstacles ahead of the train are detected, and a three-dimensional bounding box and point cloud of the obstacle are obtained; wherein, the training of the forward obstacle detection model includes: using a laser radar to scan the forward running scene of the train, obtaining three-dimensional point cloud data under different scenes, annotating obstacle point clouds, and forming a training data set; using an L-type backbone feature extraction network and a cross-dimensional automatic encoding network to construct a train forward obstacle detection algorithm, inputting the training data set for training, and obtaining a forward obstacle detection model.

[0173] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0177] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative work on the basis of the technical solution disclosed in the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting obstacles in front of a train using laser radar, characterized in that: include: Get real-time point cloud data from LiDAR; A pre-trained forward obstacle detection model is used to process real-time point cloud data input by a laser radar, obstacles ahead of the train are detected, and a three-dimensional bounding box and point cloud of the obstacle are obtained; wherein, the training of the forward obstacle detection model includes: using a laser radar to scan the forward running scene of the train, obtaining three-dimensional point cloud data under different scenes, annotating obstacle point clouds, and forming a training data set; using an L-type backbone feature extraction network and a cross-dimensional automatic encoding network to construct a train forward obstacle detection algorithm, inputting the training data set for training, and obtaining a forward obstacle detection model.

2. The method for detecting a train obstacle intrusion using a laser radar according to claim 1, characterized in that: Based on the L-type sparse residual convolution module, an L-type backbone feature extraction network is constructed, including: for the slender and narrow spatial distribution of the railway operation environment point cloud, the L-type sparse residual convolution module is composed of two 3D sub-manifold sparse convolution operations, and its calculation formula is as follows: LB=R(SMC i,j,k＝5,3,1 / s＝1 B(SMC i,j,k＝1,3,5 / s＝1 BR(Input))+Input) Among them, Input represents the input feature, SMC i,j,k / s Represents 3D submanifold sparse convolution operation, SMC i,j,k / s B represents the SMC-BN sequential structure, SMC i,j,k / s BR represents the SMC-BN-ReLU sequential operation, i, j, k represent the convolution kernel sizes of the 3D submanifold sparse convolution operation in the x, y, and z directions, s represents the step size of the convolution kernel, and R represents the ReLU activation function.

3. The method for detecting a train obstacle intrusion using a laser radar according to claim 2, characterized in that: The L-type sparse residual convolution module is used to build an L-type backbone feature extraction network, and its specific structure is as follows: Among them, LB represents the L-type sparse residual convolution module, SC i,j,k / s It represents a 3D regular sparse convolution operation, i, j, k represent the convolution kernel size of the regular sparse convolution operation in the x, y, and z directions, s represents the step size of the convolution kernel, n represents the number of layers, and L0 is the original input feature.

4. The train forward obstacle detection method using laser radar according to claim 1, characterized in that: Based on the cross-dimensional encoding module, a cross-dimensional automatic encoding network is constructed, including: the cross-dimensional encoding module consists of three parts: three-dimensional feature compression, two-dimensional feature upsampling and fusion convolution; The three-dimensional feature compression accumulates the sparse three-dimensional features of the backbone network input along the z-axis as a whole, and maps the sparse three-dimensional features into sparse BEV two-dimensional features while maintaining sparsity, which can be expressed as: Among them, SBEV represents sparse bird's-eye view features, Input 3D Represents the sparse three-dimensional features of the input; Two-dimensional feature upsampling The sparse BEV two-dimensional feature is upsampled and expressed as: in, represents the two-dimensional feature after upsampling, x p Represents the x-coordinate, y-coordinate of the original two-dimensional feature p represents the y coordinate of the original two-dimensional feature, and p represents the related index of the two-dimensional feature. The fused convolution first concatenates the sparse BEV 2D features and the upsampled 2D features, and then uses a 3D standard residual sparse convolution module to remove noise, which can be formally expressed as: Among them, Output fused Represents the fused features, SMC i,j / s Represents a 2D submanifold sparse convolution operation, i, j represent the convolution kernel size of the 2D submanifold sparse convolution operation in the x and y directions, s represents the step size of the convolution kernel, and Contact represents splicing.

5. The train forward obstacle detection method using laser radar according to claim 1, characterized in that: The train forward obstacle detection algorithm is constructed using the L-type backbone feature extraction network and the cross-dimensional automatic encoding network. The training data set is input to obtain the forward obstacle detection model, which includes: Dynamically voxelize the input 3D point cloud; The voxelized point cloud is input into the L-type backbone feature extraction network for feature extraction to obtain features L1-L6; After the L-shaped backbone feature extraction network, a cross-dimensional auto-encoding network is connected to obtain features P4, P5, and P6; The feature P4 is input into the sparse center prediction head to obtain the three-dimensional center coordinates, length, width, height and angle of each obstacle; the sparse center prediction head uses 5 prediction heads consisting of a 2D submanifold sparse convolution operation with a 3×3 convolution kernel size and a 2D submanifold sparse convolution operation with a 1×1 convolution kernel size to predict the obstacle’s (x, y) coordinates, z coordinates, (l, w, h) length, width, height, angle and heat map.

6. The method for detecting obstacles ahead of a train using a laser radar according to claim 5, characterized in that: The sparse central head is expressed as: P center =5×SMC i,j＝3,3 / s＝1 (SMC i,j＝1,1 / s＝1 (P4)) ={(x pred ,y pred ),z pred ,(l pred ,w pred ,h pred ),angle pred ,heatmap pred } Among them, SMC i,j / s Represents a 2D submanifold sparse convolution operation, i, j represent the convolution kernel size of the 2D submanifold sparse convolution operation in the x and y directions, and s represents the step size of the convolution kernel.

7. A train forward obstacle intrusion detection system using laser radar, characterized in that: include: Acquisition module, used to obtain real-time point cloud data of LiDAR; The detection module is used to process the real-time point cloud data input by the laser radar using a pre-trained forward obstacle detection model, detect obstacles in the forward direction of the train, and obtain the three-dimensional bounding box and point cloud of the obstacle; wherein, the training of the forward obstacle detection model includes: using the laser radar to scan the forward running scene of the train, obtaining the three-dimensional point cloud data in different scenes, marking the obstacle point cloud, and forming a training data set; using the L-type backbone feature extraction network and the cross-dimensional automatic encoding network to construct the train forward obstacle detection algorithm, input the training data set for training, and obtain the forward obstacle detection model.

8. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the method for detecting forward obstacle intrusion of a train using a laser radar as described in any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: It includes a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the train forward obstacle intrusion detection method using laser radar as described in any one of claims 1-6.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the method for detecting forward obstacle intrusion of a train using a laser radar as described in any one of claims 1-6.

Images