Urban rail obstacle detection method based on deep learning

Abstract

The invention relates to the technical field of image recognition, in particular to an urban rail obstacle detection method based on deep learning, which comprises the following steps: recording the road condition of a train running rail in real time, and establishing a rail transit obstacle data set; marking obstacles in the data set into six categories through a tool, and dividing the picture data into a training set and a test set; performing data enhancement on partial pictures in the training set and the test set; according to the method, an urban rail obstacle detection model is constructed, the urban rail obstacle detection model is trained, the urban rail obstacle detection model designed by the method adopts local parallel cascade parallel space channel convolution, and a loss function of bounding box regression is designed. According to the method, the relative position angle information between the predicted bounding box and the real bounding box is more accurately expressed by the algorithm, the training effectiveness of the algorithm is improved, a calibrated data set is used on the model for training, and the trained weight file can be used for obstacle detection, so that efficient and real-time detection of front obstacles is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an urban track obstacle detection method based on deep learning. Background Art

[0002] With the growth of urban population and the continuous increase in transportation demand, rail transit systems play a vital role in modern cities. According to the modern comprehensive transportation system planning notice, it is necessary to create a metropolitan area on the rail, build a multi-level rail transit network in the metropolitan area, promote the integration and connection of trunk railways, intercity railways, urban (suburban) railways and urban rail transit, and reasonably promote cross-line operation of rail transit.

[0003] Pedestrian injuries, equipment damage and property losses caused by rail train safety issues at home and abroad cannot be ignored.

[0004] However, due to the limited range and long reaction time of traditional manual observation, it cannot meet the needs of train obstacle detection under high-speed driving conditions. Conventional target detection algorithms are difficult to deploy on mobile terminals due to the high complexity of their models. Therefore, efficient and lightweight automated obstacle detection methods have become an urgent need to ensure the safe and efficient operation of rail transit systems. Summary of the invention

[0005] The purpose of the present invention is to provide an urban rail obstacle detection method based on deep learning, aiming to improve the accuracy of rail transit obstacle detection while reducing the complexity of the model and the difficulty of model deployment.

[0006] To achieve the above object, the present invention provides an urban rail obstacle detection method based on deep learning, comprising the following steps:

[0007] Use on-board industrial cameras to record the track conditions of trains in real time and establish a rail transit obstacle dataset;

[0008] The obstacles in the dataset are labeled into six categories through a tool, and the image data is divided into a training set and a test set;

[0009] Performing data augmentation on some images in the training set and the test set;

[0010] An urban rail obstacle detection model is constructed and trained.

[0011] The specific method of using the on-board industrial camera to record the train track conditions in real time and establish a rail transit obstacle data set is as follows:

[0012] When the train is running, the on-board industrial camera records the track conditions of the train in real time to obtain the collected video;

[0013] Extract key frame images of obstacles contained in the captured video and save image samples;

[0014] A rail transit obstacle dataset is established based on the image samples.

[0015] Among them, the six categories include people, traffic lights, platforms, distribution boxes, billboards and small obstacles.

[0016] The data enhancement includes flipping, rotation and local cropping.

[0017] The specific method of training the urban rail obstacle detection model is as follows:

[0018] The evaluation indicators such as average detection accuracy, single-frame image detection time, model parameter quantity and floating-point number calculation quantity are set to evaluate the model effect.

[0019] The invention discloses a method for detecting urban rail obstacles based on deep learning. The method records the road conditions of a train track in real time through an on-board industrial camera to establish a rail transit obstacle data set. The obstacles in the data set are marked into six categories through a tool, and the image data is divided into a training set and a test set. Data enhancement is performed on some images in the training set and the test set. An urban rail obstacle detection model is constructed and the urban rail obstacle detection model is trained. The method designs a lightweight urban rail obstacle detection model, adopts parallel spatial channel convolution of local parallel cascade, and designs a loss function RSCIoU of bounding box IoU regression, so that the algorithm can more accurately express the relative position angle information between the predicted bounding box and the real bounding box, thereby improving the training effectiveness of the algorithm. The calibrated data set is used for training on the model, and the weight file obtained after the model training can be used for obstacle detection, so as to realize efficient and real-time detection of obstacles ahead, reduce the work intensity of the driver, and effectively avoid the occurrence of collision accidents. The detection algorithm has the potential to be applied to various fields and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a framework diagram of the LDCD-Net network model of the present invention, wherein a is a feature extraction network, b is a feature fusion network, c is a detection head, d is a feature compilation module, e is a convolution block, and f is a lightweight convolution block.

[0022] Figure 2 It is a structural diagram of the parallel spatial channel reconstruction convolution PSCConv of the present invention, wherein a is the overall structure, b is the spatial reconstruction unit, and c is the channel reconstruction unit.

[0023] Figure 3 These are the obstacle detection results of the present invention under different environments.

[0024] Figure 4 This is a flow chart of a method for detecting urban rail obstacles based on deep learning provided by the present invention.

[0025] Figure 5 It is a schematic diagram of the steps of a method for detecting urban rail obstacles based on deep learning provided by the present invention. DETAILED DESCRIPTION

[0026] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0027] See also Figures 1 to 5 The present invention provides a method for detecting urban rail obstacles based on deep learning, comprising the following steps:

[0028] S1 uses on-board industrial cameras to record the track conditions of trains in real time and establish a rail transit obstacle dataset;

[0029] Specifically, the on-board industrial camera is used to record the track conditions of the train in real time, key frame images containing obstacles are extracted from the collected video, and the image samples are saved to establish a rail transit obstacle dataset.

[0030] S2 uses a tool to mark obstacles in the data set into six categories, and divides the image data into a training set and a test set;

[0031] Specifically, the label-making tool labelme was used to label the obstacle image dataset using a strict polygonal labeling method in six categories (people, traffic lights, platforms, distribution boxes, billboards, and small obstacles). The labeled obstacle data information includes the pixel coordinates of the bounding box, the obstacle category, and the saving path, and is saved in the specified path in json format. According to a specific protocol, the obstacle image data and annotation information are stored in coco format and divided into training sets and test sets. The image number and obstacle annotation information are then converted to txt format and stored in the specified path to complete the dataset creation.

[0032] S3 performs data enhancement on some images in the training set and the test set;

[0033] Specifically, data enhancement is performed on the training set and test set, including flipping, rotation, and partial cropping, and Gaussian noise with a mean of 0 and a standard deviation of 25 is added to some data to improve the robustness of the model. By adding Gaussian noise, the data received by the sensors of the train when the line of sight is obstructed is simulated, thereby improving the generalization ability of the data set.

[0034] S4 constructs an urban rail obstacle detection model and trains the urban rail obstacle detection model.

[0035] Specifically, we build an urban rail transit obstacle detection model, LDCD-Net. This model uses a lighter convolution method, PSCConv, to extract obstacle features, effectively reducing the complexity of the model; the RSCIoU loss function is used for bounding box regression, which effectively expresses the relative angle position information between the real box and the predicted box, improving the training effect of the model.

[0036] The LDCD-Net detection model is trained with the training set to obtain the model weight file after the training is completed, and then the detection effect of the model is evaluated with the test set.

[0037] The LDCD-Net network structure is as follows Figure 1 As shown in the figure, the feature extraction network adopts the CSPDarkNet network structure. First, the input features are downsampled once through convolution, normalization, and SiLu activation function (convolution block) to reduce the subsequent calculation amount of the model. Then, a four-layer feature compilation module is used to extract feature information at four scales in the features, and the three-layer features with the smallest scale are input into the feature fusion network. The structure of the feature compilation module is shown in the figure. Figure 1 As shown in (d), for the input feature X∈R c ×h×w , the overall calculation method of the feature compilation module is:

[0038]

[0039] First, the resolution is reduced by 2 times through two layers of convolution blocks (CBS), and the number of channels is doubled. The first layer of convolution blocks uses 3×3 convolution to reduce its resolution, and the second layer of convolution blocks uses 1×1 convolution to increase the number of channels. Separation Use lightweight convolutional blocks (LWCBS) to extract features from X1 and concatenate it with X2. Finally, use a layer of 1×1 convolutional blocks to fuse the concatenated features. In the feature extraction network, in order to achieve the fusion of local features and global features at the featureMap level, the spatial pyramid pooling module is used to fuse the features of the lowest resolution layer. The obtained feature map continues to be forward propagated. The final output feature map sizes are 160×160, 80×80, 40×40, and 20×20 from top to bottom, and the features of these three layers are passed into the feature fusion network. The feature fusion network uses PAN-FPN, such as Figure 1 As shown in (b), the three layers of features with the lowest resolution that are input into the feature fusion network are fused at multiple scales. Figure 1 As shown in (c), a decoupled structure is adopted, through L BCE and L RSCIoU Implement the classification and bounding box regression algorithms.

[0040] In the present invention, the lightweight convolution PSCConv is used in the four-layer feature processing module in the LDCD-Net feature extraction network, and its structure is as follows: Figure 2 As shown in (a), the input feature X∈R c×h×w Separated into two features X1, X2∈R along the channel direction 1 / 2c×h×w , X1 and X2 are refined through channels and space to form spatial refinement features X S and channel refinement feature X C , and then X S , X C Reconnect and finally use the residual structure to connect the input and output features to achieve channel space refinement. The specific calculation method is:

[0041]

[0042] The SRU and CRU are the space reconstruction unit and channel reconstruction unit, respectively, and their structures are as follows: Figure 2 As shown in (b) and (c), SRU takes the input feature X∈R c×h×w After normalization, weight reconstruction is performed, and spatial separation is achieved with a threshold of 0.5 to obtain information-rich features X W1 With less informative feature X W2 . X W2 Considered as redundant features, the two sets of weighted features are fully fused through cross-reconstruction operations to obtain the spatially refined feature X S . CRU takes the input feature X∈R c×h×w Along the channel direction, the flow is divided into two features X with a split ratio of 0≤α≤1 C1 ∈R αc×h×w , X C2 ∈R(1-α)c×h×w , in order to improve the computational efficiency, C1 and X C2 Use 1×1 convolution to further compress the channel by a factor of 2 to obtain X′ C1 ∈R 1 / 2αc×h×w and X′ C2 ∈R 1 / 2(1-α)c×h×w , and then the transformation fusion strategy is used to obtain the channel refinement feature X C ∈R c×h×w .

[0043] In the present invention, the loss function of bounding box regression in LDCD-Net is shown in Table 1. For a given prediction box and real frame RSCIoU considers three dimensions: overlap S, relative position relationship D, and shape difference V. The overlap calculation method uses the traditional IoU calculation method, and the relative position calculation method uses the sum of the squares of the distances between the upper left and lower right points of the real box and the predicted box to represent the relative position between the two bounding boxes. At the same time, in order to prevent the influence of the bounding box shape on the distance representation, the area of ​​the minimum covering box is used to normalize the sum of the distances between the four points. Using B gt and B prd The difference in aspect ratio is used to represent the shape difference between the real bounding box and the predicted bounding box. At the same time, to ensure that L RSCIoU The calculation can comprehensively consider the overlap, distance, and shape difference, and the expression of shape difference is also normalized.

[0044] Table 1 RSCIoU calculation method

[0045]

[0046] In the present invention, comprehensive evaluation indicators are set to evaluate the model effect: mAP (mean average precision) is an indicator used to measure the accuracy of the target detection model, which combines the accuracy and recall of the detection model; FLOPs (Floating Point Operations Per Second) is an indicator used to measure the number of floating-point operations performed per second by the model during the inference phase; Params refers to the total number of weights and biases that need to be learned in the model. Smaller Params means lower model complexity and lower deployment difficulty. Times represents the time required to detect a single frame of an image. This indicator is used to measure the detection speed of the algorithm to ensure the real-time detection.

[0047] Figure 3The following are the results of obstacle detection. It can be seen that in various environments, LDCD-Net has a good detection effect on different types of obstacles and the detection algorithm is also more robust in complex environments.

[0048] Table 2 Comparison of test results

[0049]

[0050] The comparison between LDCD-Net and mainstream obstacle detection models is shown in Table 1. Compared with the baseline model YOLOv8s, the model calculation parameters are reduced by 15.5% and the calculation amount is reduced by 16.2%. The obstacle detection mAP is increased by 1%, reaching the highest value of the current obstacle detection algorithm, and has good application prospects in rail transit obstacle detection.

[0051] The above disclosure is only a preferred embodiment of the urban rail obstacle detection method based on deep learning of the present invention. Of course, it cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A method for detecting obstacles on urban rails based on deep learning, characterized in that: The following steps are involved: Use on-board industrial cameras to record the track conditions of trains in real time and establish a rail transit obstacle dataset; The obstacles in the dataset are labeled into six categories through a tool, and the image data is divided into a training set and a test set; Performing data augmentation on some images in the training set and the test set; An urban rail obstacle detection model is constructed and trained.

2. The urban rail obstacle detection method based on deep learning according to claim 1, characterized in that: The specific method of using the on-board industrial camera to record the train track conditions in real time and establish a rail transit obstacle dataset is as follows: When the train is running, the on-board industrial camera records the track conditions of the train in real time to obtain the collected video; Extract key frame images of obstacles contained in the captured video and save image samples; A rail transit obstacle dataset is established based on the image samples.

3. The urban rail obstacle detection method based on deep learning as claimed in claim 1, characterized in that: The six categories include people, traffic lights, platforms, switchboards, billboards and small obstacles.

4. The urban rail obstacle detection method based on deep learning according to claim 1, characterized in that: The data enhancement includes flipping, rotation and local cropping.

5. The urban rail obstacle detection method based on deep learning according to claim 1, characterized in that: The specific method of training the urban rail obstacle detection model is: The evaluation indicators such as average detection accuracy, single-frame image detection time, model parameter quantity and floating-point number calculation quantity are set to evaluate the model effect.

Images