Trackside obstacle detection method, device and storage medium
By combining residual neural network models and lidar sensors, the problems of high computational load and low accuracy in obstacle detection in rail transit have been solved, achieving efficient and accurate track and obstacle identification and improving train safety.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BYD CO LTD
- Filing Date
- 2021-11-30
- Publication Date
- 2026-06-05
Smart Images
Figure CN116203581B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of rail transit technology, and more specifically to a method, apparatus and storage medium for detecting trackside obstacles. Background Technology
[0002] With the continuous development of smart city and transportation concepts in my country, automated rail transit trains have become a major mode of public transportation. Although rail transit safety regulations are gradually improving, train safety issues remain a significant concern. The primary cause of train accidents is the collision between the train and obstacles that suddenly intrude above the track or fall from tunnel structures. Therefore, how to quickly and accurately detect trackside obstacles using trackside equipment is both a key focus and a challenge in preventing traffic accidents.
[0003] Trackside obstacle detection involves two aspects. First, track identification is performed to determine the detection area. Currently, the commonly used track identification method is edge detection to identify the track and then use the entire track as the detection area. However, this method increases the computational load for subsequent obstacle detection. Furthermore, this method is easily affected by factors such as lighting and occlusion, resulting in inconsistent detection results under different environments and insufficient accuracy. Second, obstacle detection is performed within the determined detection area. Traditional trackside obstacle detection methods mainly rely on manual detection or locomotive detection. However, both of these methods consume significant manpower and resources and have low detection efficiency.
[0004] Therefore, improvements are needed to address the aforementioned issues. Summary of the Invention
[0005] This application is proposed to address the aforementioned problems. According to one aspect of this application, a trackside obstacle detection method is provided, the method comprising:
[0006] Obtain the orbital image dataset;
[0007] The first orbital image in the orbital image dataset is analyzed based on a pre-built residual neural network model to identify and label the orbital regions in the first orbital image, thereby obtaining the second orbital image.
[0008] Based on the target detection algorithm, the first obstacle present in the track area is detected;
[0009] The lidar sensor is controlled to detect the actual track area and the inter-occipital area to detect a second obstacle, wherein the actual track area corresponds to the track area.
[0010] In one embodiment of this application, constructing a residual neural network model includes:
[0011] Obtain track image samples with the track area already marked;
[0012] Based on the orbital image samples, a residual neural network model is trained using a deep learning algorithm.
[0013] In one embodiment of this application, after analyzing a first orbital image in the orbital image dataset according to a pre-built residual neural network model, identifying and labeling the orbital regions in the first orbital image, and obtaining a second orbital image, the process includes:
[0014] The orbital region was re-identified using the dilated spatial convolutional pooling pyramid algorithm.
[0015] In one embodiment of this application, the orbital region is re-identified using the dilated spatial convolutional pooling pyramid algorithm, including:
[0016] The first or second track image is re-identified using a pre-sized dilated convolutional layer and a pooling layer.
[0017] Obtain the features corresponding to all channels of the residual neural network model;
[0018] By reducing the number of channels in the residual neural network model using a convolution kernel of a preset size, all features are combined to obtain the track region for re-identification.
[0019] In one embodiment of this application, after analyzing a first orbital image in the orbital image dataset according to a pre-built residual neural network model, identifying and labeling the orbital regions in the first orbital image, and obtaining a second orbital image, the method further includes:
[0020] The orbital region identified by the residual neural network model is post-processed using dense connection network algorithm and conditional random field algorithm.
[0021] In one embodiment of this application, a deep fully connected conditional random field is used to post-process the orbital region identified by the residual neural network model, including:
[0022] Relationships are established for each pixel in the orbit region identified by the residual neural network model to construct a fully connected pixel distribution map;
[0023] Calculate the similarity between each pixel to determine whether each pixel belongs to the same category.
[0024] In one embodiment of this application, the residual neural network model includes five network layers, and each network layer includes an input channel and an output channel;
[0025] The first orbital image in the orbital image dataset is analyzed based on a pre-built residual neural network model to identify and label the orbital regions in the first orbital image, resulting in a second orbital image, including:
[0026] When the number of input and output channels in each network layer is the same, post-processing is performed on the orbital region identified by the residual neural network model.
[0027] When the number of input and output channels in each network layer is different, the number and step size of the input channels in each network layer are adjusted so that the number of input and output channels in each network layer is the same.
[0028] Each of the network layers mentioned herein contains 10 network layers.
[0029] In one embodiment of this application, detecting a first obstacle in the track region based on a target detection algorithm includes:
[0030] The second track image is preprocessed to preliminarily identify whether a first obstacle exists in the track region;
[0031] When the initial identification results show that there are obstacles in the track area, the lidar sensor detects a second obstacle in the actual track area and the inter-pillow area;
[0032] When the preliminary identification results show that there are no obstacles in the track area, a target detection model is constructed based on the second track image. The second track image is then input into the target detection model to obtain the detection result of the target detection model, wherein the detection result of the target detection model indicates whether there is a first obstacle in the track area.
[0033] In one embodiment of this application, preprocessing of the second orbital image includes:
[0034] Obtain the histogram of the orbital images in the orbital image dataset;
[0035] Detect whether there are changes in the histogram within a preset interval;
[0036] If there is a change, then it is determined that a first obstacle exists in the track region.
[0037] In one embodiment of this application, a target detection model is constructed based on the second orbital image, including:
[0038] The residual neural network model and the dilated spatial convolutional pooling pyramid algorithm are used to extract and label the features of the second track image, and then the target detection model is constructed based on the second track image with labeled features.
[0039] In one embodiment of this application, before constructing the target detection model based on the second orbital image, the method further includes:
[0040] The size of the track region marked in the second track image is expanded based on the grid method.
[0041] In one embodiment of this application, the size of the second track image is consistent with the size of the track images in the track image dataset.
[0042] In one embodiment of this application, after inputting the second orbital image into the target detection model and obtaining the detection result of the target detection model, the process includes:
[0043] Label the types of targets detected by the target detection model.
[0044] In one embodiment of this application, the target type includes at least one of the following: rails, sleepers, people, and first obstacles.
[0045] In one embodiment of this application, the second track image includes a track mask map with the track regions already labeled.
[0046] In one embodiment of this application, a monitoring device for real-time acquisition of track images is provided at a preset position next to the track;
[0047] Acquiring a track image dataset includes: receiving at least one track image sent by the monitoring device in real time, wherein the at least one track image constitutes the track image dataset.
[0048] In one embodiment of this application, the second obstacle includes a third obstacle obtained after the lidar sensor confirms the first obstacle, and a fourth obstacle present in the interoccipital region.
[0049] In one embodiment of this application, after controlling the lidar sensor to detect the actual track area and the inter-pillow area to detect the second obstacle, the process includes:
[0050] The second obstacle is sent to the train's central control system, which then sends an instruction to the train to adjust its driving status.
[0051] In one embodiment of this application, the train driving state includes at least one of the following states: constant speed driving state, acceleration driving state, deceleration driving state, and stop driving state.
[0052] According to another aspect of this application, a trackside obstacle detection device is provided, the device comprising:
[0053] The system includes a memory and a processor, wherein the memory stores a computer program that is executed by the processor, and the computer program, when executed by the processor, causes the processor to perform the aforementioned trackside obstacle detection method.
[0054] According to another aspect of this application, a storage medium is provided, on which a computer program is stored, which, when run by a processor, causes the processor to execute the above-described trackside obstacle detection method.
[0055] The trackside obstacle detection method, apparatus, and storage medium of this application first analyze the first track image in the track image dataset using a residual neural network model to identify the track region in the first track image. Then, based on a target detection algorithm, it detects the first obstacle present in the track region. Finally, it uses a lidar sensor to detect the actual track region (corresponding to the track region) and the sleeper area. This application only identifies the track region and sleeper area, rather than the entire track, reducing the detection area and improving the detection speed. It also improves the accuracy of track identification when the identification is not affected by environmental factors. Attached Figure Description
[0056] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0057] Figure 1 A schematic flowchart illustrating a trackside obstacle detection method according to an embodiment of this application is shown.
[0058] Figure 2 This diagram illustrates the location of a monitoring device according to an embodiment of this application.
[0059] Figure 3 This illustration shows a structural schematic of a residual neural network model according to an embodiment of this application;
[0060] Figure 4 This illustration shows a structural schematic of a hollow spatial convolutional pooling pyramid model according to an embodiment of this application;
[0061] Figure 5 This illustration shows a structural schematic of a target detection model according to an embodiment of the present application;
[0062] Figure 6 This illustration shows an example of track region expansion according to an embodiment of this application;
[0063] Figure 7 A schematic block diagram of a trackside obstacle detection device according to an embodiment of this application is shown. Detailed Implementation
[0064] To make the objectives, technical solutions, and advantages of this application more apparent, exemplary embodiments according to this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein. Based on the embodiments of this application described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of this application.
[0065] To address the aforementioned technical problems, this application provides a method for detecting trackside obstacles. The method involves acquiring a track image dataset; analyzing a first track image in the dataset using a pre-built residual neural network model to identify and label track regions within the first track image, thus obtaining a second track image; detecting a first obstacle within the track region using a target detection algorithm; and controlling a lidar sensor to detect the actual track region and the inter-track region to detect a second obstacle, wherein the actual track region corresponds to the first track region. This method can more comprehensively and accurately detect obstacles in the actual track region and the inter-track region.
[0066] The following describes in detail the trackside obstacle detection method according to embodiments of this application, with reference to the accompanying drawings. Unless otherwise specified, features of the various embodiments of this application can be combined with each other.
[0067] Figure 1 A schematic flowchart of a trackside obstacle detection method according to an embodiment of this application is shown; as follows: Figure 1 As shown, the trackside obstacle detection method 100 according to an embodiment of this application may include the following steps S101, S102, S103 and S104:
[0068] In step S101, the orbital image dataset is obtained.
[0069] In embodiments of this application, acquiring a track image dataset includes: receiving at least one track image sent by the monitoring device in real time, wherein the at least one track image constitutes the track image dataset. The track images are track images of different logical segments of the track captured in real time by the monitoring device, and these track images of different logical segments constitute the track image dataset. Embodiments of this application can analyze the real-time acquired track images in the track image dataset based on residual grid algorithms and target detection algorithms to determine whether there are obstacles in the corresponding track region of the logical segment.
[0070] In the embodiments of this application, such as Figure 2 As shown, monitoring equipment for real-time acquisition of track images is installed at predetermined locations beside the track. This monitoring equipment can be a camera, video camera, or other device with capturing capabilities. The monitoring equipment acquires track images in real time and transmits these images to the train's ground control system. The monitoring equipment can be deployed in different logical sections of the track.
[0071] In step S102, the first track image in the track image dataset is analyzed according to the pre-built residual neural network model, the track region in the first track image is identified and labeled, and the second track image is obtained.
[0072] In one example, constructing a residual neural network model includes: A1, obtaining track image samples with labeled track regions; A2, training a residual neural network model (ResNet) based on the track image samples using a deep learning algorithm.
[0073] In one example, the residual neural network model includes five stages, and each stage includes input and output channels. Accordingly, the first track image in the track image dataset is analyzed based on the pre-built residual neural network model to identify and label the track regions in the first track image, resulting in a second track image. This includes: D1, when the number of input and output channels in each stage is the same, post-processing is performed on the track regions identified by the residual neural network model; D2, when the number of input and output channels in each stage is different, the number of input channels and the step size of each stage are adjusted to ensure that the number of input and output channels in each stage is the same; D3, wherein each stage contains 10 network layers.
[0074] like Figure 3The diagram shows a schematic of the constructed residual neural network model. The residual neural network model constructed in this embodiment includes 50 network layers (i.e., ResNet50). First, the ResNet50 network structure is constructed using traditional deep learning algorithms. The ResNet50 backbone network is divided into 5 stages. In the diagram, (3,224,224) refers to the number of input channels, height, and width, i.e., (C,H,W). CONV represents a convolutional layer, MAXPOOL represents a max pooling layer, BN represents Batch Normalization, and RELU represents the ReLU activation function. Depending on whether the number of input channels and output channels are the same, there are two cases: First, when the number of input channels and output channels are different, the number of input channels and stride need to be adjusted to make them the same; second, when the number of input channels and output channels are the same, post-processing is performed directly on the feature map generated after extracting image features from the ResNet50 convolutional kernels. ResNet improves information flow due to its skip connection feature, effectively extracting image features and solving the degradation problem of deep networks.
[0075] In this embodiment, when acquiring track image samples with labeled track regions, one can use track image samples with labeled track regions from other models, or manually label a portion of the track images. These labeled track image samples are then used to train a residual neural network model. The residual neural network model is then used to label the track images in unlabeled track regions. In practice, since the track image dataset contains a large number of track images, manual labeling is not only costly in terms of manpower and resources, but also lacks sufficient accuracy. This application utilizes existing track image samples with labeled track regions to train the residual neural network model, thus saving manpower and resources.
[0076] In one example, after analyzing the first track image in the track image dataset according to a pre-built residual neural network model, identifying and labeling the track regions in the first track image, and obtaining the second track image, the process includes: using the Atrous Spatial Pyramid Pooling (ASPP) algorithm to identify the track regions again.
[0077] Specifically, the ASPP algorithm structure diagram shows that the dilated spatial convolution pooling pyramid algorithm is used to re-identify the track region, including: B1, using dilated convolutional layers and pooling layers of preset size to re-identify the first track image or the second track image; B2, obtaining the features corresponding to all channels of the residual neural network model; B3, reducing the number of channels of the residual neural network model by using convolutional kernels of preset size, so that all features are combined to obtain the re-identified track region.
[0078] When using the ASPP algorithm to re-identify the first or second track image, multiple dilated convolutional layers with different receptive fields can be employed to extract features from the given input image at multiple sizes. In a specific example, such as... Figure 4 As shown, the ASPP algorithm uses four dilated convolutional layers of different sizes in parallel, then uses pooling layers to process the feature map, combining the features extracted from each channel, and finally reduces the number of channels through a convolutional layer with a kernel size of 1. The result is used as the final feature.
[0079] In this embodiment of the application, the track region can be identified again after the residual neural network model has identified the track region, or after the track region has been labeled and the second track image has been obtained, so as to improve the accuracy of the identified track region.
[0080] In other examples, after analyzing the first orbital image in the orbital image dataset according to a pre-built residual neural network model, identifying and labeling the orbital regions in the first orbital image, and obtaining the second orbital image, the method further includes: using a deep fully connected conditional random field (Dense CRF) algorithm to post-process the orbital regions identified by the residual neural network model.
[0081] This application performs post-processing on the second track image to more accurately identify the track region. Specifically, a deep fully connected conditional random field is used to post-process the track region identified by the residual neural network model, including: C1, establishing relationships between pixels in the track region identified by the residual neural network model to construct a fully connected pixel distribution map; C2, calculating the similarity between the pixels to determine whether the pixels belong to the same category.
[0082] In this embodiment, after ResNet50 is trained, it can roughly identify the track area and mark the positions of the track and sleepers. However, the image at this time is not a segmentation network, and the boundary contours are not clear. Therefore, Dense CRF is introduced to post-process the image to improve the segmentation effect of the track and sleeper boundaries. When processing the image using the Dense CRF algorithm, relationships are established between each pixel in the image to form a fully connected pixel distribution map. This method fully considers the contextual relationships between pixels and determines whether they belong to the same category by calculating the similarity probability between pixels, further optimizing the coarse labeling of ResNet50, making the boundary segmentation of the track and sleepers clearer. The energy function formula of Dense CRF is as follows:
[0083] E[i]=∑ i θ i (x i )+∑ ij θ ij (x i ,x j )
[0084] Where i represents a pixel, x i The label representing pixel i, θ i (x i ) represents a unary potential, used to determine the category of pixel i; θ ij (x i ,x j ) represents a binary potential, used to describe the relationship between pixels.
[0085] Where, θ ij (x i ,x j The calculation formula for ) is as follows:
[0086] θ i (x i )=-lgP(x i )
[0087]
[0088] When processing image pixels using Dense CRF, an iterative algorithm can be used. When the energy function value is minimized, the final trapezoidal profile of the track and sleepers can be obtained.
[0089] In this embodiment, ResNet, Dense, and CRF are used to perform semantic segmentation on the track image to generate a trapezoidal contour of the track and sleepers. A schematic diagram of the trapezoidal contour is shown below. Figure 2As shown in the figure, the gray-black area represents the track region to be delineated. Compared with edge detection methods, this method, using a residual network to identify tracks, is unaffected by environmental factors and adaptively optimizes the identification model. Furthermore, by finely dividing the track region to reduce irrelevant areas, the computational area and computational load for downstream obstacle detection tasks are reduced, thus improving the overall computational speed.
[0090] In this embodiment of the application, the track image is processed by the ResNet50 model and the ASPP and Dense CRF algorithms to finally obtain a track mask image with the track region identified and labeled, which is the second track image.
[0091] In step S103, based on the target detection algorithm, a first obstacle is detected in the track area.
[0092] In one example, based on a target detection algorithm, detecting a first obstacle in the track region includes: E1, preprocessing the second track image to initially identify whether a first obstacle exists in the track region; E2, when the initial identification result shows that an obstacle exists in the track region, a lidar sensor detects a second obstacle in the actual track region and the inter-occipital region; E3, when the initial identification result shows that no obstacle exists in the track region, constructing a target detection model (YOLO v3) based on the second track image, inputting the second track image into the target detection model, and obtaining the detection result of the target detection model, wherein the detection result of the target detection model indicates whether a first obstacle exists in the track region.
[0093] In this embodiment of the application, when detecting a target, the second orbital image is first preprocessed. If a first obstacle is detected in the image during preprocessing, a lidar sensor is used to detect the first obstacle to confirm it.
[0094] If the first obstacle is not detected in the image during preprocessing, then the target detection model is used to detect whether the first obstacle exists in the track region.
[0095] In one example, the second track image is preprocessed, including: F1, obtaining the histogram of the track image in the track image dataset; F2, detecting whether there is a change in the histogram within a preset interval; F3, if there is a change, determining that there is a first obstacle in the track region.
[0096] In one example, constructing a target detection model based on the second track image includes: using the residual neural network model and the dilated spatial convolutional pooling pyramid algorithm to extract and label the features of the second track image, and then constructing the target detection model based on the second track image with labeled features.
[0097] like Figure 5 The diagram shown is a structural schematic of the target detection model according to an embodiment of this application. In this step, the backbone network extracts features from the second track image to identify targets in the second track image, such as tracks, sleepers, people, obstacles, etc. In this embodiment, the YOLO v3 backbone network can reuse the ResNet50 and ASPP constructed in step S102, because the ResNet50 trained in step S102 is more sensitive to track data, and ASPP can increase the detail of multi-scale features in the feature map.
[0098] In one example, before constructing the object detection model based on the second orbit image, the method further includes: expanding the size of the marked orbit region in the second orbit image based on a grid method. Figure 6 As shown in the embodiment of this application, the track region in the image can be expanded before constructing the target detection model to avoid missing the first obstacle. Figure 6 In the diagram, w is the final dimension extended inwards and outwards from the track and sleepers based on the second track image, and ABCD are the four corner points of the search area contour that YOLO v3 can process.
[0099] In practice, regions ABCD are divided according to a w*w grid, and the corresponding grid images are classified and labeled according to the target type, which is equivalent to different anchor points.
[0100] In one example, after inputting the second orbital image into the target detection model and obtaining the detection result of the target detection model, the process includes: labeling the target type detected by the target detection model.
[0101] In one example, the target type includes at least one of the following: rails, sleepers, people, and a first obstacle. The first obstacle may include obstacles such as stones on the rails or stones under the rails. In the target detection model, targets can be labeled according to these types. After labeling, it can be determined whether a first obstacle exists and what type of first obstacle it is.
[0102] In one example, the size of the second orbital image is consistent with the size of the orbital images in the orbital image dataset. This embodiment directly uses the original orbital images without adjusting their sizes to reduce computational load.
[0103] In this embodiment, YOLO v3 uses dilated convolution to increase the receptive field of the convolution, resulting in higher accuracy in identifying classified objects. While reducing the number of classification categories and image retrieval areas, it ensures detection accuracy while improving detection speed.
[0104] In step S104, the lidar sensor is controlled to detect the actual track area and the inter-pillow area to detect the second obstacle, wherein the actual track area corresponds to the track area.
[0105] In one example, the second obstacle includes a third obstacle identified by the lidar sensor after confirming the first obstacle, and a fourth obstacle present in the interoccipital region.
[0106] In this embodiment of the application, when using a lidar sensor for obstacle detection, the lidar sensor can confirm the first obstacle to improve detection accuracy, and can also supplement the detection of the sleeper area, and can detect obstacles not detected by the visual sensor as well as small obstacles between sleepers.
[0107] The lidar sensor can be positioned beside the track, combined with... Figure 2 A column is installed next to the track, with a camera at the top of the column to capture the first track image. At the bottom of the column, a lidar sensor is installed to detect second obstacles in the actual track area and the area between the sleepers.
[0108] In addition, in this embodiment of the application, after controlling the lidar sensor to detect the actual track area and the sleeper area to detect the second obstacle, the process includes: sending the second obstacle to the train's central control system, so that the central control system can send an instruction to the train to adjust the train's driving state.
[0109] The train driving state includes at least one of the following states: constant speed driving state, acceleration driving state, deceleration driving state, and stop driving state.
[0110] In a specific example, V2I technology can be used to enable information interaction between trackside detection equipment and the train's ground control system. The information of the second obstacle can be transmitted to the train's ground control system, which can then send instructions to the train control system to control the train, such as deceleration or stopping.
[0111] The embodiments of this application have the following advantages:
[0112] 1. Compared with traditional trackside obstacle detection methods, this application only identifies the track area and sleeper area, rather than the entire track, which reduces the detection area and improves the detection speed. It also improves the accuracy of track identification when the identification is not affected by environmental factors.
[0113] 2. Compared with traditional trackside obstacle detection methods, this application adopts the YOLO v3 algorithm, which improves detection accuracy by increasing the receptive field and preprocessing. At the same time, it can also improve detection speed by not adjusting the size before input during training. When there is obstacle information, it can detect it in time and send the obstacle information to the ground control system, thereby improving the interaction efficiency with the ground control system.
[0114] The following is combined Figure 7 The trackside obstacle detection device of this application is described, wherein, Figure 7 A schematic block diagram of a trackside obstacle detection device according to an embodiment of this application is shown.
[0115] like Figure 7 As shown, the trackside obstacle detection device 700 includes: one or more memories 701 and one or more processors 702. The memories 701 store a computer program that is executed by the processors 702. When the computer program is executed by the processors 702, the processors 702 perform the trackside obstacle detection method described above.
[0116] The trackside obstacle detection device 700 can be part or all of a computer device that can implement a trackside obstacle detection method through software, hardware, or a combination of software and hardware.
[0117] like Figure 7 As shown, the trackside obstacle detection device 700 includes one or more memories 701, one or more processors 702, a display (not shown), and a communication interface, etc., which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). It should be noted that... Figure 7 The components and structure of the trackside obstacle detection device 700 shown are exemplary and not limiting. The trackside obstacle detection device 700 may also have other components and structures as needed.
[0118] Memory 701 is used to store various data and executable program instructions generated during train operation, such as algorithms for storing various application programs or implementing various specific functions. It may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.
[0119] The processor 702 may be a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other processing units with data processing capabilities and / or instruction execution capabilities, and may be other components in the trackside obstacle detection device 700 to perform the desired functions.
[0120] In one example, the trackside obstacle detection device 700 also includes an output device that can output various information (such as images or sounds) to the outside (e.g., a user), and may include one or more of a display device, a speaker, etc.
[0121] The communication interface can be any known communication protocol interface, such as a wired interface or a wireless interface. The communication interface may include one or more serial ports, USB interfaces, Ethernet ports, WiFi, wired networks, DVI interfaces, device integrated interconnect modules, or other suitable ports, interfaces, or connections.
[0122] Furthermore, according to embodiments of this application, a storage medium is also provided, on which program instructions are stored. When the program instructions are executed by a computer or processor, they are used to perform corresponding steps of the trackside obstacle detection method of this application. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media.
[0123] The trackside obstacle detection device and storage medium of this application embodiment have the same advantages as the aforementioned method because they can implement the aforementioned method.
[0124] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.
[0125] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0126] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.
[0127] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0128] Similarly, it should be understood that, in order to streamline this application and aid in understanding one or more of the various inventive aspects, features of this application may sometimes be grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of this application. However, this approach should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with features fewer than all features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.
[0129] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.
[0130] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.
[0131] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules according to the embodiments of this application. This application can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
[0132] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.
[0133] The above description is merely a specific embodiment or illustration of the embodiments of this application. The scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. The scope of protection of this application shall be determined by the scope of the claims.
Claims
1. A method for detecting trackside obstacles, characterized in that, The method includes: Obtain the orbital image dataset; The first orbital image in the orbital image dataset is analyzed based on a pre-built residual neural network model to identify and label the orbital regions in the first orbital image, thereby obtaining the second orbital image. Based on the target detection algorithm, the first obstacle present in the track area is detected; The lidar sensor is controlled to detect the actual track area and the inter-pillow area to detect a second obstacle, wherein the actual track area corresponds to the track area; Based on the target detection algorithm, the first obstacle present in the track region is detected, including: The second track image is preprocessed to preliminarily identify whether a first obstacle exists in the track region; When the initial identification results show that there are obstacles in the track area, the lidar sensor detects a second obstacle in the actual track area and the inter-pillow area; When the preliminary identification results show that there are no obstacles in the track area, a target detection model is constructed based on the second track image. The second track image is then input into the target detection model to obtain the detection result of the target detection model, wherein the detection result of the target detection model indicates whether there is a first obstacle in the track area.
2. The method as described in claim 1, characterized in that, Constructing a residual neural network model includes: Obtain track image samples with the track area already marked; Based on the orbital image samples, a residual neural network model is trained using a deep learning algorithm.
3. The method as described in claim 1, characterized in that, After analyzing the first orbital image in the orbital image dataset according to the pre-built residual neural network model, identifying and labeling the orbital regions in the first orbital image, and obtaining the second orbital image, the process includes: The orbital region was re-identified using the dilated spatial convolutional pooling pyramid algorithm.
4. The method as described in claim 3, characterized in that, The orbital region is re-identified using the dilated spatial convolutional pooling pyramid algorithm, including: The first or second track image is re-identified using a pre-sized dilated convolutional layer and a pooling layer. Obtain the features corresponding to all channels of the residual neural network model; By reducing the number of channels in the residual neural network model using a convolution kernel of a preset size, all features are combined to obtain the track region for re-identification.
5. The method as described in claim 1, characterized in that, After analyzing the first orbital image in the orbital image dataset according to a pre-built residual neural network model, identifying and labeling the orbital regions in the first orbital image, and obtaining the second orbital image, the method further includes: The orbital region identified by the residual neural network model is post-processed using dense connection network algorithm and conditional random field algorithm.
6. The method as described in claim 5, characterized in that, A deep fully connected conditional random field is used to post-process the orbital region identified by the residual neural network model, including: Relationships are established among the pixels in the orbit region identified by the residual neural network model to construct a fully connected pixel distribution map; Calculate the similarity between each pixel to determine whether each pixel belongs to the same category.
7. The method as described in claim 1, characterized in that, The residual neural network model includes five network layers, and each network layer includes an input channel and an output channel; The first orbital image in the orbital image dataset is analyzed based on a pre-built residual neural network model to identify and label the orbital regions in the first orbital image, resulting in a second orbital image, including: When the number of input and output channels in each network layer is the same, post-processing is performed on the orbital region identified by the residual neural network model. When the number of input and output channels in each network layer is different, the number and step size of the input channels in each network layer are adjusted so that the number of input and output channels in each network layer is the same. Each of the network layers mentioned herein contains 10 network layers.
8. The method as described in claim 1, characterized in that, Preprocessing the second orbital image includes: Obtain the histogram of the orbital images in the orbital image dataset; Detect whether there are changes in the histogram within a preset interval; If there is a change, then it is determined that a first obstacle exists in the track region.
9. The method as described in claim 1, characterized in that, Based on the second orbital image, a target detection model is constructed, including: The residual neural network model and the dilated spatial convolutional pooling pyramid algorithm are used to extract and label the features of the second track image, and then the target detection model is constructed based on the second track image with labeled features.
10. The method as described in claim 1, characterized in that, Before constructing the target detection model based on the second orbital image, the method further includes: The size of the track region marked in the second track image is expanded based on the grid method.
11. The method as described in claim 1, characterized in that, The size of the second track image is consistent with the size of the track images in the track image dataset.
12. The method as described in claim 1, characterized in that, After inputting the second track image into the target detection model and obtaining the detection result of the target detection model, the process includes: Label the types of targets detected by the target detection model.
13. The method as described in claim 12, characterized in that, The target type includes at least one of the following: rails, sleepers, people, and first obstacles.
14. The method as described in claim 1, characterized in that, The second track image includes a track mask map with the track regions already labeled.
15. The method as described in claim 1, characterized in that, in, Monitoring equipment for real-time acquisition of track images is installed at a predetermined location next to the track; Acquiring a track image dataset includes: receiving at least one track image sent by the monitoring device in real time, wherein the at least one track image constitutes the track image dataset.
16. The method as described in claim 1, characterized in that, The second obstacle includes a third obstacle obtained after the lidar sensor confirms the first obstacle, and a fourth obstacle present in the interoccipital region.
17. The method as described in claim 16, characterized in that, After controlling the lidar sensor to detect the actual track area and the inter-cushion area to detect the second obstacle, the following steps are included: The second obstacle is sent to the train's central control system, so that the central control system can send instructions to the train to adjust the train's driving status based on the second obstacle.
18. The method as described in claim 17, characterized in that, in, The train driving state includes at least one of the following states: constant speed driving state, acceleration driving state, deceleration driving state, and stop driving state.
19. A trackside obstacle detection device, characterized in that, The device includes: A memory and a processor, wherein the memory stores a computer program that is executed by the processor, the computer program, when executed by the processor, causes the processor to perform the trackside obstacle detection method as described in any one of claims 1 to 18.
20. A storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, causes the processor to perform the trackside obstacle detection method as described in any one of claims 1 to 18.