A method for realizing a track virtual balise

Abstract

The present invention provides a method for implementing a track virtual transponder. The method includes: building an on-board image acquisition and processing system arranged at the end of the train, using the on-board image acquisition and processing system to collect a set of driving record images of the front view and the side view during the operation of the train; using an improved YOLOv3 network to perform real-time target detection on the driving record image set, and obtain the category and position information of existing markers on the train's track operation line; based on the improved CRNN network, the driving record image set is subjected to content recognition according to the position information of the existing markers, and the text information of the existing markers on the train's track operation line is obtained. The present invention uses a machine vision method to detect the position and identify the content of existing markers on the line, which can be used for response positioning during the operation of rail transit vehicles, completely getting rid of the dependence on ground transponders and satellites, and realizing completely autonomous train response positioning.

Description

Technical Field

[0001] The present invention relates to the field of rail transit communication technology, and in particular to a method for realizing a track virtual transponder. Background Art

[0002] The 13th Five-Year Plan period is a critical period for the rapid advancement and development of my country's railway network construction. By the end of 2020, the total operating mileage of my country's railways has reached 146,300 kilometers. With the continuous expansion of the railway network, the operating mileage and passenger volume are also increasing, and the running speed and driving density of trains are also constantly improving, which puts higher requirements on driving safety risk control and intelligent line maintenance and inspection.

[0003] At present, mobile block is widely used at home and abroad to ensure the efficient and safe operation of railway lines, and mobile block has high requirements for train operation control. The train operation control system is the nerve center of the rail transit system, and the precise positioning of the train is the key technical basis of train operation control. Providing accurate and reliable train position information is crucial to the entire system and is the prerequisite for ensuring the normal operation of the system.

[0004] The existing high-speed railway train positioning is mainly achieved by querying the transponder to correct the accumulated error of the odometer. The transponder is an important signal infrastructure and is widely used in foreign railways, subways, and urban light rail transportation. However, since the transponder needs to be laid on the line, the construction cost of this method is high and the daily maintenance is difficult. In addition, the feasibility of adding query transponders to a large number of existing lines is also poor. With the rapid development of the Beidou satellite navigation system independently developed by my country and the realization of global networking, its performance has been comparable to navigation systems such as GPS (Global Positioning System) and GLONASS. The improvement of the Beidou satellite navigation system provides a good solution for the precise and autonomous positioning of trains, but this method still has defects, that is, in special terrains such as valleys and long tunnels, satellite signals may be lost, that is, the satellite signal "blind spot" problem, which is the main defect of this solution.

[0005] Therefore, it is necessary to develop a reliable train autonomous positioning system that can operate under extreme conditions, and the system must be able to achieve virtual response relying solely on the existing characteristics of the line.

[0006] A method for implementing a virtual transponder based on a GNSS (Global Navigation Satellite System) satellite navigation system in the prior art includes: applying the positioning technology supported by GNSS to the train control system, supplemented by map matching technology, to implement a virtual transponder, thereby reducing the demand for real transponders, and even replacing real transponders, thereby reducing construction and maintenance costs. Figure 1 This is a structural diagram of a virtual transponder based on a GNSS satellite navigation system in the prior art. The definition of a virtual transponder is as follows: the virtual transponder simulates a real transponder placed on the track. When the train runs to a reference point on the track, the virtual transponder sends a location information to the transponder transmission module (BTM). This information is exactly the same as the information sent by the real transponder on the track. The virtual transponder is composed of three modules: a positioning unit module, a safety judgment module, and a transponder (VB) module. The positioning unit module includes a GNSS receiver and a Kalman filter. After receiving the GPS signal, the positioning information of the receiver is subjected to Kalman filtering.

[0007] The safety determination module includes two modules: safety determination and map matching, which determine the location information transmitted by the positioning unit and match it with the map in the database to determine the positioning error.

[0008] The virtual transponder (VB) module includes a VB database, a VB capturer and a VB information generator. The VB database stores the location information of the virtual transponder. When the train is running on the track, the train location information is confirmed by the safety identification module and then passed to the VB capturer. The VB capturer performs capture calculations on the information and the location information in the VB database based on the speed of the previous train positioning through dynamic prediction. If the capture is successful, the VB information generator generates a message information compatible with the actual transponder and passes it to the transponder transmission module (BTM), which then transmits the information to the on-board computer for processing.

[0009] Virtual transponders are used to achieve real-time positioning. In order to achieve real-time and accurate positioning, it is necessary to capture the virtual transponder when the train passes by the point where the virtual transponder is located. Since the GPS receiver receives data at a certain frequency, the train cannot receive positioning information exactly at the virtual transponder point in most cases. Therefore, a certain capture area must be set with the virtual transponder point as the center.

[0010] The implementation method of the virtual transponder includes the following steps:

[0011] Step 1, determine the capture area of ​​the virtual transponder, define the capture area as a circle with the virtual transponder point as the center and r2 as the radius. The calculation formula of the capture radius r2 of the virtual transponder at the current moment is:

[0012] r2=(v0 / 2H+a / 4H2+ξ)q

[0013] Where: v0——current speed of the train, in m / s;

[0014] H——frequency, in Hz;

[0015] a——acceleration, in m / s2;

[0016] ξ——Correction value of capture radius, ξ<L, L is the distance between two positioning points;

[0017] q——Coefficient for controlling capture rate, 0<q≤1.

[0018] Step 2: The current point has not yet entered the capture area of ​​the virtual transponder, and the virtual transponder enters the pre-capture state. The current positioning result will not trigger the virtual transponder to give positioning information. The judgment conditions of the pre-capture state are: L≤L1 and L2≥r2; L1 is the distance from the previous point to the virtual transponder point, L2 is the distance from the current point to the virtual transponder point, and L is the distance between the two positioning points, that is, the previous point to the current point;

[0019] Step 3, the current point enters the capture area of ​​the virtual transponder, the virtual transponder is in the capture state, and the judgment condition of transponder capture is: L2<r2; if L>L1 and L2≥r2, it is a missed capture state;

[0020] Step 4: After the virtual transponder captures the position, it generates the same message as the actual transponder and transmits it to the transponder transmission module.

[0021] The disadvantages of the above-mentioned prior art implementation method of the virtual transponder based on the GNSS satellite navigation system are:

[0022] This solution uses the GNSS satellite positioning system as the source of positioning information, and constructs a virtual transponder system by installing a satellite signal receiving device on the train end to obtain satellite positioning information. Although this virtual transponder implementation solution can replace traditional ground transponders and other physical response devices, it provides a virtual location response point on the train route, reducing the installation and maintenance costs of ground equipment. However, the prerequisite for this virtual transponder based on the GNSS satellite system to provide a response is that the GNSS satellite system signal can be received at the virtual response location. The GNSS satellite positioning system has the problem of signal loss in shielded environments such as valleys, tunnels, under bridges, and underground sections. In this case, the virtual transponders in the existing technology cannot provide good location information, and the system will lose its corresponding function.

[0023] Therefore, the virtual transponders in the prior art are often not suitable for complex and harsh terrain with many mountains and tunnels, nor can they cope with various extreme situations where satellite signals are lost. In addition, the GNSS satellite positioning system often has large errors. To achieve high-precision positioning, it is necessary to set up differential base stations on the positioning line, which also requires certain construction and maintenance costs. In the event that the base station is damaged by extreme circumstances, the response positioning accuracy of the virtual transponder equipment in the prior art will also be greatly affected. Summary of the invention

[0024] The embodiment of the present invention provides a method for implementing a track virtual transponder to overcome the problems of the prior art.

[0025] In order to achieve the above object, the present invention adopts the following technical scheme.

[0026] A method for implementing a track virtual transponder, comprising:

[0027] Step 1: Building an on-board image acquisition and processing system arranged at the train end, and using the on-board image acquisition and processing system to collect a set of driving record images of the front view and the side view during the operation of the train;

[0028] Step 2: Use the improved YOLOv3 network to perform real-time target detection on the driving record image set to obtain the category and location information of existing landmarks on the train track;

[0029] Step 3: Based on the improved CRNN network, content recognition is performed on the driving record image set according to the location information of the existing markers to obtain text information of the existing markers on the train track.

[0030] Step 4: Based on the obtained text information of the existing landmarks, the train operation positioning information is calculated by matching it with the electronic map.

[0031] Preferably, the step 1 comprises:

[0032] An on-board image acquisition and processing system arranged at the train end is constructed, the system comprising a power module, a processor, a front-view camera, a side-view camera, a network switch and a hard disk. The front-view camera is arranged in the cab and faces straight ahead, and the side-view camera is arranged in the cab and faces the side window, and the field of view is perpendicular to the direction of travel of the train. The front-view camera and the side-view camera are connected through a switch, and parameters of the front-view camera and the side-view camera are set and secondary development is performed. The front-view camera and the side-view camera are used to collect driving record videos with specific resolution, specific shutter time and frame rate parameters during the operation of the train, and the driving record videos are stored as images frame by frame. All images are used to form an image set, and the image set is stored in the hard disk.

[0033] Preferably, the step 2 comprises:

[0034] The YOLOv3 network is selected as the benchmark network, the input method of the YOLOv3 network is improved to reduce the pixel loss of small targets during downsampling, the activation function and multi-scale feature fusion structure of the YOLOv3 network are improved, and the target box weighted fusion WBF is used to improve the output processing of the YOLOv3 network to obtain the improved YOLOv3 network;

[0035] The images in the image set are input into the improved YOLOv3 network, and the YOLOv3 network outputs the coordinates of the bounding box of the marker in the image and the category number of the marker, so as to obtain the category and location information of the existing markers on the train track.

[0036] Preferably, the input method of the YOLOv3 network is improved to reduce the pixel loss of small targets during downsampling, including:

[0037] For the front-view image of the railway scene, the YOLOv3 network uses a sliding window method for preprocessing. A 640×640 window is slid on the original 1920×1080 image, the image part within the sliding window is cut out and resized to 416×416 for input into the network, and the target object coordinates detected by the network are mapped back to the original image for post-processing to obtain the final target box parameters on the original image.

[0038] Preferably, the activation function and multi-scale feature fusion structure of the YOLOv3 network are improved, and the output processing of the YOLOv3 network is improved by using the target box weighted fusion WBF, including:

[0039] A spatial pyramid pooling layer is added at the end of the YOLOv3 backbone network to pool feature maps at different scales, and the feature maps obtained after pooling are spliced ​​in the depth direction to obtain new feature maps. The backbone network of the YOLOv3 adopts a darknet53 network structure, which is composed of 52 layers stacked by residual blocks, wherein the activation function of each layer adopts a Leaky-ReLU activation function, and multiple overlapping detection frames are processed by using a target box weighted fusion WBF method. The YOLOv3 network model is built using a Pytorch deep learning framework, and the weights of the YOLOv3 network are trained using images collected by a manually labeled image acquisition system.

[0040] Preferably, the step 3 comprises:

[0041] The CRNN network architecture introduces a residual connection structure, uses the Pytorch deep learning framework to build a CNN and a bidirectional LSTM neural network, extracts features from the image and converts them into sequence features for output prediction, and uses the connectionist temporal classification loss CTCLoss for optimization learning for the output prediction, so that the network automatically learns how to align the time series with the actual text content, and obtains an improved CRNN network;

[0042] The category and location information of existing markers on the train's track route are input into the improved CRNN network, and the improved CRNN network outputs the text information of the existing markers on the train's track route.

[0043] It can be seen from the technical solutions provided by the above-mentioned embodiments of the present invention that the present invention can be used for response positioning during the operation of rail transit vehicles. Machine vision methods are used to detect the positions and identify the contents of existing markers on the line. The detection and identification algorithms use deep learning methods, which can well meet the accuracy and real-time requirements of the algorithms, completely get rid of dependence on ground transponders and satellites, and realize fully autonomous train response positioning.

[0044] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0046] Figure 1It is a structural diagram of a virtual transponder based on the GNSS satellite navigation system in the prior art;

[0047] Figure 2 A schematic diagram of a visual virtual transponder detection and recognition algorithm provided in an embodiment of the present invention;

[0048] Figure 3 A processing flow chart of a visual virtual transponder detection and recognition algorithm provided by an embodiment of the present invention;

[0049] Figure 4 A YOLOv3 network structure diagram provided for an embodiment of the present invention;

[0050] Figure 5 A spatial pyramid pooling layer structure diagram provided by an embodiment of the present invention;

[0051] Figure 6 A schematic diagram of a Mish activation function compared with a Leaky-ReLU activation function proposed in an embodiment of the present invention;

[0052] Figure 7 A schematic diagram comparing WBF and NMS provided in an embodiment of the present invention.

[0053] Figure 8 A CRNN network structure diagram provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0055] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0057] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.

[0058] Existing railway virtual transponder solutions generally rely on GNSS to provide positioning information, which cannot be used in situations where satellite signals are lost, and the cost of high precision is high. Therefore, the present invention uses a machine vision method based on deep learning to detect and identify targets on existing landmarks on the railway line, extracts its content and matches it with a pre-established electronic map library to complete the response. The data collected during the experiment included 2,000 line images. The algorithm was trained and tested with the data collected during the experiment. The algorithm can achieve a response accuracy of 99.5%, and can achieve a response time of less than 150ms on the Nvidia AGX Xavier platform, which can meet real-time requirements.

[0059] The embodiment of the present invention adopts a deep neural network (for example: Yolov3) target detection network architecture to detect and locate the visual feature virtual transponder, extracts the area of ​​its text content from the front-view camera image, detects and locates the visual feature virtual transponder in the side-view camera, and thus accurately determines the moment when the train crosses the virtual transponder.

[0060] The implementation principle of a visual virtual transponder detection and recognition algorithm provided by an embodiment of the present invention is as follows: Figure 2 As shown, the following processing process is included: for the text recognition part, considering the advantages of recurrent neural networks in processing sequential data, the architecture of convolutional neural network cascade recurrent neural network (CRNN) is used to recognize the text content, obtain the text content, and finally match the text content with the electronic map to calculate the train positioning information. In practical application, firstly, the front view camera and side view camera images are captured in real time by cyclic acquisition of camera nodes, and the captured images are preprocessed to narrow the target search area, reduce the amount of calculation, improve real-time performance, and increase the proportion of the target in the network input image. The two parts of the network are cascaded, and the preprocessed image is used as the input of the target detection network, and the output of the detection network is used as the input of the recognition network, so as to realize the two-stage visual virtual transponder detection and recognition algorithm.

[0061] The processing flow of a visual virtual transponder detection and recognition algorithm provided by an embodiment of the present invention is as follows: Figure 3 As shown, the processing steps include the following:

[0062] Step S10: constructing a vehicle-mounted image acquisition and processing system.

[0063] In order to capture existing markers on the line in real time during train operation, and perform image processing and detection and recognition, it is necessary to build an on-board image acquisition and processing system arranged at the train end. The system mainly consists of a core processor, two cameras (front view and side view), and a network switch.

[0064] The front-view camera is arranged in the cab facing straight ahead, and the installation position and angle are based on the requirement that the contents of the trackside markers are clearly visible. The side cameras are arranged in the cab facing the side windows, and the field of view is perpendicular to the direction of travel of the train. The angle is based on the requirement that the trackside markers are within the field of view of the camera.

[0065] Step S20: Establish a sample database.

[0066] The built on-board image acquisition and processing system is used to collect driving record videos from the front and side perspectives during the operation of the train, and the driving record videos are stored as images frame by frame, and all the images are used to form an image set.

[0067] The dataset is expanded by cropping, scaling, brightness adjustment and contrast adjustment to establish a database of existing landmarks in railway scenes.

[0068] Step S30: Detect route visual markers.

[0069] Since it is necessary to utilize the location information contained in the landmarks to realize the virtual response function, it is also necessary to mark the location of each landmark on the line and establish an electronic map library.

[0070] The process of labeling an image set can be divided into two parts: labeling the category and position of the landmarks in the image set and labeling the specific text content of the landmarks. The labeling of the category and position of the landmarks is what needs to be completed for YOLOv3 network training, that is, providing the real frame information required for target detection network training. The labeling of the specific text content of the landmarks is what is required for CRNN network training, that is, providing the real label information corresponding to the network output. Only by completing the training of the two parts of the network can the detection and recognition process of a specific target be achieved.

[0071] To use the existing markers on the line for virtual response, it is first necessary to perform real-time target detection on the images collected by the two cameras during the train operation. In order to ensure the accuracy and real-time performance of the detection algorithm, the present invention preferably uses the YOLOv3 network as the reference network and improves the algorithm on this basis. A YOLOv3 network structure provided in an embodiment of the present invention is as follows: Figure 4 shown.

[0072] The improvement of YOLOv3 in the present invention mainly includes the improvement of network input mode, reducing the pixel loss of small targets in the downsampling process, improving the network activation function and the multi-scale feature fusion structure, improving the detection performance of the network, and improving the network output processing by using WBF (Weighted Boxes Fusion, target box weighted fusion) to optimize the final target box position more accurately. The input data of the network is the original image collected by the image acquisition system. The output result is the coordinates of the bounding box of the marker and the category number of the marker.

[0073] The first problem that the present invention must solve is that the target to be detected occupies fewer pixels in the image and is a small target. Small target detection is a difficult problem in the field of target detection. From the network structure of YOLOv3, it can be found that the input size received by YOLOv3 is 416×416, and the size of the last feature map output is only 13×13. Although the network borrows the spatial pyramid structure of FPN to fuse feature maps of different layers as output, even the largest output feature map has a size of only 52×52. Compared with the input size of the network, its downsampling rate is 8 times, which means that a target with a size of 16×16 on the input image can only save 2×2 feature pixels on the largest output feature map, and can only share one pixel with its surrounding pixels on the smallest output feature map to express features. If the image to be detected is directly scaled to a size of 416×416 and input into the network, according to the previous analysis of the size of the data set targets, it can be seen that there will be a large number of targets with a width less than 16 pixels when input to the network, which will cause their features to be lost during the downsampling process.

[0074] The present invention adopts a sliding window method to preprocess the forward-view image of the railway scene, sliding a 640×640 window on the original 1920×1080 image, cutting out the image part within the sliding window and resizing it to 416×416 for input to the network. By adopting this method, the downsampling rate of the original image before inputting the network is greatly reduced, so that the target object can retain more pixels before inputting the network, and as much as possible, it is avoided to input small targets into the network for detection. Finally, the coordinates of the target object detected by the network are mapped back to the original image for post-processing to obtain the final target frame parameters on the original image.

[0075] For the highway scene dataset, since the route is short, the area where the line markers are located within the camera's field of view on the experimental route is relatively fixed and is located on the right side of the road. Therefore, the original image can be directly cropped to remove the area where the markers are unlikely to appear, thereby reducing the downsampling ratio when the image is input into the network, while reducing the image pixels to be processed and improving the real-time performance of the network.

[0076] In order to further integrate information of different scales, improve the detection effect of smaller-scale targets, and enable the network to take into account both local features and global features, the present invention draws on the idea of ​​spatial pyramid network (SPP-Net) and adds a spatial pyramid pooling layer at the end of the YOLOv3 backbone network to pool feature maps at different scales, and splice the feature maps obtained after pooling in the depth direction to obtain a new feature map. A spatial pyramid pooling layer provided in an embodiment of the present invention is as follows Figure 5 As shown in the figure, the backbone network of YOLOv3 adopts the darknet53 network structure, which draws on the residual block structure of ResNet. It consists of 52 layers stacked by residual blocks, and the activation function of each layer adopts the Leaky-ReLU activation function.

[0077] The formula of the Mish activation function is:

[0078]

[0079] Where x is the input and y is the output.

[0080] Figure 6 Schematic diagram of a Mish activation function compared with a Leaky-ReLU activation function proposed in an embodiment of the present invention. The Mish activation function curve is as follows Figure 6 As shown in the left figure. It can be seen that the Mish activation function is significantly smoother than the Leaky-ReLU activation function, and a smoother activation function can allow information to penetrate the neural network better, thereby improving network performance. Diganta Misra also proved through experiments in the paper that the activation effect of the activation function in the convolutional neural network is much better than the commonly used activation functions such as Leaky-ReLU. Therefore, the present invention uses the Mish activation function to replace the Leaky-ReLU activation function in the original Yolov3 backbone network.

[0081] In addition, since the present invention requires accurate positioning of the detected target, the traditional non-maximum suppression (NMS) method only retains the coordinate information of the detection frame with the highest confidence based on the intersection-and-union ratio threshold when processing multiple overlapping detection frames, while ignoring the information provided by other detection frames. The embodiment of the present invention uses WBF (Weighted Boxes Fusion) to replace NMS. The algorithm steps of WBF are first to add each predicted frame to list B, and sort this list in descending order according to the confidence score. Then, an empty list L and a list F (for fusion) are established, and then list B is traversed in a loop, and a matching target frame is found in F (MIOU>0.55 for the same category). If no matching target frame is found in the previous step, this frame is added to the tail of L and F, otherwise this frame is added to L. The added position is the index of the matching frame of the target frame in F. There may be multiple frames at each position in L, and the value at the corresponding F index needs to be updated according to these multiple frames. A schematic diagram of the comparison between WBF and NMS provided by an embodiment of the present invention is shown in FIG. Figure 7 shown.

[0082] Step S40: Responder text content recognition

[0083] The improvement of the present invention on CRNN mainly lies in the optimization of its backbone network. The backbone network of the original CRNN network adopts the structure of VGG-16 for feature extraction. The structure of this network is relatively complex and cannot meet the real-time requirements, and the effect of feature extraction is general. Therefore, the present invention uses the Pytorch deep learning framework to build a new feature extraction network. The residual connection structure is introduced into the network so that the network can achieve better feature extraction effect with a smaller amount of calculation. In addition, the convolution kernel pruning operation is performed on the trained CRNN network to further reduce the amount of calculation and ensure real-time performance. The input data is the marker area obtained in step 2, and the output result is the specific marker text content.

[0084] Scene text detection or recognition methods have generally utilized the development of deep learning architectures and have demonstrated excellent accuracy on public datasets when processing multi-resolution and multi-directional texts. Considering that the input image of the present invention may not contain irregular shapes and distorted images, a processing architecture based on CTC (Connectionist Temporal Classification) for general regular images is selected to achieve text recognition through a cascade of convolutional layers, recurrent neural network layers, and CTC transcription layers. This structure is also the structure of a CRNN network, but the CRNN network also has certain defects. The present invention optimizes the original CRNN for application scenarios.

[0085] Figure 8A CRNN network structure diagram provided for an embodiment of the present invention, the original CRNN network is designed for horizontal text, and the extracted features ultimately compress the input image into one pixel in the vertical direction. For the common vertical contact network column numbers in railways, if they are directly input into the network, it is obviously impossible to extract sequence features. Therefore, when using railway scene images for network training, the samples of vertical text need to be rotated 90° counterclockwise, and then adjusted to a size suitable for network input and input into the network, so that the network can effectively learn the features of vertical text. In the simulation experiment of highway scenes, since all samples are vertical text, it is directly chosen to modify the convolution kernel size when building the network, and horizontal rectangular convolution kernels and pooling kernels are used instead of vertical rectangular convolution kernels and pooling kernels, so as to compress the input image into one pixel in the width direction and extract its vertical sequence features.

[0086] The original CRNN network uses VGG-16 as the feature extraction network, which has a large number of network parameters, a slow forward propagation speed, and average feature extraction performance. The present invention draws on the ResNet network structure to introduce a residual block structure into the network, so that the network can better utilize deep features. At the same time, in order to reduce the amount of calculation and the amount of parameters, the number of convolutional layers is appropriately reduced, and the idea of ​​deep separable convolution in Mobile-Net is drawn on to use deep separable convolution instead of traditional convolution in some convolutional layers. Through the above transformation, better recognition accuracy and reasoning speed are finally achieved than the original network.

[0087] Finally, in order to further compress the network model and ensure the real-time performance of the algorithm, the network convolution layer is pruned to reduce the amount of network calculation and parameters. Convolutional neural networks are usually composed of multiple convolutional layers stacked together, and each convolutional layer contains multiple convolutional kernels. Assuming that the weight tensor of a convolutional kernel is K, its basic convolution operation can be expressed as:

[0088]

[0089] In the formula: x is the input of the convolutional neural network. During its forward propagation along the network, an input feature map X is formed at the convolution kernel K, and an output feature map Fx is formed. Any convolution kernel K can be regarded as a feature template. The convolution operation is to scan the input image and calculate the degree of match between its different positions and the feature template. The more convolution kernels there are, the more patterns the convolutional neural network can express and the stronger its learning ability. Therefore, in practical applications, for a specific task, it is usually easier to achieve the training goal by training a network structure containing more convolution kernels, but this often results in many redundant or useless convolution kernels in the network. In order to identify these unnecessary convolution kernels, the current common method is to calculate the L1 norm of each convolution kernel after training is completed. Convolution kernels with large L1 norms are considered important, while convolution kernels with small L1 norms are considered unimportant and can be removed from the network to reduce the amount of computation and avoid over-training. However, according to the characteristics of the convolution operation, each convolution kernel actually represents a feature template at a certain abstract level. For example, in image recognition applications, the convolution kernels of the first layer are usually edge detection templates in different directions. Therefore, the amplitude of the convolution kernel elements does not necessarily accurately reflect whether the features represented by a convolution kernel are useful for the target problem. In other words, the L1 norm of the convolution kernel The size of only represents the amplitude of the feature, and cannot reflect the importance of the convolution kernel. The present invention uses the scaling coefficient of the batch normalization layer connected after the convolution layer as the evaluation standard for the importance of the convolution channel for pruning. The principle is that the batch normalization layer normalizes the feature map output by the previous convolution layer, and its scaling coefficient just reflects the amplitude of the feature map response, so it can be used to judge the importance of the corresponding channel feature. The formula of the batch normalization layer is:

[0090]

[0091] After model pruning, fine-tuning training on the pruned model can ensure the model accuracy as much as possible while achieving faster forward propagation speed.

[0092] Finally, to realize the complete line visual transponder recognition algorithm, it is necessary to cascade the detection network and the recognition network. The cascading method is to first determine the target object category label obtained by the detection network. For targets with the label category as text areas, the target box coordinates obtained by the detection network are mapped back to the original image, and the corresponding area is cropped out and input into the second-stage recognition network. Obtaining the recognition network input from the original image can obtain a higher resolution so that the network can learn more detailed features, which is conducive to character recognition.

[0093] To summarize, the embodiments of the present invention provide a method for implementing a track virtual transponder, which can be used for response positioning during the operation of rail transit vehicles. A machine vision method is used to detect the position and identify the content of existing markers on the line. The detection and identification algorithm adopts a deep learning method, which can well meet the accuracy and real-time requirements of the algorithm, completely get rid of the dependence on ground transponders and satellites, and realize fully autonomous train response positioning, so that the system has better response capabilities and reliability for various complex and severe situations such as extreme weather, tunnel sections, mountainous sections, equipment damage, and communication interruptions.

[0094] The line feature virtual transponder based on machine vision in the embodiment of the present invention is compatible with the existing transponder system of the line and the train operation control system. Through technical upgrades, the existing system can achieve reliable and continuous autonomous positioning under adverse conditions such as extreme weather, geological disasters, ground equipment failures, mountainous areas and tunnel environments, which is of great significance for improving the safety level of railway operations.

[0095] The present invention uses existing line markers such as kilometer marks and contact network column numbers as the source of virtual transponder location information to avoid additional installation of other equipment on the line, greatly reducing construction and maintenance costs.

[0096] The present invention adopts a machine vision method based on deep learning to detect and identify targets, avoiding manual design and feature extraction, so that the algorithm has better robustness for application under different light, different backgrounds and different weather conditions.

[0097] The image feature extraction network structure adopted by the present invention has been light-weighted and optimized, with small parameter and calculation amount, and can be directly operated on the embedded computing platform without sending the image data to the server for processing, making the system more real-time and reliable.

[0098] The system retains interfaces with other sensors and vehicle positioning solutions to facilitate expansion and optimization of the system.

[0099] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0100] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.

[0101] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0102] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for implementing a track virtual balise, characterized in that: include: Step 1: Building an on-board image acquisition and processing system arranged at the train end, and using the on-board image acquisition and processing system to collect a set of driving record images of the front view and the side view during the operation of the train; Step 2: Use the improved YOLOv3 network to perform real-time target detection on the driving record image set to obtain the category and location information of existing landmarks on the train track; Step 3: Based on the improved CRNN network, content recognition is performed on the driving record image set according to the location information of the existing markers to obtain text information of the existing markers on the track running line of the train; Step 4: Based on the obtained text information of the existing landmarks, the train operation positioning information is calculated by matching it with the electronic map.

2. The method according to claim 1, characterized in that The step 1 comprises: An on-board image acquisition and processing system arranged at the train end is constructed, the system comprising a power module, a processor, a front-view camera, a side-view camera, a network switch and a hard disk. The front-view camera is arranged in the cab and faces straight ahead, and the side-view camera is arranged in the cab and faces the side window, and the field of view is perpendicular to the direction of travel of the train. The front-view camera and the side-view camera are connected through a switch, and parameters of the front-view camera and the side-view camera are set and secondary development is performed. The front-view camera and the side-view camera are used to collect driving record videos under resolution, shutter time and frame rate parameters during the operation of the train, and the driving record videos are stored as images frame by frame. All images are used to form an image set, and the image set is stored in the hard disk.

3. The method according to claim 1 or 2, characterized in that: The step 2 comprises: The YOLOv3 network is selected as the benchmark network, the input method of the YOLOv3 network is improved to reduce the pixel loss of small targets during downsampling, the activation function and multi-scale feature fusion structure of the YOLOv3 network are improved, and the target box weighted fusion WBF is used to improve the output processing of the YOLOv3 network to obtain the improved YOLOv3 network; The images in the image set are input into the improved YOLOv3 network, and the YOLOv3 network outputs the coordinates of the bounding box of the marker in the image and the category number of the marker, so as to obtain the category and location information of the existing markers on the train track.

4. The method according to claim 3, characterized in that: The above improvements to the input method of the YOLOv3 network reduce the pixel loss of small targets during downsampling, including: For the front-view image of the railway scene, the YOLOv3 network uses a sliding window method for preprocessing. A 640×640 window is slid on the original 1920×1080 image, the image part within the sliding window is cut out and resized to 416×416 for input into the network, and the target object coordinates detected by the network are mapped back to the original image for post-processing to obtain the final target box parameters on the original image.

5. The method according to claim 3, characterized in that: The activation function and multi-scale feature fusion structure of the YOLOv3 network are improved, and the output processing of the YOLOv3 network is improved by using the target box weighted fusion WBF, including: A spatial pyramid pooling layer is added at the end of the YOLOv3 backbone network to pool feature maps at different scales, and the feature maps obtained after pooling are spliced ​​in the depth direction to obtain new feature maps. The backbone network of the YOLOv3 adopts a darknet53 network structure, which is composed of 52 layers stacked by residual blocks, wherein the activation function of each layer adopts a Leaky-ReLU activation function, and multiple overlapping detection frames are processed by using a target box weighted fusion WBF method. The YOLOv3 network model is built using a Pytorch deep learning framework, and the weights of the YOLOv3 network are trained using images collected by a manually labeled image acquisition system.

6. The method according to claim 3, characterized in that The step 3 comprises: The CRNN network architecture introduces a residual connection structure, uses the Pytorch deep learning framework to build a CNN and a bidirectional LSTM neural network, extracts features from the image and converts them into sequence features for output prediction, and uses the connectionist temporal classification loss CTCLoss for optimization learning for the output prediction, so that the network automatically learns how to align the time series with the actual text content, and obtains an improved CRNN network; The category and location information of existing markers on the train's track route are input into the improved CRNN network, and the improved CRNN network outputs the text information of the existing markers on the train's track route.

Images