Subway multi-color signal lamp recognition method, device and medium based on deep learning

Abstract

The present application relates to a kind of subway multi-color light signal recognition method, equipment and medium based on deep learning, the method includes: step S1, vehicle-mounted camera real-time monitoring and obtaining the video data of front track line, constructs color light signal machine data set;Step S2, color light signal machine data set is input after deep learning network training is accelerated processing, obtains the model of detecting color light signal machine;Step S3, according to video data using deep learning semantic segmentation network identifies the track area that train will soon travel in front, obtains the image containing track area identification result;Step S4, the original image of the right side of track area is input into the model of detecting color light signal machine, obtains the target frame of all color light positions;Step S5, the color of all color lights is judged, and the color light closest to train is selected;Step S6, the color light position and color closest to train are printed in real time output.Compared with prior art, the present application has improved recognition speed and precision and the like advantages.

Description

TECHNICAL FIELD

[0001] The present application relates to the detection of color light signals in the field of rail transit, in particular to a subway multi-color light signal recognition method, device and medium based on deep learning. BACKGROUND

[0002] The subway is an important part of the city's comprehensive three-dimensional transportation system and an important symbol of the city's modernization level. The construction of "smart subway" will further promote the digital transformation and high-quality development of China's rail transit industry, and provide support for the construction of a traffic power. At present, the CBTC (Communication Based Train Control) system is widely used in the unmanned line of "smart subway" at home and abroad. When the CBTC signal system is degraded or disabled due to failure, the vehicle still needs to strictly follow the driving instructions of the trackside color light signal machine, and cannot violate the red light signal. At this time, on the unattended vehicle, a set of equipment is needed to detect and filter the trackside signal machine, to ensure that the train does not violate the red light, and to filter out other color light signal machines in the picture, and only keep the color light signal machine closest to the right front of the train as the driving instruction.

[0003] At present, the detection and filtering algorithm of the signal light mainly focuses on identifying the red and green lights at the ground intersection, and there are few algorithms for detecting and filtering color light signal machines in the subway main line and vehicle depot. The existing technology mainly fuses HOG-LBP features for feature fusion, then inputs the HOG-LBP features of the obtained image into a support vector machine (SVM) algorithm for training to obtain an SVM signal light classifier, and then uses the obtained SVM classifier to detect the image.

[0004] After searching, Chinese patent No. CN112215089A discloses a video recognition method for subway color light signal machines, specifically discloses that the method is used for unmanned vehicles in subway tunnel environment to recognize trackside color light signal machines, and in view of the special environment in the tunnel and the characteristics of the collected video data, the image is processed in multiple ways, finally excluding external interference, and accurately and quickly recognizing the signal light view.

[0005] However, the detection result of simply using the traditional scheme to identify the color light is poor, the color light far away cannot be detected, or the color light in the tunnel is easily affected by the environment and has a large halo around the color light, affecting the identification; in view of the complex and variable scene, the accuracy is poor. Therefore, in the face of unattended trains, in order to ensure the safe driving of the train, it is necessary to identify the signal light in front of the train in real time, and when the color light signal machine appears in the video picture, the information of the color light signal machine can be identified, and other color lights in the picture need to be quickly filtered out, and only the color light giving driving instruction to the train in the forward process is kept. SUMMARY

[0006] The purpose of the present application is to provide a subway multi-color light signal recognition method, device and medium based on deep learning to overcome the defects of the prior art.

[0007] The purpose of the present application can be achieved by the following technical solutions:

[0008] According to the first aspect of the present application, a subway multi-color light signal recognition method based on deep learning is provided, which specifically comprises the following steps:

[0009] Step S1, the vehicle-mounted camera monitors and acquires video data of the front track line in real time, and constructs a color light signal machine data set;

[0010] Step S2, input the color light signal machine data set into the deep learning network for acceleration processing after training, and obtain the model for detecting color light signal machines;

[0011] Step S3, according to the video data, the track area to be traveled in front of the train is identified by using a deep learning semantic segmentation network, and an image containing the track area identification result is obtained;

[0012] Step S4, input the original image on the right side of the track area into the model for detecting color light signal machines, and obtain the target frame of all color light positions;

[0013] Step S5, judge the color of all color lights, and select the color light closest to the train;

[0014] Step S6, the position and color of the color light closest to the train are printed and output in real time. As a preferred technical solution, the color light signal machine data set is classified according to the color of the color light signal machine.

[0015] As a preferred technical solution, the deep learning network is constructed by yolov5 algorithm.

[0016] As a preferred technical solution, the model of the deep learning network after training is accelerated by TensorRT.

[0017] As a preferred technical solution, the model for detecting color light signal machines is deployed on the vehicle-mounted video processing unit host.

[0018] As a preferred technical solution, the track area is obtained by the way of track area identification of video data by deep learning semantic segmentation network.

[0019] As a preferred technical solution, if there are multiple tracks in the video data picture, the track to be traveled by the train is determined according to the turnout position, and the track that is not on the train's forward line is cut off.

[0020] As a preferred technical scheme, the track area right side original image obtaining method is as follows: a leftmost point of the track area is found, a vertical tangent is drawn in the picture through the point, and a picture on the right side of the tangent is cut out to obtain the track area image.

[0021] As a preferred technical scheme, the color lamp position is obtained from a maximum circumscribed rectangle target box of the color lamp containing coordinate information.

[0022] As a preferred technical scheme, the color lamp color judgment method is as follows: color lamp images in the target box of all color lamp positions are converted into HSV color masks, an HSV color space range is used to screen out a proportion of each color lamp color, and the color of the color lamp is judged according to the proportion.

[0023] As a preferred technical scheme, the color lamp distance is judged by a center point y coordinate of the color lamp position information, and the color lamp closest to the train is the color lamp with the largest y coordinate value.

[0024] As a preferred technical scheme, the color lamp position and color are printed and output in real time on a video picture.

[0025] According to a second aspect of the present application, an electronic device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the method when executing the program.

[0026] According to a third aspect of the present application, a computer readable storage medium is provided, storing a computer program, and the program is executed by a processor to implement the method.

[0027] Compared with the prior art, the present application has the following advantages:

[0028] 1) The present application first proposes a detection and filtering scheme for subway line and depot color lamp signal machines, laying a foundation for subsequent development and improvement;

[0029] 2) The present application applies deep learning technology to color lamp signal machine recognition and detection, and compared with traditional methods, has greatly improved in recognition speed and accuracy;

[0030] 3) The present application can recognize the color lamp state when the light-emitting area of the color lamp signal machine enters the camera video capture area, leaving sufficient safety distance for the train;

[0031] 4) The deep learning yolov5 network used in the present application has the advantages of fast running speed, high recognition accuracy and light weight, and TensorRT is used to accelerate the yolov5 network model, TensorRT quantizes the model to FP16 or INT8 to maximize throughput while maintaining high accuracy, reduces the operation cost, and greatly guarantees the real-time performance of the algorithm;

[0032] 5) The detection filtering scheme proposed in the present application is only based on real-time video data, and is not realized in cooperation with other systems and devices, and has high stability. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 A flowchart of the subway multi-color light signal recognition method based on deep learning of the present application is shown in the figure.

[0034] Figure 2 A flowchart of the model for detecting color light signal machines is shown in the figure.

[0035] Figure 3 A flowchart of the image of the track area is shown in the figure.

[0036] Figure 4 A flowchart of the target frame of all color light positions is shown in the figure.

[0037] Figure 5 A flowchart of judging the color of all color lights is shown in the figure.

[0038] Figure 6 A flowchart of selecting the color light closest to the train is shown in the figure.

[0039] Figure 7 A real-time printing output interface diagram of color light position and color is shown in the figure. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0041] The purpose of the present application is to provide a subway multi-color light signal recognition method based on deep learning in the main line and shunting vehicle depot of the subway, as shown in the figure. Figure 1 The method comprises the following steps:

[0042] Step S1: The vehicle-mounted camera detects and acquires the video data of the front track line in real time, constructs a subway color light signal machine data set, and the category is the color of the color light signal machine.

[0043] Step S2: input the color light signal into the deep learning yolov5 network for training to obtain a yolov5 model for detecting the color light signal.

[0044] Step S3: accelerate the trained yolov5 model using TensorRT, and then deploy the yolov5 model to a vehicle-mounted video processing unit host.

[0045] Step S4: use a deep learning semantic segmentation network to obtain a track area in front of the train to be driven, if there are multiple tracks in the picture, determine the track to be driven by the train according to the turnout position (positioning / repositioning), and cut off the track that is not the train forward line.

[0046] Step S5: find the leftmost point of the cut track area, draw a vertical tangent line on the original picture according to the leftmost point of the track area, and cut out the image to the right of the tangent line.

[0047] Step S6: input the cut image data into the yolov5 acceleration model to identify the color light signal, and obtain the position information of all color light signals.

[0048] Step S7: according to the color light signal position information obtained in step 5, frame all color lights on the original picture using the maximum circumscribed rectangle to obtain the target frame of the color light position.

[0049] Step S8: convert the color light image in all target frames on the original picture into an HSV color mask, and select the proportion of each color light color using the HSV color space range, so as to determine the color of the color light according to the proportion.

[0050] Step S9: according to the center point y coordinate of the color light position information, select the color light closest to the train, that is, the color light with the maximum y coordinate value.

[0051] Step S10: print and output the detected color light position and color information in real time on the picture.

[0052] As Figure 2As shown in the figure, after the video of the subway mainline and the subway depot acquired by the front camera is divided into images, the images containing color light signals are selected and labeled with the color category of the color light, resulting in a color light signal dataset. The color light signal dataset is then input into a YOLOv5 network for training to obtain a color light signal recognition model. The YOLOv5 model is accelerated using TensorRT. Deep learning networks typically use 32-bit or 16-bit data during training. TensorRT maximizes throughput while maintaining high accuracy by quantizing the model to FP16 or INT8. Moreover, TensorRT can perform layer and tensor fusion, optimizing the use of GPU memory and bandwidth by fusing nodes in the kernel, thus obtaining an accelerated model for recognizing color light signals.

[0053] like Figure 3 As shown in the image, the model trained using a semantic segmentation network can acquire the track area of ​​the train's path and overlay a mask on it. If there are multiple tracks in the image, the model determines the track the train will take based on the switch position (local / reverse) and cuts off the track that is not part of the train's path. A perpendicular tangent line is drawn on the original video image using the leftmost point of the track area, and the image to the right of the perpendicular line is cropped.

[0054] like Figure 4 As shown in the figure, the cropped image is input into the YOLOv5 acceleration model to identify the color light, obtain the position coordinate information of the color light, and use the coordinate information to draw the maximum bounding rectangle of the color light on the original image.

[0055] like Figure 5 As shown in the figure, the cropped RGB image is converted into an HSV color mask and then a color threshold is determined. The proportion of each color light is filtered out using the HSV color space range, and the color of the light light is determined based on the proportion, thus obtaining the color result of the light light.

[0056] like Figure 6 As shown in the figure, when multiple colored lights are displayed, the color light closest to the train is determined based on the y-coordinate of the center point of the colored light.

[0057] like Figure 7 As shown in the figure, the final detected position and color information of the colored light are printed out in real time on the screen. If there are multiple colored lights on the screen, the current colored light is the one closest to the train.

[0058] The above is an introduction to the method embodiments. The following embodiments using electronic devices and storage media will further illustrate the solution of the present invention.

[0059] The electronic device of the present application includes a central processing unit (CPU) that can perform various appropriate actions and processes in accordance with computer program instructions stored in a read-only memory (ROM) or computer program instructions loaded from a storage unit into a random access memory (RAM). Various programs and data required for device operation can also be stored in the RAM. The CPU, ROM, and RAM are connected to each other by a bus. An input / output (I / O) interface is also connected to the bus.

[0060] A plurality of components in the device are connected to the I / O interface, including: an input unit such as a keyboard, mouse, etc.; an output unit such as various types of displays, speakers, etc.; a storage unit such as a magnetic disk, optical disk, etc.; and a communication unit such as a network card, modem, wireless communication transceiver, etc. The communication unit allows the device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0061] The processing unit performs the various methods and processes described above, such as methods S1-S10. For example, in some embodiments, methods S1-S10 can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on the device via the ROM and / or the communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1-S10 described above can be performed. Alternatively, in other embodiments, the CPU can be configured to perform methods S1-S10 by any other appropriate means, such as by means of firmware.

[0062] The functionality described above in this document can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, example types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), etc.

[0063] Program code for carrying out methods of the present application can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, causes the machine to perform the functions / acts specified in the flowcharts and / or block diagrams. The program code can execute entirely on a machine, partly on a machine, as a stand-alone software package, partly on a machine and partly on a remote machine or entirely on a remote machine or server.

[0064] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable storage medium can include, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage medium would include one or more lines of electrical wire, portable computer diskette, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0065] The above description is only specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for recognizing a subway multi-color signal based on deep learning, characterized in that, Specifically comprising the following steps: Step S1, the vehicle-mounted camera monitors and acquires video data of the front track line in real time, and constructs a color light signal machine data set; Step S2, the color light signal machine data set is input into a deep learning network for acceleration processing after training, and a model for detecting color light signal machines is obtained; Step S3, the video data is identified by using a deep learning semantic segmentation network to identify the track area to be traveled in front of the train, and an image containing track area identification results is obtained; Step S4, the original image on the right side of the track area is input into the model for detecting color light signal machines, and the target frame of all color light positions is obtained; Step S5, the colors of all color lights are judged, and the color light closest to the train is selected; Step S6, the position and color of the color light closest to the train are printed and output in real time; The deep learning network is constructed by a yolov5 algorithm; The track area is obtained by using a deep learning semantic segmentation network to identify the track area from the video data; The method for obtaining the original image on the right side of the track area is as follows: find the leftmost point of the track area, draw a vertical tangent line through the point in the original image, and cut out the image on the right side of the tangent line as the image on the right side of the track area. 2.The metro multi-color signal light recognition method based on deep learning according to claim 1, wherein, The color of the color light signal machine data set is the color of the color light signal machine. 3.The metro multi-color signal light recognition method based on deep learning according to claim 1, characterized in that, The model of the deep learning network after training is accelerated by TensorRT.

4. The method according to claim 1, wherein, The model for detecting color light signal machines is deployed on a vehicle-mounted video processing unit host.

5. The method according to claim 1, wherein, If there are multiple tracks in the video data picture, the track to be traveled by the train is determined according to the turnout position, and the track that is not on the train's forward line is cut off.

6. The method according to claim 1, wherein, The color light position is obtained from the maximum circumscribed rectangular target frame containing coordinate information of the color light.

7. The method according to claim 1, wherein, The color judgment method of the color light is as follows: the color light image in the target frame of all color light positions is converted into an HSV color mask, the proportion of each color light color is screened out by using the HSV color space range, and the color of the color light is determined according to the proportion.

8. The method according to claim 1, wherein, The distance of the color light is determined by the center point y coordinate of the color light position information, and the color light closest to the train is the color light with the maximum y coordinate value.

9. The method according to claim 1, wherein, The color light position and color are printed and output in real time on the video picture.

10. An electronic device comprising a memory and a processor, said memory having stored thereon a computer program, characterized in that, The processor executes the program to implement the method of any one of claims 1-9.

11. A computer readable storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the method of any one of claims 1-9.

Images