Signal light recognition method, device and computer readable storage medium

Through multi-label object detection and semantic segmentation optimization, the accuracy problem of traffic light recognition is solved, and high-precision and real-time signal light recognition is achieved to meet the high-precision needs of intelligent traffic.

CN112597874BActive Publication Date: 2024-11-05ZHEJIANG DAHUA TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202011507831.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-18
Publication Date
2024-11-05
Estimated Expiration
2040-12-18

AI Technical Summary

Technical Problem

The existing technology is difficult to accurately identify traffic lights, resulting in misjudgment of violations and navigation errors, and cannot meet the requirements of intelligent traffic for high accuracy.

Method used

The signal light group image is obtained based on the regional position information of the signal light group, and multi-label object detection is performed. Combined with atmospheric light filtering and semantic segmentation, segmentation boundary optimization is performed to output the signal light recognition results.

Benefits of technology

It realizes high-precision and real-time recognition of signal lights, ensures clear resolution of signal light areas in the image, and meets the requirements of intelligent traffic for high accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112597874B_ABST
    Figure CN112597874B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a method, device and computer-readable storage medium for identifying a traffic light. The method includes: obtaining a traffic light group image from a first monitoring image based on the regional location information of the traffic light group; performing multi-label target detection based on the traffic light group image to determine the color of the traffic lights in the traffic light group, or the color and shape of the traffic lights in the traffic light group; performing semantic segmentation on the area where the traffic lights with the target color are located in the traffic light group based on the color of the traffic lights in the traffic light group; performing segmentation boundary optimization on the result of the semantic segmentation, and outputting the recognition result of the traffic light based on the result after the segmentation boundary optimization. Through the present invention, the problem of accurate recognition of traffic lights is solved. The solution has the characteristics of real-time and precision, and can well meet the requirements of intelligent transportation for high-precision traffic light recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of video surveillance, and in particular, to a signal light recognition method, device and computer-readable storage medium. Background Art

[0002] With the development of video surveillance technology, the monitoring of urban traffic conditions has become more and more intelligent, and the results of video surveillance have been applied in various fields. For example, video surveillance is used to capture violations, and video surveillance is used to identify traffic lights and pedestrian and vehicle flows at intersections for intelligent navigation.

[0003] The widespread implementation of video surveillance relies on accurate recognition technology. If the recognition result is inaccurate, it will inevitably lead to misjudgment of illegal capture or navigation errors. Whether in illegal capture, intelligent navigation or other monitoring scenarios involving vehicles and pedestrians on the road, accurate recognition of traffic lights is crucial. How to accurately identify traffic lights is a problem that needs to be solved urgently. Summary of the invention

[0004] Embodiments of the present invention provide a method and device for identifying a signal light, and a computer-readable storage medium, to at least solve the problem of accurately identifying a signal light in the related art.

[0005] According to one embodiment of the present invention, a method for identifying a traffic light is provided, comprising: acquiring a traffic light group image from a first monitoring image based on regional location information of the traffic light group; performing multi-label target detection based on the traffic light group image to determine the color of the traffic lights in the traffic light group, or the color and shape of the traffic lights in the traffic light group; performing semantic segmentation on the area where the traffic lights with the target color in the traffic light group are located based on the color of the traffic lights in the traffic light group; performing segmentation boundary optimization on the result of the semantic segmentation, and outputting the recognition result of the traffic light based on the result after the segmentation boundary optimization.

[0006] In at least one exemplary embodiment, before acquiring the signal light group image from the first monitoring image based on the regional location information of the signal light group, the method further includes: performing target detection of the signal light group based on the second monitoring image according to a predetermined detection cycle to obtain the regional location information of the signal light group.

[0007] In at least one exemplary embodiment, target detection of the signal light group is performed based on the second monitoring image according to a predetermined detection cycle to obtain the regional location information of the signal light group, including: acquiring the second monitoring image according to the predetermined detection cycle; dividing the second monitoring image into multiple sub-images; performing target detection of the signal light group based on the multiple sub-images to obtain a signal light group detection frame in the multiple sub-images; restoring the signal light group detection frame in the multiple sub-images to the second monitoring image for fusion to obtain the regional location information of the signal light group.

[0008] In at least one exemplary embodiment, there is partial overlap between adjacent sub-images among the divided multiple sub-images, and the signal light group detection frames in the multiple sub-images are restored to the second monitoring image for fusion to obtain the regional position information of the signal light group, including: restoring the signal light group detection frame to the second monitoring image based on the position of the signal light group detection frame in the multiple sub-images; in the case of overlapping or partially overlapping signal light group detection frames, merging or deduplicating the overlapping or partially overlapping signal light group detection frames; and obtaining the regional position information of the signal light group based on the merged or deduplicated signal light group detection frames.

[0009] In at least one exemplary embodiment, obtaining a signal light group image from a first monitoring image based on the area location information of the signal light group includes: obtaining target area location information by expanding the width and / or height of the area indicated by the area location information by a predetermined proportion or value; and obtaining the signal light group image from the first monitoring image based on the target area location information.

[0010] In at least one exemplary embodiment, before performing multi-label target detection based on the signal light group image to determine the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group, the method further includes: performing atmospheric light filtering on the signal light group image based on an atmospheric scattering model, wherein the atmospheric scattering model is x represents the index of the pixel point, I(x) represents the signal light group image before atmospheric light filtering, J(x) represents the signal light group image after atmospheric light filtering, A represents the atmospheric light value, and t(x) represents the transmittance at the pixel point x.

[0011] In at least one exemplary embodiment, after semantically segmenting the area where the traffic lights with the target color in the traffic light group are located based on the colors of the traffic lights in the traffic light group, the method further includes: constraining the result of the semantic segmentation based on the detection box of the traffic lights corresponding to the target color of the multi-label target detection.

[0012] In at least one exemplary embodiment, constraining the result of the semantic segmentation based on the detection box of the traffic light corresponding to the target color of the multi-label target detection includes: filtering out the results of the semantic segmentation that are outside the detection box of the traffic light corresponding to the target color.

[0013] In at least one exemplary embodiment, optimizing the segmentation boundary of the semantic segmentation result includes: predicting an edge mask and a direction mask based on the semantic segmentation result, wherein the edge mask carries indication information for distinguishing edge pixels from non-edge pixels, and the direction mask carries indication information for indicating the pixel direction; generating an edge repair mask based on the edge mask and the direction mask; and mapping the semantic segmentation result through the edge repair mask to obtain the segmentation boundary optimized result.

[0014] In at least one exemplary embodiment, in the edge mask, the indication information of the edge pixels is set to 1, and the indication information of the non-edge pixels is set to 0; generating an edge repair mask based on the edge mask and the direction mask includes: multiplying corresponding elements in the edge mask and the direction mask to obtain the edge repair mask.

[0015] In at least one exemplary embodiment, outputting the recognition result of the signal light based on the result after the segmentation boundary optimization includes at least one of the following: painting the signal light having the target color with the target color based on the result after the segmentation boundary optimization; outputting the color of the signal light in the signal light group, or the color and shape of the signal light in the signal light group.

[0016] In at least one exemplary embodiment, when multi-label target detection is performed based on the signal light group image to determine the color and shape of the signal lights in the signal light group, outputting the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group includes: determining whether the signal light group is a mixed direction signal light group or a unidirectional signal light group; when the signal light group is the mixed direction signal light group, outputting the color and shape of the signal lights in the signal light group; when the signal light group is the unidirectional signal light group, outputting the color of the signal lights in the signal light group.

[0017] According to another embodiment of the present invention, a device for identifying a traffic light is provided, comprising: an acquisition module, configured to acquire a traffic light group image from a first monitoring image based on regional location information of the traffic light group; a multi-label detection module, configured to perform multi-label target detection based on the traffic light group image to determine the color of the traffic lights in the traffic light group, or the color and shape of the traffic lights in the traffic light group; a semantic segmentation module, configured to perform semantic segmentation on an area in the traffic light group where traffic lights with a target color are located, based on the color of the traffic lights in the traffic light group; and a boundary optimization and output module, configured to perform segmentation boundary optimization on a result of the semantic segmentation, and output a recognition result of the traffic light based on the result after the segmentation boundary optimization.

[0018] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0019] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0020] Through the present invention, by acquiring a traffic light group image based on the regional position information of the traffic light group, and then performing multi-label target detection based on the traffic light group image, a higher detection recall rate can be achieved. In addition, by performing semantic segmentation on the area where the traffic lights with the target color in the traffic light group are located based on the color of the traffic lights in the traffic light group, the segmentation boundary of the result of the semantic segmentation is optimized, and the recognition result of the traffic light is output based on the result after the segmentation boundary optimization, it can be ensured that the human eye can clearly distinguish the traffic light area in the collected image. Therefore, the problem of accurate recognition of traffic lights can be solved. The recognition scheme of the traffic light has the characteristics of real-time and precision, and can well meet the requirements of intelligent transportation for high-precision traffic light recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a hardware structure block diagram of a monitoring server of a signal light recognition method according to an embodiment of the present invention;

[0022] Figure 2 is a flow chart of a method for identifying a signal light according to an embodiment of the present invention;

[0023] Figure 3 is a structural block diagram of a signal light recognition device according to an embodiment of the present invention;

[0024] Figure 4is an exemplary structural block diagram of a signal light recognition device according to an embodiment of the present invention;

[0025] Figure 5 is a schematic diagram of an application flow of a signal light recognition solution according to an embodiment of the present invention;

[0026] Figure 6 is a schematic diagram of equally dividing an image according to an embodiment of the present invention;

[0027] Figure 7 is a schematic diagram comparing image effects before and after filtering out atmospheric light according to an embodiment of the present invention;

[0028] Figure 8 is a schematic diagram of a boundary optimization network according to an embodiment of the present invention;

[0029] Fig. 9 is a schematic diagram of an edge mask, a direction mask and an edge repair mask according to an embodiment of the present invention;

[0030] Fig.10 is a schematic diagram of an edge repair process according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In the related technologies, the recognition of traffic lights has a wide range of application needs in many fields. For example, in the scene of capturing traffic violations, a series of sound and light equipment is needed to automatically capture and collect evidence of the behavior process of the violators on the scene. Red light running is an important branch of traffic violation penalties. The use of computer vision technology to analyze red light running behavior is currently an effective processing method. This method generally requires multiple frames of images within a certain period of time, and determines whether a red light is run according to the changes in the state of the traffic lights in the image and the vehicle's travel trajectory. The determination of red light running behavior requires the identification of traffic lights on the road, and the association of the states of traffic lights in multiple frames of images. A single image is not enough to determine whether a vehicle has run a red light. If the state of the traffic light in a certain frame is incorrectly identified, it will cause the entire red light running behavior to be biased, reducing the accuracy of the penalty.

[0032] It can be seen that the relevant application scenarios have high requirements for the recognition precision and accuracy of traffic lights, and the current signal light recognition solutions cannot meet the high recognition precision and accuracy. To this end, the embodiments of the present invention provide a signal light recognition method, device and computer-readable storage medium, which can realize high-precision real-time recognition of traffic lights.

[0033] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0035] The method embodiments provided in the embodiments of the present application can be executed in a device or device collection with computing functions such as a monitoring server, a monitoring platform, a monitoring device, etc. Taking running on a monitoring server as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a monitoring server of a signal light recognition method according to an embodiment of the present invention. Figure 1 As shown, the monitoring server may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a central processing unit CPU, a microprocessor MCU or a programmable logic device FPGA and the like) and a memory 104 for storing data, wherein the monitoring server may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the monitoring server. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0036] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the signal light recognition method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the monitoring server via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0037] The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network or a wired network provided by a communication provider of the monitoring server. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with other devices in the Internet by wireless means, and can also be a transceiver port, which is used to communicate with other devices in the Internet by wired means.

[0038] In this embodiment, a method for identifying a signal light running on a computing device such as a monitoring server, a monitoring platform, or a monitoring device is provided. Figure 2 is a flow chart of a method for identifying a signal light according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0039] Step S202: acquiring a signal light group image from a first monitoring image based on the regional location information of the signal light group.

[0040] In order to improve the accuracy of detecting the signal lights in the light group, in at least one exemplary embodiment, step S202 may include:

[0041] The target area location information is obtained by expanding the width and / or height of the area indicated by the area location information by a predetermined ratio or value;

[0042] A signal light group image is acquired from the first monitoring image based on the target area location information.

[0043] In a specific implementation, the first monitoring image is the image to be identified, which can generally be collected by a monitoring device, for example, by a monitoring camera installed at an intersection.

[0044] In a specific implementation, the area location information of the signal light group used in step S202 may be pre-stored area location information, or may be area location information acquired through regular detection.

[0045] For example, in at least one exemplary embodiment, the regional location information of the signal light group used in step S202 may be regional location information obtained by regular detection. Accordingly, before step S202, the method may further include: performing target detection of the signal light group based on the second monitoring image according to a predetermined detection period to obtain the regional location information of the signal light group. In this process, the second monitoring image can generally be collected by a monitoring device, for example, it can be a monitoring image collected in real time by the monitoring device when the predetermined detection period arrives. Since the location of the monitoring device is generally fixed, the monitored scene is relatively fixed, and the signal light group itself is also fixed, therefore, the predetermined detection period can be set to a slightly longer time, for example, 1 hour, and between the two target detections of the signal light group controlled by the predetermined detection period, the most recent target detection result (that is, the regional location information of the signal light group) can be used. Target detection of the signal light group can be performed based on the target detection network according to the characteristics of the signal light group. The specific target detection process can follow the method in the relevant technology, which will not be repeated here.

[0046] Since the light group in the second monitoring image is a very small target relative to the second monitoring image as a whole in some scenarios, for example, the height may be less than 12 pixels in a 900W pixel image, and the recall rate of direct detection on the large image is low, in this embodiment, in the process of performing target detection of the signal light group based on the second monitoring image according to a predetermined detection cycle, the input image can be segmented and sent to the target detection network for target detection in the following manner:

[0047] Acquire the second monitoring image according to the predetermined detection period;

[0048] Divide the second monitoring image into a plurality of sub-images. For example, the second monitoring image may be divided into four equal parts and sent to the target detection network respectively.

[0049] Performing target detection of the signal light group based on the multiple sub-images to obtain a signal light group detection frame in the multiple sub-images;

[0050] The signal light group detection frames in the multiple sub-images are restored to the second monitoring image for fusion to obtain the regional position information of the signal light group.

[0051] Considering the accuracy and completeness of detection, in at least one exemplary embodiment, there may be partial overlap between adjacent sub-images in the divided multiple sub-images. For example, in the above example, the sub-images divided into four equal parts may have a 1 / 4 intersection between each adjacent equal part. Accordingly, restoring the signal light group detection frame in the multiple sub-images to the second monitoring image for fusion, and obtaining the regional location information of the signal light group may include:

[0052] Restoring the signal light group detection frame to the second monitoring image based on the positions of the signal light group detection frames in the multiple sub-images;

[0053] In the case of overlapping or partially overlapping signal light group detection frames, the overlapping or partially overlapping signal light group detection frames are merged or deduplicated. In the process of merging or deduplicating, since the results existing at the edge of the sub-image may be truncated, the larger frame is preferentially selected as the main frame or the larger frame is preferentially selected to be retained;

[0054] The regional position information of the signal light group is obtained based on the merged or deduplicated signal light group detection frame.

[0055] Step S204: performing multi-label target detection based on the signal light group image to determine the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group.

[0056] Through multi-label target detection, the status of the traffic lights in the traffic light group can be detected from multiple dimensions. For example, the following three categories of standard rooms can be divided into multi-label target detection:

[0057] The first type of labels includes two categories: light groups and signal lights;

[0058] The second category of labels includes 5 categories: red, green, yellow, off, and mixed colors;

[0059] The third category of labels includes 15 categories: round light, digital light, left turn, left front, left turn and U-turn, right turn, right front, go straight, go straight and turn left, go straight and turn right, non-motor vehicle, non-motor vehicle turn left, pedestrian, progress bar, and unknown.

[0060] Considering that the red, green and yellow colors of the signal lights will all tend to be white under the influence of atmospheric light, resulting in reduced differentiation of the three colors of signal lights, which is prone to misidentification, before performing multi-label target detection in step S204, the light group area may be firstly subjected to atmospheric light filtering, thereby reducing the color cast effect of atmospheric light on the color of the signal lights and making the texture clearer. Therefore, in at least one exemplary embodiment, before performing multi-label target detection based on the signal light group image in step S204 to determine the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group, the method may further include:

[0061] The atmospheric light is filtered out of the signal light group image based on an atmospheric scattering model, wherein the atmospheric scattering model is: x represents the index of the pixel, I(x) represents the signal light group image before atmospheric light filtering, J(x) represents the signal light group image after atmospheric light filtering, A represents the atmospheric light value, and t(x) represents the transmittance at the pixel x. Among them, the atmospheric light value A and the transmittance t(x) are unknown parameters, which can be estimated by the dark channel prior theory.

[0062] Through this method, the atmospheric light in the light group area can be filtered out through the atmospheric scattering model and the dark channel method, thereby reducing the color cast effect of ambient light on the signal light.

[0063] Step S206: Based on the colors of the traffic lights in the traffic light group, semantic segmentation is performed on the area where the traffic lights with the target color in the traffic light group are located.

[0064] In order to improve the accuracy of semantic segmentation, in at least one exemplary embodiment, after performing semantic segmentation on the area where the traffic lights having the target color in the traffic light group are located based on the colors of the traffic lights in the traffic light group in step S206, the method may further include:

[0065] The result of the semantic segmentation is constrained based on the detection box of the traffic light corresponding to the target color of the multi-label target detection.

[0066] In at least one exemplary embodiment, constraining the result of the semantic segmentation based on the detection box of the traffic light corresponding to the target color of the multi-label target detection may include:

[0067] The results of the semantic segmentation that are outside the detection box of the traffic light corresponding to the target color are filtered out.

[0068] Step S208 , optimizing the segmentation boundary of the semantic segmentation result, and outputting the recognition result of the traffic light based on the segmentation boundary optimized result.

[0069] In at least one exemplary embodiment, in step S208, optimizing the segmentation boundary of the semantic segmentation result may specifically include:

[0070] Predicting an edge mask and a direction mask based on the result of the semantic segmentation, wherein the edge mask carries indication information for distinguishing edge pixels from non-edge pixels, and the direction mask carries indication information for indicating pixel directions;

[0071] generating an edge repair mask based on the edge mask and the direction mask;

[0072] The semantic segmentation result is mapped through the edge repair mask to obtain the segmentation boundary optimized result.

[0073] In at least one exemplary embodiment, in the edge mask, the indication information of the edge pixels can be set to 1, and the indication information of the non-edge pixels can be set to 0; generating an edge repair mask based on the edge mask and the direction mask can include: multiplying corresponding elements in the edge mask and the direction mask to obtain the edge repair mask.

[0074] In at least one exemplary embodiment, outputting the recognition result of the signal light based on the result after the segmentation boundary optimization in step S208 may include at least one of the following:

[0075] Based on the result of the segmentation boundary optimization, the traffic light having the target color is painted with the target color. In this method, since the segmentation boundary optimization has been performed on the segmentation result before, the coloring effect of painting the traffic light with the target color using this method will be more fitting;

[0076] The color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group is output.

[0077] In at least one exemplary embodiment, in the case where multi-label target detection is performed based on the signal light group image to determine the color and shape of the signal lights in the signal light group, outputting the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group may include:

[0078] Determining whether the signal light group is a mixed direction signal light group or a single direction signal light group;

[0079] When the signal light group is the mixed direction signal light group, outputting the color and shape of the signal lights in the signal light group;

[0080] When the signal light group is the one-way signal light group, the colors of the signal lights in the signal light group are output.

[0081] This method performs logical judgment based on the characteristics of traffic lights and outputs the final recognition result. By analyzing the prior characteristics of traffic lights, traffic lights are divided into motor vehicle lights, non-motor vehicle lights and pedestrian lights, and corresponding logical judgments are performed on each of them to output the final result.

[0082] In the above scheme, a traffic light group image is obtained based on the regional position information of the traffic light group, and multi-label target detection is performed based on the traffic light group image, so that a higher detection recall rate can be achieved. In addition, semantic segmentation is performed on the area where the traffic lights with the target color in the traffic light group are located based on the color of the traffic lights in the traffic light group, the segmentation boundary of the result of the semantic segmentation is optimized, and the recognition result of the traffic light is output based on the result after the segmentation boundary optimization. This can ensure that the human eye can clearly distinguish the traffic light area in the collected image. Therefore, the problem of accurate recognition of traffic lights can be solved. The traffic light recognition scheme has the characteristics of real-time and accuracy, and can well meet the requirements of intelligent transportation for high-precision traffic light recognition.

[0083] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0084] In this embodiment, a signal light recognition device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made are omitted. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0085] Figure 3 is a structural block diagram of a signal light recognition device according to an embodiment of the present invention. Figure 3 As shown, the device includes: an acquisition module 32, a multi-label detection module 34, a semantic segmentation module 36 and a boundary optimization and output module 38.

[0086] The acquisition module 32 is configured to acquire the signal light group image from the first monitoring image based on the regional location information of the signal light group.

[0087] In order to improve the accuracy of signal light detection in a light group, in at least one exemplary embodiment, the acquisition module 32 can be configured to: obtain target area location information by expanding the width and / or height of the area indicated by the area location information by a predetermined proportion or value; and obtain the signal light group image from the first monitoring image based on the target area location information.

[0088] In a specific implementation, the first monitoring image is the image to be identified, which can generally be collected by a monitoring device, for example, by a monitoring camera installed at an intersection.

[0089] In a specific implementation, the area location information of the signal light group used by the acquisition module 32 may be pre-stored area location information or area location information acquired through regular detection.

[0090] For example, in at least one exemplary embodiment, the area location information of the signal light group used by the acquisition module 32 may be area location information acquired through regular detection. Figure 4 is an exemplary structural block diagram of a signal light recognition device according to an embodiment of the present invention. Figure 4 As shown, the device may also include: a target detection module 42, which is configured to perform target detection of the signal light group based on the second monitoring image according to a predetermined detection cycle, and obtain the regional location information of the signal light group. In this process, the second monitoring image can generally be collected by a monitoring device, for example, it can be a monitoring image collected in real time by the monitoring device when the predetermined detection cycle arrives. Since the position of the monitoring device is generally fixed, the monitored scene is also relatively fixed, and the signal light group itself is also fixed, therefore, the predetermined detection cycle can be set to a slightly longer time, such as 1 hour, and between the two target detections of the signal light group controlled by the predetermined detection cycle, the most recent target detection result (that is, the regional location information of the signal light group) can be used. Target detection of the signal light group can be performed based on the target detection network according to the characteristics of the signal light group. The specific target detection process can follow the method in the relevant technology, which will not be repeated here.

[0091] Since the light group in the second monitoring image is a very small target relative to the second monitoring image as a whole in some scenarios, for example, the height may be less than 12 pixels in a 900W pixel image, and the recall rate of direct detection on the large image is low, in this embodiment, when the target detection module 42 performs target detection of the signal light group based on the second monitoring image according to a predetermined detection cycle, the input image can be segmented and sent to the target detection network for target detection in the following manner:

[0092] Acquire the second monitoring image according to the predetermined detection period;

[0093] Divide the second monitoring image into a plurality of sub-images. For example, the second monitoring image may be divided into four equal parts and sent to the target detection network respectively.

[0094] Performing target detection of the signal light group based on the multiple sub-images to obtain a signal light group detection frame in the multiple sub-images;

[0095] The signal light group detection frames in the multiple sub-images are restored to the second monitoring image for fusion to obtain the regional position information of the signal light group.

[0096] Considering the accuracy and completeness of detection, in at least one exemplary embodiment, there may be partial overlap between adjacent sub-images in the divided multiple sub-images. For example, in the above example, the sub-images divided into four equal parts may have a 1 / 4 intersection between each adjacent equal part. Accordingly, restoring the signal light group detection frame in the multiple sub-images to the second monitoring image for fusion, and obtaining the regional location information of the signal light group may include:

[0097] Restoring the signal light group detection frame to the second monitoring image based on the positions of the signal light group detection frames in the multiple sub-images;

[0098] In the case of overlapping or partially overlapping signal light group detection frames, the overlapping or partially overlapping signal light group detection frames are merged or deduplicated. In the process of merging or deduplicating, since the results existing at the edge of the sub-image may be truncated, the larger frame is preferentially selected as the main frame or the larger frame is preferentially selected to be retained;

[0099] The regional position information of the signal light group is obtained based on the merged or deduplicated signal light group detection frame.

[0100] The multi-label detection module 34 is configured to perform multi-label target detection based on the signal light group image to determine the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group.

[0101] Through multi-label target detection, the status of the traffic lights in the traffic light group can be detected from multiple dimensions. For example, the following three categories of standard rooms can be divided into multi-label target detection:

[0102] The first type of labels includes two categories: light groups and signal lights;

[0103] The second category of labels includes 5 categories: red, green, yellow, off, and mixed colors;

[0104] The third category of labels includes 15 categories: round light, digital light, left turn, left front, left turn and U-turn, right turn, right front, go straight, go straight and turn left, go straight and turn right, non-motor vehicle, non-motor vehicle turn left, pedestrian, progress bar, and unknown.

[0105] Considering that the red, green and yellow colors of the signal lights will all tend to be white under the influence of atmospheric light, resulting in reduced differentiation of the three colors of signal lights, which is prone to misidentification, the multi-label detection module 34 can filter out atmospheric light from the light group area before performing multi-label target detection, thereby reducing the color cast effect of atmospheric light on the color of the signal lights and making the texture clearer. Therefore, in at least one exemplary embodiment, before performing multi-label target detection based on the signal light group image to determine the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group, the multi-label detection module 34 can also be set to:

[0106] The atmospheric light is filtered out of the signal light group image based on an atmospheric scattering model, wherein the atmospheric scattering model is: x represents the index of the pixel, I(x) represents the signal light group image before atmospheric light filtering, J(x) represents the signal light group image after atmospheric light filtering, A represents the atmospheric light value, and t(x) represents the transmittance at the pixel x. Among them, the atmospheric light value A and the transmittance t(x) are unknown parameters, which can be estimated by the dark channel prior theory.

[0107] Through this solution, the atmospheric light in the light group area can be filtered out through the atmospheric scattering model and the dark channel method, thereby reducing the color cast effect of ambient light on the signal light.

[0108] The semantic segmentation module 36 is configured to perform semantic segmentation on the area where the traffic lights with the target color in the traffic light group are located based on the colors of the traffic lights in the traffic light group.

[0109] In order to improve the accuracy of semantic segmentation, in at least one exemplary embodiment, after semantic segmentation of the area where the traffic lights having the target color in the traffic light group are located is performed by the semantic segmentation module 36 based on the colors of the traffic lights in the traffic light group, the semantic segmentation module 36 may also be set as follows:

[0110] The result of the semantic segmentation is constrained based on the detection box of the traffic light corresponding to the target color of the multi-label target detection.

[0111] In at least one exemplary embodiment, constraining the result of the semantic segmentation based on the detection box of the traffic light corresponding to the target color of the multi-label target detection may include:

[0112] The results of the semantic segmentation that are outside the detection box of the traffic light corresponding to the target color are filtered out.

[0113] The boundary optimization and output module 38 is configured to perform segmentation boundary optimization on the result of the semantic segmentation, and output the recognition result of the traffic light based on the result after the segmentation boundary optimization.

[0114] In at least one exemplary embodiment, the boundary optimization and output module 38 may be specifically configured to:

[0115] Predicting an edge mask and a direction mask based on the result of the semantic segmentation, wherein the edge mask carries indication information for distinguishing edge pixels from non-edge pixels, and the direction mask carries indication information for indicating pixel directions;

[0116] generating an edge repair mask based on the edge mask and the direction mask;

[0117] The semantic segmentation result is mapped through the edge repair mask to obtain the segmentation boundary optimized result.

[0118] In at least one exemplary embodiment, in the edge mask, the indication information of the edge pixels can be set to 1, and the indication information of the non-edge pixels can be set to 0; generating an edge repair mask based on the edge mask and the direction mask can include: multiplying corresponding elements in the edge mask and the direction mask to obtain the edge repair mask.

[0119] In at least one exemplary embodiment, the boundary optimization and output module 38 may also be configured to perform at least one of the following:

[0120] Based on the result of the segmentation boundary optimization, the traffic light having the target color is painted with the target color. In this method, since the segmentation boundary optimization has been performed on the segmentation result before, the coloring effect of painting the traffic light with the target color using this method will be more fitting;

[0121] The color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group is output.

[0122] In at least one exemplary embodiment, when performing multi-label object detection based on the signal light group image to determine the color and shape of the signal lights in the signal light group, the boundary optimization and output module 38 may be configured to output the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group, in the following manner:

[0123] Determining whether the signal light group is a mixed direction signal light group or a single direction signal light group;

[0124] When the signal light group is the mixed direction signal light group, outputting the color and shape of the signal lights in the signal light group;

[0125] When the signal light group is the one-way signal light group, the colors of the signal lights in the signal light group are output.

[0126] This method performs logical judgment based on the characteristics of traffic lights and outputs the final recognition result. By analyzing the prior characteristics of traffic lights, traffic lights are divided into motor vehicle lights, non-motor vehicle lights and pedestrian lights, and corresponding logical judgments are performed on each of them to output the final result.

[0127] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0128] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0129] In this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0130] S1, acquiring a signal light group image from a first monitoring image based on regional location information of the signal light group;

[0131] S2, performing multi-label target detection based on the signal light group image to determine the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group;

[0132] S3, based on the colors of the traffic lights in the traffic light group, performing semantic segmentation on the area where the traffic lights with the target color in the traffic light group are located;

[0133] S4, optimizing the segmentation boundary of the semantic segmentation result, and outputting the recognition result of the traffic light based on the segmentation boundary optimized result.

[0134] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0135] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0136] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0137] In an exemplary embodiment, the processor may be configured to perform the following steps via a computer program:

[0138] S1, acquiring a signal light group image from a first monitoring image based on regional location information of the signal light group;

[0139] S2, performing multi-label target detection based on the signal light group image to determine the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group;

[0140] S3, based on the colors of the traffic lights in the traffic light group, performing semantic segmentation on the area where the traffic lights with the target color in the traffic light group are located;

[0141] S4, optimizing the segmentation boundary of the semantic segmentation result, and outputting the recognition result of the traffic light based on the segmentation boundary optimized result.

[0142] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0143] The following describes in detail a signal light recognition solution according to an embodiment of the present invention based on a specific application scenario.

[0144] With the continuous development of society and the continuous advancement of science and technology, the intelligent management of urban road traffic is becoming more and more mature. Traditional traffic signal light receivers require wiring and additional hardware facilities, which causes many inconveniences. In addition, many intelligent traffic violation capture systems are highly dependent on the detection results of traffic lights. Errors in traffic signal light recognition results will lead to the miscapture of violations, which seriously affects the actual performance of the intelligent traffic violation capture system. Therefore, realizing a high-precision real-time traffic signal light recognition system meets the current urgent needs. This embodiment provides a high-precision real-time traffic signal light recognition method.

[0145] Figure 5 FIG. 1 is a schematic diagram of an application flow of a signal light recognition solution according to an embodiment of the present invention. Figure 5As shown in the figure, the solution includes five process steps: input video sequence, signal light group detection based on large image, signal light group and small light detection based on signal light area, red small light coloring, and output signal light recognition result. Detailed examples are given below to illustrate each step.

[0146] (1) Traffic light group detection based on large images

[0147] First, since the light groups in some scenes are extremely small targets in the image, the height of a 900W pixel image is less than 12 pixels, and the recall rate of direct detection on the large image is low, so the input image is divided into 4 equal parts, and there is a 1 / 4 intersection between each adjacent equal part, and they are sent to the target detection network (such as yolov4-tiny) respectively. Figure 6 is a schematic diagram of equally dividing an image according to an embodiment of the present invention.

[0148] Secondly, the light group positioning coordinates obtained from the four sub-images are restored to the original image for fusion. Non-Maximum Suppression (NMS) is used to filter out overlapping detection frames (larger frames are preferred because the results at the edge of the sub-image may be truncated). Finally, the light group detection results based on the large image are obtained. Since target detection based on the large image is time-consuming, it cannot be performed for each frame. In addition, according to the video signal, the position of the light will only change due to camera shake, so the algorithm is executed once every hour to correct the light group coordinates.

[0149] (2) Light group and small light detection based on signal light area

[0150] First, the height and width of the light group detection results based on the large image are expanded by 1 / 2 the width of the light group.

[0151] Secondly, under the influence of atmospheric light, the red, green and yellow colors of the signal light will tend to be white, which reduces the distinction between the three colors of signal lights and easily leads to misidentification. Therefore, the atmospheric light is filtered in this area to reduce the color cast caused by atmospheric light on the color of the signal light and make the texture clearer. The atmospheric light filtering method adopts the atmospheric scattering model as shown in formula (1):

[0152]

[0153] Where I(x) is the original image, such as Figure 3 In the image on the left, A is the atmospheric light value, t(x) is the transmittance, and J(x) is the image after filtering out the atmospheric light. x represents each pixel. Here, only the atmospheric light value A and the transmittance t(x) are unknown parameters, which can be estimated through the dark channel prior theory, so that the image J(x) after filtering out the atmospheric light can be calculated. Figure 7 is a schematic diagram comparing image effects before and after filtering out atmospheric light according to an embodiment of the present invention, Figure 7 The image on the left is the original image I(x) before the atmospheric light is filtered out. Figure 7 The image on the right is the image J(x) after the atmospheric light is filtered out.

[0154] Next, multi-label target detection is performed on the image J(x) after filtering out atmospheric light. The first category of labels includes light group and small light, which has 2 categories in total. The second category of labels includes red, green, yellow, light off, and mixed colors, which has 5 categories in total. The third category of labels includes round light, digital light, left turn, left front, left turn and U-turn, right turn, right front, straight ahead, straight ahead and turn left, straight ahead and turn right, non-motor vehicle, non-motor vehicle turn left, pedestrian, progress bar, and unknown, which has 15 categories in total.

[0155] (3) Paint the small red light red

[0156] First, semantic segmentation is performed on the red small lights in each light group area detected, such as the U-NET algorithm. The segmentation results are first constrained by the red small light frames contained in the light group in the previous detection results, and the segmentation results outside the red small light frames are filtered out.

[0157] Secondly, in order to make the red light more closely matched, the segmentation boundary is optimized. Figure 8 is a schematic diagram of a boundary optimization network according to an embodiment of the present invention. Figure 8 As shown in the figure, the main network consists of a CNN network and an encoder and decoder. An edge segmentation branch is used to predict the edge mask, namely the Boundary Map. The edge pixels are set to 1, and the pixels in the remaining area are set to 0. At the same time, the direction category of each pixel is predicted through the direction branch to obtain the direction mask (Direction Map). The edge mask Boundary Map and the direction mask Direction Map are multiplied by the corresponding elements to obtain the edge repair mask. Fig. 9 is a schematic diagram of an edge mask, a direction mask, and an edge repair mask according to an embodiment of the present invention. Fig.10 is a schematic diagram of an edge repair process according to an embodiment of the present invention, such as Fig.10 As shown in the figure, the segmentation result is mapped through the edge repair mask to obtain the edge repair result. The repaired effect is as follows Figure 5 As shown in the lower right picture.

[0158] (4) Output signal light recognition results

[0159] Traffic lights are mainly divided into three categories, namely motor vehicle lights, non-motor vehicle lights, and pedestrian lights. The detection results that include non-motor vehicles and non-motor vehicle left turn categories are classified as non-motor vehicle categories, those that include pedestrian categories are classified as pedestrian categories, and the rest are motor vehicle lights.

[0160] The following describes the three types of signal light result output strategies:

[0161] Motor vehicle signal lights: There are two situations, one-way signal light group and mixed-way signal light group. By judging whether the detection result contains multiple small light detection results in different directions and the color of the light group is not a mixed color, it is calculated whether the light group is a mixed-way light group. If it is not a mixed light group, the color category of the light group is output directly (because the light group contains the phase information of the small lights, the color result of the light group is more accurate). If it is a mixed light group, the color direction results of the small lights included are output in sequence.

[0162] Non-motor vehicle signal lights: There are two situations, one-way signal light group and mixed-way signal light group. By judging whether the detection result contains multiple small light detection results in different directions and the color of the light group is not a mixed color, it is calculated whether the light group is a mixed-way light group. If it is not a mixed light group, the color category of the light group is output directly (because the light group contains the phase information of the small lights, the color result of the light group is more accurate). If it is a mixed light group, the color direction results of the small lights included are output in sequence.

[0163] Pedestrian signal light: There is only a single-direction signal light group, and the light group color detection result is directly output.

[0164] Through this high-precision real-time traffic light recognition method, first, a target detection method based on large-scale image segmentation and fusion is performed once every hour to provide a light group area to meet the high-precision requirements of the light group area coordinates. Secondly, the atmospheric light is filtered out after the light group area is expanded, which can also have good robustness in complex ambient light scenes. The light group and small lights are then detected in the area. Then, the red small lights are painted red through target segmentation and edge repair algorithms to ensure that the human eye can clearly distinguish the signal light area of ​​the collected image. Finally, according to the prior characteristics of the traffic light, the signal light is logically judged and the final recognition result is output. This method is used for traffic light recognition, has real-time and accurate characteristics, and has high robustness for complex ambient light scenes, meeting the high-precision requirements of intelligent transportation for signal light recognition.

[0165] In summary, the solution of the embodiment of the present invention has the following advantages:

[0166] 1. This solution provides a high-precision real-time traffic light recognition method. First, a target detection method based on image segmentation and fusion of a large image is performed every hour to provide a light group area. Secondly, the atmospheric light in the light group area is filtered out, and then the light group and small lights are detected in the area. Then, the red small lights are painted red through the target segmentation and edge repair algorithm. Finally, the final recognition result is output based on the logic judgment of the traffic light characteristics.

[0167] 2. The target detection method based on large-scale image segmentation and fusion is to segment the image into four equal parts with 1 / 4 intersection, perform target detection on each part, and then fuse the detection results.

[0168] 3. Filter out the atmospheric light in the light group area through the atmospheric scattering model and dark channel method, thereby reducing the color cast effect of ambient light on the signal light;

[0169] 4. The target segmentation and edge repair algorithm paints the red light red by repairing the edge of the target segmentation result, so that the red-painted pixels are more in line with the actual red light pixels;

[0170] 5. The final recognition result is output by logical judgment based on the characteristics of traffic lights. This is achieved by analyzing the prior characteristics of traffic lights, dividing traffic lights into motor vehicle lights, non-motor vehicle lights and pedestrian lights, and performing corresponding logical judgments on each of them to output the final result.

[0171] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0172] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying a signal light, characterized in that: include: Acquire a signal light group image from the first monitoring image based on the regional location information of the signal light group; Performing multi-label target detection based on the signal light group image to determine the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group; Based on the colors of the traffic lights in the traffic light group, semantic segmentation is performed on the area where the traffic lights with the target color in the traffic light group are located; Optimizing the segmentation boundary of the semantic segmentation result, and outputting the recognition result of the traffic light based on the segmentation boundary optimized result; Outputting the recognition result of the traffic light based on the result after the segmentation boundary optimization includes at least one of the following: painting the traffic light having the target color with the target color based on the result after the segmentation boundary optimization; outputting the color of the traffic light in the traffic light group, or the color and shape of the traffic light in the traffic light group.

2. The method according to claim 1, characterized in that: Before acquiring the signal light group image from the first monitoring image based on the regional location information of the signal light group, the method further includes: According to a predetermined detection cycle, target detection of the signal light group is performed based on the second monitoring image to obtain the regional position information of the signal light group.

3. The method according to claim 2, characterized in that According to a predetermined detection cycle, target detection of the signal light group is performed based on the second monitoring image, and the regional location information of the signal light group is obtained, including: Acquire the second monitoring image according to the predetermined detection period; dividing the second monitoring image into a plurality of sub-images; Performing target detection of the signal light group based on the multiple sub-images to obtain a signal light group detection frame in the multiple sub-images; The signal light group detection frames in the multiple sub-images are restored to the second monitoring image for fusion to obtain the regional position information of the signal light group.

4. The method according to claim 3, characterized in that There is partial overlap between adjacent sub-images in the divided multiple sub-images, and restoring the signal light group detection frame in the multiple sub-images to the second monitoring image for fusion, and obtaining the regional position information of the signal light group includes: Restoring the signal light group detection frame to the second monitoring image based on the positions of the signal light group detection frames in the multiple sub-images; In the case where there are overlapping or partially overlapping signal light group detection frames, merging or deduplicating the overlapping or partially overlapping signal light group detection frames; The regional position information of the signal light group is obtained based on the merged or deduplicated signal light group detection frame.

5. The method according to claim 1, characterized in that Acquiring the signal light group image from the first monitoring image based on the regional location information of the signal light group includes: The target area location information is obtained by expanding the width and / or height of the area indicated by the area location information by a predetermined ratio or value; A signal light group image is acquired from the first monitoring image based on the target area location information.

6. The method according to claim 1, characterized in that Before performing multi-label target detection based on the signal light group image to determine the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group, the method further includes: The atmospheric light is filtered out of the signal light group image based on an atmospheric scattering model, wherein the atmospheric scattering model is: x represents the index of the pixel point, I(x) represents the signal light group image before atmospheric light filtering, J(x) represents the signal light group image after atmospheric light filtering, A represents the atmospheric light value, and t(x) represents the transmittance at the pixel point x.

7. The method according to claim 1, characterized in that After semantic segmentation is performed on the area where the traffic lights with the target color in the traffic light group are located based on the colors of the traffic lights in the traffic light group, the method further includes: The result of the semantic segmentation is constrained based on the detection box of the traffic light corresponding to the target color of the multi-label target detection.

8. The method according to claim 1, characterized in that Constraining the result of the semantic segmentation based on the detection frame of the traffic light corresponding to the target color of the multi-label target detection includes: The results of the semantic segmentation that are outside the detection box of the traffic light corresponding to the target color are filtered out.

9. The method according to claim 1, characterized in that: Optimizing the segmentation boundary of the semantic segmentation result includes: Predicting an edge mask and a direction mask based on the result of the semantic segmentation, wherein the edge mask carries indication information for distinguishing edge pixels from non-edge pixels, and the direction mask carries indication information for indicating pixel directions; generating an edge repair mask based on the edge mask and the direction mask; The semantic segmentation result is mapped through the edge repair mask to obtain the segmentation boundary optimized result.

10. The method according to claim 9, characterized in that In the edge mask, the indication information of the edge pixel is set to 1, and the indication information of the non-edge pixel is set to 0; Generating an edge repair mask based on the edge mask and the direction mask includes: multiplying corresponding elements in the edge mask and the direction mask to obtain the edge repair mask.

11. The method according to claim 1, characterized in that: In the case where multi-label target detection is performed based on the signal light group image to determine the color and shape of the signal lights in the signal light group, outputting the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group includes: Determining whether the signal light group is a mixed direction signal light group or a single direction signal light group; When the signal light group is the mixed direction signal light group, outputting the color and shape of the signal lights in the signal light group; When the signal light group is the one-way signal light group, the colors of the signal lights in the signal light group are output.

12. A signal light recognition device, characterized in that: include: an acquisition module, configured to acquire a signal light group image from a first monitoring image based on the regional location information of the signal light group; a multi-label detection module, configured to perform multi-label target detection based on the signal light group image to determine the color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group; A semantic segmentation module, configured to perform semantic segmentation on a region where a signal light having a target color in the signal light group is located based on the color of the signal lights in the signal light group; A boundary optimization and output module, configured to perform segmentation boundary optimization on the result of the semantic segmentation, and output the recognition result of the traffic light based on the result after the segmentation boundary optimization; The output module outputs the recognition result of the signal light based on the result after the segmentation boundary optimization by at least one of the following methods: painting the signal light having the target color with the target color based on the result after the segmentation boundary optimization; The color of the signal lights in the signal light group, or the color and shape of the signal lights in the signal light group is output.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 11 when executed.

Citation Information

Patent Citations

  • System for real-time identifying urban traffic lights based on single eye vision and GPS integrated navigation system

    CN105930819A

  • Traffic signal lamp identification method and device

    CN107644538A

  • Domain adaptation via class-balanced self-training with spatial priors

    CN109724608A

  • Traffic signal lamp identification method and system based on hybrid model

    CN110069986A

  • Traffic signal lamp identification method and device, electronic equipment and storage medium

    CN111079563A