Signal lamp identification method and device, electronic equipment and storage medium

Abstract

The invention provides a signal lamp identification method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a signal lamp picture in front of a target vehicle to obtain a target signal lamp picture; acquiring a target vehicle position and a target vehicle driving direction; performing signal lamp detection on the target signal lamp picture through a signal lamp detection model to obtain candidate signal lamps; performing type reading on type mapping information stored in the intersection knowledge base according to the target vehicle position and the target vehicle driving direction to obtain a target signal lamp arrangement type; the intersection knowledge base also stores candidate signal lamp arrangement types and candidate signal lamp identification information; if the target signal lamp arrangement type is the same as the candidate signal lamp arrangement type, determining the candidate signal lamp identification information as target signal lamp identification information; and performing signal lamp identification on the candidate signal lamp according to the target signal lamp identification information to obtain a target signal lamp. The accuracy of signal lamp identification can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to a traffic signal recognition method and device, an electronic device, and a storage medium. Background Art

[0002] In related technologies, the red-light running detection technology can be used to detect whether a vehicle runs a red light, so as to give a reminder of running a red light, and further realize the driving management of the vehicle.

[0003] The red-light running detection technology in related technologies mainly includes: performing traffic signal recognition based on pictures of vehicles on roads, intersections, etc., and judging whether the vehicle runs a red light based on the recognized traffic signal. However, this solution is limited by the picture quality of the collected images. For example, the image is unclear due to vehicle movement, or the picture is blurred due to climatic reasons such as rain, snow, or cloudy days, and it is difficult to find the traffic signal in the driving direction of the vehicle.

[0004] The disadvantage of related technologies is that the accuracy of traffic signal recognition is relatively low. Summary of the Invention

[0005] The main purpose of the embodiments of this application is to propose a vehicle red-light running detection method and device, an electronic device, and a storage medium, which can improve the accuracy of traffic signal recognition.

[0006] To achieve the above object, a first aspect of the embodiments of this application proposes a traffic signal recognition method, and the method includes:

[0007] Obtain a traffic signal picture in front of the target vehicle to obtain a target traffic signal picture;

[0008] Obtain the position of the target vehicle to obtain a target vehicle position;

[0009] Obtain the driving direction of the target vehicle to obtain a target vehicle driving direction;

[0010] Perform traffic signal detection on the target traffic signal picture through a preset traffic signal detection model to obtain candidate traffic signals;

[0011] Read the type mapping information stored in the preset intersection knowledge base according to the target vehicle position and the target vehicle driving direction to obtain a target traffic signal arrangement type; the intersection knowledge base also stores candidate type annotation information, and the candidate type annotation information includes a candidate traffic signal arrangement type and candidate traffic signal recognition information;

[0012] If the target traffic signal arrangement type is the same as the candidate traffic signal arrangement type, then determine the candidate traffic signal recognition information as the target traffic signal recognition information;

[0013] Perform signal light recognition on the candidate signal lights according to the target signal light recognition information to obtain the target signal lights; wherein, the target signal lights are the signal lights of the target vehicle in the driving direction of the target vehicle.

[0014] In some embodiments, before reading the type mapping information stored in the preset intersection knowledge base according to the target vehicle position and the target vehicle driving direction to obtain the target signal light arrangement type, the method further includes:

[0015] Constructing the intersection knowledge base includes:

[0016] Obtain the signal light picture in front of the sample vehicle during driving to obtain the sample signal light picture;

[0017] Perform annotation based on the sample signal light picture to obtain the candidate signal light arrangement type and the candidate signal light recognition information;

[0018] Obtain the position of the sample vehicle to obtain the sample vehicle position, and obtain the driving direction of the sample vehicle to obtain the sample vehicle driving direction;

[0019] Perform first information association on the sample vehicle position, the sample vehicle driving direction, and the candidate signal light arrangement type to obtain the type mapping information;

[0020] Perform second information association on the candidate signal light arrangement type and the candidate signal light recognition information to obtain the candidate type annotation information;

[0021] Perform information merging based on the type mapping information and the candidate type annotation information to obtain the intersection knowledge base.

[0022] In some embodiments, the performing annotation based on the sample signal light picture to obtain the candidate signal light arrangement type and the candidate signal light recognition information includes:

[0023] Perform signal light detection on the sample signal light picture through the signal light detection model to obtain sample candidate signal lights;

[0024] Perform signal light arrangement type annotation based on the sample candidate signal lights to obtain the candidate signal light arrangement type;

[0025] Perform signal light screening on the sample candidate signal lights based on the sample vehicle driving direction to obtain sample target signal lights; wherein, the sample target signal lights are the signal lights of the sample vehicle in the sample vehicle driving direction;

[0026] Perform signal light annotation based on the candidate signal light arrangement type and the sample target signal light to obtain the candidate signal light recognition information; wherein, the candidate signal light recognition information represents information for recognizing the sample target signal light from the sample candidate signal lights.

[0027] In some embodiments, the step of performing signal light screening on the sample candidate signal lights based on the sample vehicle driving direction to obtain the sample target signal light includes:

[0028] Group the sample candidate signal lights to obtain candidate signal light groups;

[0029] Determine the central axis of the sample signal light image to obtain the target central axis;

[0030] Calculate the distance between the candidate signal light group and the target central axis to obtain the target distance;

[0031] Determine the candidate signal light group with the minimum target distance as the target signal light group;

[0032] Perform screening on the sample candidate signal lights in the target signal light group based on the sample vehicle driving direction to obtain the sample target signal light.

[0033] In some embodiments, the step of performing signal light annotation based on the candidate signal light arrangement type and the sample target signal light to obtain the candidate signal light recognition information includes:

[0034] Perform position annotation on the sample target signal light based on the target signal light group to obtain the signal light position; wherein, the signal light position includes the signal light arrangement direction and the signal light position mark; wherein, the signal light arrangement direction represents the arrangement direction of the sample candidate signal lights in the target signal light group, and the signal light position mark represents the position where the sample target signal light is located in the signal light arrangement direction;

[0035] If the candidate signal light arrangement type is the first type, perform first information merging on the signal light arrangement direction and the signal light position mark to obtain the candidate signal light recognition information; wherein, the first type represents that the sample candidate signal lights in the target signal light group are arranged in the signal light arrangement direction.

[0036] In some embodiments, the step of performing signal light annotation based on the candidate signal light arrangement type and the sample target signal light to obtain the candidate signal light recognition information further includes:

[0037] If the candidate signal light arrangement type is the second type, perform light group annotation on the candidate signal light group to obtain signal light group information; wherein, the second type indicates that there are at least two candidate signal light groups, and the sample candidate signal lights in the target signal light group are arranged in the signal light arrangement direction, and the signal light group information represents the information for finding the target signal light group from the candidate signal light groups;

[0038] Perform second information merging on the signal light group information, the signal light arrangement direction, and the signal light position marking to obtain the candidate signal light recognition information.

[0039] In some embodiments, the grouping of the sample candidate signal lights to obtain candidate signal light groups includes:

[0040] Establish a coordinate system with a preset first direction as the abscissa direction and a direction perpendicular to the first direction as the ordinate direction;

[0041] Based on any two of the sample candidate signal lights, obtain a first sample candidate signal light and a second sample candidate signal light;

[0042] In the coordinate system, obtain the coordinates of the first sample candidate signal light to get a first abscissa and a first ordinate, and obtain the coordinates of the second sample candidate signal light to get a second abscissa and a second ordinate;

[0043] If the first abscissa is the same as the second abscissa, add the first sample candidate signal light and the second sample candidate signal light to the same candidate signal light group;

[0044] If the first ordinate is the same as the second ordinate, add the first sample candidate signal light and the second sample candidate signal light to the same candidate signal light group.

[0045] To achieve the above object, a second aspect of the embodiments of the present application proposes a signal light recognition device, the device includes:

[0046] An image acquisition module, configured to acquire a signal light image in front of the target vehicle to obtain a target signal light image;

[0047] A position acquisition module, configured to acquire the position of the target vehicle to obtain a target vehicle position;

[0048] A direction acquisition module, configured to acquire the driving direction of the target vehicle to obtain a target vehicle driving direction;

[0049] A detection module, configured to perform signal light detection on the target signal light image through a preset signal light detection model to obtain candidate signal lights;

[0050] A type reading module, configured to perform type reading on type mapping information stored in a preset intersection knowledge base according to the target vehicle position and the target vehicle driving direction, so as to obtain a target signal light arrangement type; the intersection knowledge base further stores candidate type annotation information, and the candidate type annotation information includes a candidate signal light arrangement type and candidate signal light recognition information;

[0051] An information determination module, configured to, if the target signal light arrangement type is the same as the candidate signal light arrangement type, determine the candidate signal light recognition information as the target signal light recognition information;

[0052] A recognition module, configured to perform signal light recognition on the candidate signal lights according to the target signal light recognition information, so as to obtain a target signal light; wherein, the target signal light is the signal light in the driving direction of the target vehicle.

[0053] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the signal light recognition method described in the first aspect above is implemented.

[0054] To achieve the above object, a fourth aspect of the embodiments of the present application provides a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the signal light recognition method described in the first aspect above is implemented.

[0055] The embodiments of the present application provide a signal light recognition method, a device, an electronic device, and a storage medium. In the embodiments of the present application, an intersection knowledge base is constructed in advance, and the intersection knowledge base includes type mapping information and candidate type annotation information. The type mapping information includes candidate signal light arrangement types related to the position and driving direction. The candidate type annotation information includes candidate signal light recognition information related to the candidate signal light arrangement type. Based on the position and driving direction of the target vehicle, the target signal light arrangement type can be read from the type mapping information, and then based on the target signal light arrangement type, the target signal light recognition information can be read from the candidate type annotation information. Then, the candidate signal lights on the target signal light picture are subjected to signal light recognition by using the target signal light recognition information, so as to obtain the target signal light. It can be seen that the embodiments of the present application use the intersection knowledge base to provide effective recognition information for how to recognize signal lights, and can accurately recognize the signal lights in the driving direction of the command / indication vehicle. While improving the accuracy of signal light recognition, it can also accurately and timely perform red light running detection based on the recognized signal lights, thereby reducing the pressure of manual red light running detection.

[0056] Other features and advantages of the present application will be described in the following specification, and in part will become apparent from the specification or will be understood by implementing the present application. The objectives and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the specification, claims and drawings. Description of the Drawings

[0057] Figure 1 is a flowchart of the traffic signal recognition method provided by an embodiment of the present application;

[0058] Figure 2 is a flowchart of the traffic signal recognition method provided by another embodiment of the present application;

[0059] Figure 3 is Figure 2 a flowchart of step 202 in

[0060] Figure 4 is Figure 3 a flowchart of step 303 in

[0061] Figure 5 is Figure 3 a flowchart of step 304 in

[0062] Figure 6A is a schematic diagram of the traffic signal arrangement direction from left to right;

[0063] Figure 6B is a schematic diagram of the traffic signal arrangement direction from right to left;

[0064] Figure 6C is a schematic diagram of the traffic signal arrangement direction from bottom to top;

[0065] Figure 6D is a schematic diagram of the traffic signal arrangement direction from top to bottom;

[0066] Figure 7 is Figure 3 a flowchart of step 304 in

[0067] Figure 8 is a block diagram of the module structure of the traffic signal recognition device provided by an embodiment of the present application;

[0068] Figure 9 is a schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present application. Detailed Embodiments

[0069] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] It should be noted that although the functional modules are divided in the schematic diagram of the device and the logical sequence is shown in the flowchart, in some cases, the steps shown or described may be executed in a different module division from that in the device or a different order from that in the flowchart. Terms such as "first", "second", etc. in the description, claims and the above-mentioned drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0072] First, several terms involved in this application are analyzed as follows:

[0073] Artificial intelligence (AI): It is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence; artificial intelligence is a branch of computer science. Artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. The research in this field includes robots, speech recognition, image recognition, natural language processing and expert systems, etc. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence also refers to the theory, method, technology and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0074] Image recognition: It refers to the process of analyzing and recognizing images using computer vision technology. Through image recognition technology, a computer can recognize objects, scenes, text, etc. in an image, so as to achieve automated image understanding and classification. Image recognition technology has a wide range of applications in fields such as object detection.

[0075] In the related art, due to limitations such as the picture quality of the collected images, such as motion blur in the picture, or the picture being blurred due to rain / cloudy days, it is very difficult to find the traffic lights in the current driving direction, and thus it is impossible to accurately and timely give a reminder of running a red light.

[0076] The traffic light recognition method, device, electronic device and storage medium provided by the embodiments of this application aim to realize traffic light recognition by means of the traffic light recognition information in a pre-constructed intersection knowledge base, reduce the influence of poor picture quality of the image or complex and diverse traffic lights on the recognition accuracy, improve the accuracy of traffic light recognition, and thus can accurately and timely give a reminder of running a red light.

[0077] The signal light recognition method provided by the embodiments of the present application is applied to the server side, or can also be software running on the server side. The server side can be configured as an independent physical server, or can be configured as a server cluster or distributed system composed of multiple physical servers, or can also be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the signal light recognition method, etc., but is not limited to the above forms.

[0078] The present application can be used in many general or specific computer system environments or configurations. For example: server computers, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, distributed computing environments including any of the above systems or devices, and so on. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0079] The embodiments of the present application provide a signal light recognition method, device, electronic device, and storage medium, which will be specifically described through the following embodiments. First, the signal light recognition method in the embodiments of the present application will be described.

[0080] Figure 1 It is an optional flowchart of the signal light recognition method provided by the embodiments of the present application, which may include but is not limited to steps 101 to 107.

[0081] Step 101, obtain a signal light picture in front of the target vehicle to obtain a target signal light picture;

[0082] Step 102, obtain the position of the target vehicle to obtain the target vehicle position;

[0083] Step 103, obtain the driving direction of the target vehicle to obtain the target vehicle driving direction;

[0084] Step 104, perform signal light detection on the target signal light picture through a preset signal light detection model to obtain candidate signal lights;

[0085] Step 105: Read the type mapping information stored in the preset intersection knowledge base according to the target vehicle position and the target vehicle driving direction to obtain the target signal light arrangement type. The intersection knowledge base also stores candidate type annotation information, and the candidate type annotation information includes candidate signal light arrangement types and candidate signal light recognition information.

[0086] Step 106: If the target signal light arrangement type is the same as the candidate signal light arrangement type, determine the candidate signal light recognition information as the target signal light recognition information.

[0087] Step 107: Perform signal light recognition on the candidate signal lights according to the target signal light recognition information to obtain the target signal lights. Among them, the target signal lights are the signal lights in the driving direction of the target vehicle.

[0088] Steps 101 to 107 illustrated in the embodiments of the present application pre-construct an intersection knowledge base, which includes type mapping information and candidate type annotation information. The type mapping information contains candidate signal light arrangement types related to the position and driving direction. The candidate type annotation information contains candidate signal light recognition information related to the candidate signal light arrangement types. Based on the position and driving direction of the target vehicle, the target signal light arrangement type can be read from the type mapping information, and then based on the target signal light arrangement type, the target signal light recognition information can be read from the candidate type annotation information. Then, the candidate signal lights on the target signal light picture are recognized using the target signal light recognition information to obtain the target signal lights. It can be seen that the embodiments of the present application use the intersection knowledge base to provide effective recognition information for how to recognize signal lights, and can accurately recognize the signal lights in the driving / indicating direction of the vehicle. While improving the accuracy of signal light recognition, it can also accurately and timely perform red light running detection based on the recognized signal lights, thereby reducing the pressure of manual red light running detection.

[0089] In step 101 of some embodiments, a signal light picture in front of the target vehicle is obtained to obtain the target signal light picture. The target signal light picture can be a picture taken by an image acquisition device, such as a camera, a driving recorder, etc. The image acquisition device can continuously (such as continuously according to a certain period) obtain the pictures in front of the target vehicle during driving (operation), or can obtain the pictures in front when the vehicle and the intersection of the road meet a preset distance. When to obtain the pictures in front can be configured according to specific needs, and the embodiments of the present application do not limit this.

[0090] There may be one or more signal lights in the target signal light picture. A signal light refers to an indicator light / traffic light that a vehicle needs to observe when driving. A signal light usually consists of three colors of lights: red, yellow, and green, and is used to direct traffic. For example, when the green light is on, vehicles are allowed to pass. When the yellow light is flashing, vehicles that have crossed the stop line can continue to pass; those that have not passed should slow down and stop before the stop line and wait. When the red light is on, vehicles are prohibited from passing.

[0091] In step 102 of some embodiments, the position of the target vehicle is obtained to get the target vehicle position. The position of the target vehicle can be obtained through a high-precision positioning system installed in or external to the vehicle. The high-precision positioning system can achieve an accurate estimation of the vehicle's position. For example, during the driving process of the target vehicle, the position of the vehicle can be obtained through the high-precision positioning system at a certain sampling period to get position information. Therefore, in one example, the position information can be read based on the moment when the target signal light picture is taken to obtain the target vehicle position.

[0092] In step 103 of some embodiments, the driving direction of the target vehicle is obtained to get the target vehicle driving direction. The driving mode of the vehicle can also be obtained through a high-precision positioning system installed in or external to the vehicle. The high-precision positioning system can achieve an accurate estimation of the vehicle's driving direction. For example, during the driving process of the target vehicle, the driving direction of the vehicle can be obtained through the high-precision positioning system at a certain sampling period to get driving direction information. Therefore, in one example, the driving direction information can be read based on the moment when the target signal light is taken to obtain the target driving direction.

[0093] In step 104 of some embodiments, the target signal light picture is subjected to signal light detection through a preset signal light detection model to obtain candidate signal lights. The signal light detection model is a deep neural network model. The signal light detection model can be a YOLO (You Only Look Once) model, a convolutional neural network model, etc. After performing signal light detection on the target signal light picture, 0, 1, or more candidate signal lights may be obtained.

[0094] In one embodiment, referring to Figure 2 , before step 105, the signal light recognition method provided by the embodiments of the present application further includes: constructing an intersection knowledge base, specifically including:

[0095] Step 201, obtain the signal light picture in front of the sample vehicle during driving to get the sample signal light picture;

[0096] Step 202, perform annotation based on the sample signal light picture to obtain the candidate signal light arrangement type and candidate signal light recognition information;

[0097] Step 203: Obtain the position of the sample vehicle to get the sample vehicle position, and obtain the driving direction of the sample vehicle to get the sample vehicle driving direction;

[0098] Step 204: Perform first information association on the sample vehicle position, the sample vehicle driving direction, and the candidate signal light arrangement type to obtain type mapping information;

[0099] Step 205: Perform second information association on the candidate signal light arrangement type and the candidate signal light recognition information to obtain candidate type annotation information;

[0100] Step 206: Perform information merging based on the type mapping information and the candidate type annotation information to obtain an intersection knowledge base.

[0101] In step 201 of some embodiments, the sample vehicle is similar to the target vehicle, both referring to vehicles driving on the road. However, the sample vehicle is the vehicle used to construct the intersection knowledge base, while the target vehicle is the vehicle used for signal light recognition using the intersection knowledge base. The sample signal light picture can be a picture taken by an image acquisition device, such as a camera or a driving recorder.

[0102] In one embodiment, step 101 includes:

[0103] Obtain the video in front of the vehicle driving to get the sample video;

[0104] Extract frames from the sample video to get the initial frames;

[0105] Perform zebra crossing detection on the initial frames to get the zebra crossing detection category;

[0106] If the zebra crossing detection category is the first category, then determine the initial frame as the candidate frame; where the first category indicates that the initial frame has a zebra crossing;

[0107] Based on the last candidate frame in the sample video, obtain the sample signal light picture.

[0108] Specifically, the sample video refers to the frame sequence composed of the picture in front of the vehicle driving, and this sequence is used as a sample. In one example, the picture in front of the vehicle driving is collected by a mobile acquisition device (such as a driving recorder) mounted on the vehicle to obtain the sample video.

[0109] The sample video includes multiple frames. In one example, the target video is {frame 0, frame 1, frame 2, frame 3}. After frame extraction from the sample video, frame 0, frame 1, frame 2, and frame 3 can be used as the initial frames in sequence.

[0110] Among the above initial frames, some initial frames may not have zebra crossings, while some initial frames have zebra crossings. Since red light running detection requires determining the last frame of the vehicle in front of the zebra crossing, it is necessary to perform zebra crossing detection on the initial frames. A classification network model can be used to perform zebra crossing detection on each initial frame to obtain the zebra crossing detection category. The zebra crossing detection category includes a first category and a second category. The first category indicates that the initial frame has a zebra crossing. The second category indicates that the initial frame does not have a zebra crossing. If the zebra crossing detection category is the first category, the initial frame is determined as a candidate frame. If the zebra crossing detection category is the second category, the initial frame remains unchanged.

[0111] In an example, after performing zebra crossing detection on the initial frames of a sample video, the labeled sample video obtained is {Frame 0, without a zebra crossing; Frame 1, with a zebra crossing; Frame 2, with a zebra crossing; Frame 3, without a zebra crossing}. It can be seen that both Frame 1 and Frame 2 are candidate frames. Since Frame 2 is the last candidate frame in the target video, Frame 2 is determined as the sample signal light image.

[0112] The advantage of the above embodiment is that the flexible acquisition of the sample signal light image can be achieved by extracting the sample signal light picture from the sample video, and the sample signal light picture can be quickly obtained from the target video based on zebra crossing detection. It can be seen that this embodiment improves the acquisition flexibility while improving the acquisition efficiency.

[0113] In one embodiment, acquiring the video in front of the vehicle during driving to obtain a sample video includes:

[0114] Acquiring the videos of the vehicle in front of the target road during different time periods to obtain multiple candidate videos;

[0115] If it is determined that a candidate video is incomplete, the candidate video is deleted;

[0116] Performing video detection on each candidate video through a signal light detection network to obtain video detection data;

[0117] If the video detection data of each candidate video is consistent, any one of the multiple candidate videos is used as the sample video.

[0118] Specifically, it can be determined whether a candidate video is complete manually or by a pre-trained model. The signal light detection network can be a YOLO (You Only Look Once) model. If the signal light detection data of each candidate video is consistent, it means that the signal light image in the video of this intersection is not blocked and subsequent signal light detection can be performed.

[0119] The advantage of the above embodiment is that the video quality of the sample video can be ensured, thereby improving the accuracy of the sample signal light image obtained from the sample video.

[0120] In step 202 of some embodiments, the sample signal light pictures can be labeled by manual labeling or algorithm labeling to obtain candidate signal light arrangement types and candidate signal light recognition information.

[0121] In one embodiment, referring to Figure 3 , step 202 includes:

[0122] Step 301, performing signal light detection on the sample signal light pictures through a signal light detection model to obtain sample candidate signal lights;

[0123] Step 302, performing signal light arrangement type labeling based on the sample candidate signal lights to obtain candidate signal light arrangement types;

[0124] Step 303, performing signal light screening on the sample candidate signal lights based on the sample vehicle driving direction to obtain sample target signal lights;

[0125] Step 304, performing signal light labeling based on the candidate signal light arrangement types and the sample target signal lights to obtain candidate signal light recognition information; wherein, the candidate signal light recognition information represents the information of recognizing the sample target signal lights from the sample candidate signal lights.

[0126] In step 301 of some embodiments, the signal light detection model is a deep neural network model. The signal light detection model can be a YOLO (You Only Look Once) model, a convolutional neural network model, etc. After performing signal light detection on the sample signal light pictures, 0, 1, or multiple sample candidate signal lights may be obtained.

[0127] In step 302 of some embodiments, signal light arrangement type labeling can be performed by manual labeling or algorithm labeling. The sample candidate signal lights may include two horizontal lights, three horizontal lights, a single vertical light, two vertical lights, two groups of signal lights, three groups of signal lights, signal lights with a large spacing in the same group, and lights observed in special areas. In one example, if the sample candidate signal lights are two horizontal lights, three horizontal lights, a single vertical light, or two vertical lights, the candidate signal light arrangement type is the first type. In one example, if the sample candidate signal lights are two groups of signal lights or three groups of signal lights, the candidate signal light arrangement type is the second type. In one example, if the sample candidate signal lights are lights observed in special areas, the signal light arrangement type is the third type. In one example, if the sample candidate signal lights are signal lights with a large spacing in the same group, the signal light arrangement type is the fourth type.

[0128] In step 303 of some embodiments, the sample target signal light is the signal light of the sample vehicle in the driving direction of the sample vehicle. In one example, the sample candidate signal lights are two lateral signal lights. The left signal light is the signal light for the sample vehicle to turn left in the driving direction. The right signal light is the signal light for the sample vehicle to go straight in the driving direction. The sample vehicle does not need to look at the signal light when turning right in the driving direction. In another example, the sample candidate signal lights are three lateral signal lights. The left signal light is the signal light for the sample vehicle to turn left in the driving direction. The middle signal light is the signal light for the sample vehicle to go straight in the driving direction. The right signal light is the signal light for the sample vehicle to turn right in the driving direction.

[0129] In one embodiment, referring to Figure 4 , step 303 includes:

[0130] Step 401, grouping the sample candidate signal lights to obtain candidate signal light groups;

[0131] Step 402, determining the central axis of the sample signal light picture to obtain the target central axis;

[0132] Step 403, calculating the distance between the candidate signal light group and the target central axis to obtain the target distance;

[0133] Step 404, determining the candidate signal light group with the minimum target distance as the target signal light group;

[0134] Step 405, screening the sample candidate signal lights in the target signal light group based on the driving direction of the sample vehicle to obtain the sample target signal light.

[0135] In step 401 of some embodiments, there are multiple sample candidate signal lights on the sample signal light picture. These sample candidate signals may have a certain association in terms of position and size. Their association can be used for grouping.

[0136] In one embodiment, step 401 includes:

[0137] Taking the preset first direction as the abscissa direction and the direction perpendicular to the first direction as the ordinate direction, establish a coordinate system;

[0138] Based on any two sample candidate signal lights, obtain the first sample candidate signal light and the second sample candidate signal light;

[0139] In the coordinate system, obtain the coordinates of the first sample candidate signal light to get the first abscissa and the first ordinate, and obtain the coordinates of the second sample candidate signal light to get the second abscissa and the second ordinate;

[0140] If the first abscissa is the same as the second abscissa, the first sample candidate signal lamp and the second sample candidate signal lamp are added to the same candidate signal lamp group;

[0141] If the first ordinate is the same as the second ordinate, the first sample candidate signal lamp and the second sample candidate signal lamp are added to the same candidate signal lamp group.

[0142] In this embodiment, all sample candidate signal lamps are mainly traversed, and screened according to conditions such as whether they belong to the same direction (whether they are all horizontal / vertical) and whether they are at the same level. If the conditions are met, they are considered to be the same group of signal lamps.

[0143] The advantage of this embodiment is that grouping of sample candidate signal lamps is realized based on the coordinate system, which improves the grouping efficiency while ensuring the grouping accuracy.

[0144] In one embodiment, step 401 further includes:

[0145] Calculate the distance between the first sample candidate signal lamp and the second sample candidate signal lamp according to the first abscissa, the first ordinate, the second abscissa, and the second ordinate to obtain a target distance. If the target distance is an integer multiple of a preset distance threshold, the first sample candidate signal lamp and the second sample candidate signal lamp are added to the same candidate signal lamp group.

[0146] In this embodiment, grouping is mainly realized by using whether the distance is less than n distance thresholds as a condition, which improves the universality of grouping. The distance threshold can be set according to the width of the signal lamp, or other setting methods can be adopted, which are not limited in this embodiment.

[0147] In step 402 of some embodiments, the target central axis is a line that divides the sample signal lamp picture into two equal parts in the vertical direction. For example, map the sample signal lamp picture to the coordinate system; in the coordinate system, obtain the coordinates of the picture center pixel point of the sample signal lamp picture to get the center point coordinates; generate a line perpendicular to the abscissa direction based on the center point coordinates to obtain the target central axis; or generate a line parallel to the ordinate direction based on the center point coordinates to obtain the target central axis.

[0148] In step 403 of some embodiments, the target distance represents the distance between the candidate signal lamp group and the target central axis. In an example, a perpendicular line is drawn from the coordinates of the candidate signal lamp group to the target central axis to obtain a target line segment, and the target distance is obtained based on the length of the target line segment.

[0149] In step 404 of some embodiments, sort in ascending order of the target distance, and select the candidate signal lamp group with the smallest target distance as the target signal lamp group

[0150] In step 405 of some embodiments, the target signal light group includes a plurality of sample candidate signal lights, and different sample signal lights indicate different driving directions. Therefore, based on the driving direction of the sample vehicle, the sample candidate signal lights in the target signal light group are screened to obtain sample target signal lights.

[0151] Steps 401 to 405 illustrated in the embodiments of the present application can cover the situation where the signal light pictures in actual applications contain complex and diverse signal lights on the basis of being able to screen out sample target signal lights from sample candidate signal lights, and have high universality.

[0152] In step 305 of some embodiments, signal light annotation can be performed by manual annotation or algorithm annotation.

[0153] In one embodiment, referring to Figure 5 , step 305 includes:

[0154] Step 501, perform position annotation on the sample target signal lights based on the target signal light group to obtain the signal light position; wherein, the signal light position includes the signal light arrangement direction and the signal light position mark, the signal light arrangement direction characterizes the arrangement direction of the sample candidate signal lights in the target signal light group, and the signal light position mark characterizes the position of the sample target signal light in the signal light arrangement direction;

[0155] Step 502, if the candidate signal light arrangement type is the first type, perform first information merging on the signal light arrangement direction and the signal light position mark to obtain candidate signal light recognition information; wherein, the first type characterizes that the sample candidate signal lights in the target signal light group are arranged in the signal light arrangement direction.

[0156] Specifically, if the candidate signal light arrangement type is the first type, it means that a candidate signal light group appears in the sample signal light picture, and this candidate signal light group is determined as the target signal light group. The signal light position indicates that in the target signal light group, in the signal light arrangement direction, the signal light corresponding to the signal light position mark is the signal light for the current driving direction. The signal light arrangement direction refers to the arrangement direction of the sample candidate signal lights. The signal light arrangement direction includes from left to right, from right to left, from top to bottom, or from bottom to top. As Figure 6A shown, the sample candidate signal lights include signal light 01 and signal light 02, and the signal light arrangement direction of signal light 01 and signal light 02 is from left to right. As Figure 6B shown, the sample candidate signal lights include signal light 01 and signal light 02, and the signal light arrangement direction of signal light 01 and signal light 02 is from right to left. As Figure 6C shown, the sample candidate signal lights include signal light 01 and signal light 02, and the signal light arrangement direction of signal light 01 and signal light 02 is from top to bottom. As Figure 6DAs shown, the sample candidate signal lights include signal light 01 and signal light 02, and the arrangement direction of signal light 01 and signal light 02 is from bottom to top. In practice, the signal light arrangement direction is generally from left to right or from right to left.

[0157] In one example, the sample candidate signal lights are three horizontal signal lights. The left sample candidate signal light indicates a left turn, the middle sample candidate signal light indicates going straight, and the right sample candidate signal light indicates a right turn. Assuming the driving direction of the target vehicle is a right turn, after position marking, the signal light arrangement direction is obtained as from left to right, and the signal light position is marked as 03. If the candidate signal light arrangement type is the first type, only the signal light arrangement direction and the signal light position marking need to be combined to obtain the candidate signal light recognition information.

[0158] Steps 501 to 502 illustrated in the embodiments of the present application can be used to mark the candidate signal light recognition information when the candidate signal light arrangement type is the first type based on the signal light arrangement direction and the signal light position marking, improving the marking accuracy and efficiency.

[0159] In one embodiment, referring to Figure 7 , step 305 further includes:

[0160] Step 701, if the candidate signal light arrangement type is the second type, perform light group marking on the candidate signal light group to obtain the signal light group information; wherein, the second type represents that there are at least two candidate signal light groups, and the signal light group information represents the information for finding the target signal light group from the candidate signal light groups;

[0161] Step 702, perform second information combination on the signal light group information, the signal light arrangement direction, and the signal light position marking to obtain the candidate signal light recognition information.

[0162] Specifically, if the candidate signal light arrangement type is the second type, it means that there are multiple candidate signal light groups in the sample signal light picture. The signal light group information includes the signal light group arrangement direction and the signal light group position marking. The signal light group information indicates that among the multiple candidate signal light groups, in the signal light group arrangement direction, the candidate signal light group corresponding to the signal light group position marking is the target signal light group at the current position. The signal light group direction refers to the arrangement direction of the candidate signal light group. The signal light group direction includes from top to bottom, from bottom to top, from left to right, or from right to left. The difference from the above embodiment where the candidate signal light arrangement type is the first type is that it is necessary to find the target signal light group based on the signal light group information before identifying the target signal light from the sample candidate signal lights in the target signal light group based on the signal light arrangement direction and the signal light position marking. Therefore, it is necessary to perform second information combination on the signal light group information, the signal light arrangement direction, and the signal light position marking to obtain the candidate signal light recognition information.

[0163] Steps 701 to 702 shown in the embodiments of the present application can jointly implement the annotation of candidate signal light recognition information when the candidate signal light arrangement type is the second type based on the signal light group information, the signal light arrangement direction, and the signal light position marking, improving the annotation universality.

[0164] In one embodiment, step 305 further includes:

[0165] If the candidate signal light arrangement type is the third type, region generation is performed on the sample target signal light in the sample signal light picture to obtain a signal light region;

[0166] Region annotation is performed on the signal light region to obtain region position information;

[0167] Signal light number annotation is performed on the signal lights in the signal light region to obtain the number of signal lights in the region;

[0168] Signal light information annotation is performed on each sample candidate signal light in the signal light region to obtain region signal light information; wherein, the region signal light information includes the signal light size and / or the signal light position;

[0169] The region position information, the number of signal lights in the region, the region signal light information, the signal light arrangement direction, and the signal light position marking are subjected to a third information merge to obtain candidate signal light recognition information.

[0170] Specifically, if the candidate signal light arrangement type is the third type, it indicates that the sample target signal light on the sample signal light picture is relatively difficult to identify, and a specific region needs to be generated for signal light recognition. In this embodiment, the signal light region can be automatically generated according to the coordinates of the sample target signal light. The range of the signal light region can be set according to actual requirements. The region position information represents the position information of the signal light region. For example, the region position information includes the minimum abscissa of the region, the maximum abscissa of the region, the minimum ordinate of the region, and the maximum ordinate of the region. There may be more than one sample candidate signal light in the signal light region. After the number annotation is performed, the number of signal lights in the region is obtained. The region signal light information includes the signal light size and / or the signal light position. The signal light group where the sample target signal light is located can be determined based on the signal light size or the signal light position. The signal light position includes the signal light coordinates. The difference from the above embodiment where the candidate signal light arrangement type is the first type is that, based on the region position information, the number of signal lights in the region, and the region signal light information, the target signal light group where the sample target signal light is located needs to be found, and then the target signal light can be searched based on the signal light arrangement direction and the signal light position marking. Therefore, the region position information, the number of signal lights in the region, the region signal light information, the signal light arrangement direction, and the signal light position marking are subjected to a third information merge to obtain candidate signal light recognition information.

[0171] The advantages of the above embodiments are as follows. For the sample traffic light pictures with relatively difficult-to-recognize traffic lights, the annotation of the candidate traffic light recognition information is realized by means of region generation and region annotation, improving the annotation universality.

[0172] In one embodiment, step 305 further includes:

[0173] If the candidate traffic light arrangement type is the fourth type, calculate the distance between any two sample candidate traffic lights to obtain the traffic light distance threshold;

[0174] Obtain the candidate traffic light recognition information based on the traffic light distance threshold.

[0175] Specifically, if the candidate traffic light arrangement type is the fourth type, it indicates that the distance between the sample candidate traffic lights in the same group on the sample traffic light picture is relatively large. The difference from the above embodiment where the candidate traffic light arrangement type is the first type is that it may not be possible to find the sample target traffic light based on the traffic light arrangement direction and traffic light position markers. At this time, the traffic light closest to the central axis of the picture can be selected as the sample target traffic light. For the specific process of how to identify the traffic light, please refer to the detailed description of step 107 below.

[0176] It should be noted that in addition to the above-mentioned type mapping information and candidate type annotation information, the intersection knowledge base also stores other information related to the intersection. For example, the intersection knowledge base includes intersection identification (ID), intersection direction identification (RoadID), sample vehicle position (GpsInfo), sample vehicle driving direction (LightDirection), and annotation time (time_labeled), etc. Among them, the intersection identification (ID) represents the unique primary key to distinguish intersections and directions. The intersection direction identification (RoadID) is associated with the ID in the intersection knowledge base. The sample vehicle position can be filled in as a whole in the form of gps or separately in the form of latitude and longitude. The intersection knowledge base also records the types of each field. For example, the type of the field for the sample vehicle driving position is string. The types of other fields can be set according to actual needs and are not limited here. The sample vehicle position (GpsInfo) can take the gps coordinates X meters before driving to the traffic light, which is connected to the traffic light position to represent the current driving direction.

[0177] The intersection knowledge base also includes: traffic light annotation information (TrafficLightObject). The traffic light annotation information includes picture label (label) and picture annotation area information (bndbox). The picture annotation area information includes the minimum abscissa of the picture, the minimum ordinate of the picture, the maximum abscissa of the picture, and the maximum ordinate of the picture.

[0178] The intersection knowledge base also includes: sample signal light pictures (Img), picture information (ImageInfo), zebra crossing detection category (CrossWalk), zebra crossing coordinates (CrossWalkLock), red light time (RedLastTime), intersection type (RoadType), sample vehicle driving direction (CarDirection), yellow light driving status mark (RunYellow), red light driving status mark (RunRed), etc. Among them, the picture information includes the picture width (width) and the picture height (height). The zebra crossing detection category indicates whether there is a zebra crossing.

[0179] The yellow light driving status mark (RunYellow) indicates whether it is determined that the vehicle runs a yellow light. For example, at some special intersections, the yellow light will flash constantly, and no yellow light running judgment is made at this intersection. If the yellow light driving status mark is the first mark value (false), even if the vehicle runs a yellow light, it is not considered a violation. If the yellow light driving status mark is the second mark value (true), if the vehicle runs a yellow light, it is considered a violation. By default, this parameter does not exist, and if a vehicle runs a yellow light, it is considered a violation.

[0180] The red light driving status mark (RunRed) indicates whether it is determined that the vehicle runs a red light. For example, at some intersections without signal lights, there is no need to determine whether the vehicle runs a red light. If the red light driving status mark is the first mark value (false), even if the vehicle runs a red light, it is not considered a violation. If the red light driving status mark is the second mark value (true), if the vehicle runs a red light, it is considered a violation. By default, this parameter does not exist, and if a vehicle runs a red light, it is considered a violation.

[0181] In step 105 of some embodiments, according to the target vehicle position and the target vehicle driving direction, type reading is performed on the type mapping information stored in the preset intersection knowledge base to obtain the target signal light arrangement type. The intersection knowledge base also stores candidate type annotation information, and the candidate type annotation information includes candidate signal light arrangement types and candidate signal light recognition information.

[0182] Specifically, the candidate signal light arrangement types (RedParam) include the first type (Normal), the second type (ChooseGroup), the third type (ROIRedRegion), and the fourth type (IngroupThre). Different candidate signal light arrangement types have corresponding candidate signal light recognition information. According to the target vehicle position and the target vehicle driving direction, the target signal light arrangement type can be one of the first type, the second type, the third type, and the fourth type, so that the target signal light recognition information can be determined based on whether it is the same as the candidate signal light arrangement type.

[0183] The first candidate type annotation information in the intersection knowledge base includes the first type, the signal light arrangement direction, and the signal light position mark. The second candidate type annotation information in the intersection knowledge base includes the second type, the signal light group information, the signal light arrangement direction, and the signal light position mark. The third candidate type annotation information in the intersection knowledge base includes the third type, the signal light area position, the total number of signal lights, the signal light size, the signal light arrangement direction, and the signal light position mark. The fourth candidate type annotation information in the intersection knowledge base includes the fourth type and the signal light spacing threshold.

[0184] In step 106 of some embodiments, if the target signal light arrangement type is the same as the candidate signal light arrangement type, the candidate signal light recognition information is determined as the target signal light recognition information.

[0185] In an example, assume that the target signal light arrangement type is the first type, which is the same as the first type in the first candidate type annotation information. Then, the candidate signal light recognition information in the first candidate type annotation information is used as the target signal light recognition information, so that the obtained target signal light recognition information includes the signal light arrangement direction and the signal light position mark.

[0186] In step 107 of some embodiments, signal light recognition is performed on the candidate signal lights according to the target signal light recognition information to obtain the target signal lights; wherein, the target signal lights are the signal lights of the target vehicle in the target vehicle driving direction.

[0187] In an embodiment, the target signal light recognition information includes the signal light arrangement direction and the signal light position mark; step 107 includes:

[0188] According to the signal light arrangement direction, position marking is performed on the candidate signal lights to obtain candidate position marks.

[0189] For each candidate signal light, if the candidate position mark is the same as the signal light position mark, the candidate signal light is determined as the target signal light.

[0190] The signal light arrangement direction refers to the direction in which the candidate signal lights are arranged. The signal light arrangement direction includes from top to bottom, from bottom to top, from left to right, or from right to left. In practice, the signal light arrangement direction is generally from left to right. In an example, the candidate signal lights are three horizontal signal lights, and the signal light arrangement direction can be from left to right or from right to left. Assume that the signal light arrangement direction is from left to right, and assume that the signal light position mark is 03. After position marking, the candidate position mark of the left candidate signal light is 01, the candidate position mark of the middle candidate signal light is 02, and the candidate position mark of the right candidate signal light is 03. Using the signal light position mark as an index, since the candidate position mark is 03 and the signal light position mark is 03 are the same, the right candidate signal light is determined as the target signal light.

[0191] The advantage of this embodiment is that the target signal lamp can be accurately and quickly determined from the candidate signal lamps based on the arrangement direction of the signal lamps and the signal lamp position mark, improving the determination accuracy and efficiency.

[0192] In one embodiment, the target signal lamp identification information includes signal lamp group information, signal lamp arrangement direction, and signal lamp position mark; step 107 includes:

[0193] Group the candidate signal lamps to obtain candidate signal lamp groups;

[0194] Filter the candidate signal lamp group information according to the signal lamp group information to obtain the target signal lamp group;

[0195] Mark the positions of the candidate signal lamps in the target signal lamp group according to the signal lamp arrangement direction to obtain candidate position marks;

[0196] For each candidate signal lamp in the target signal lamp group, if the candidate position mark is the same as the signal lamp position mark, determine the candidate signal lamp as the target signal lamp.

[0197] Specifically, it can be judged whether they are signal lamps in the same group by the horizontal position of the signal lamp center point and the signal lamp size condition. The specific grouping conditions can be set according to actual needs, and the embodiments of the present application do not make specific limitations.

[0198] The signal lamp group information includes the signal lamp group direction and the signal lamp group position mark. The signal lamp group direction refers to the direction in which the candidate signal lamp group is arranged. The signal lamp group direction includes from top to bottom, from bottom to top, from left to right, or from right to left. In one embodiment, mark the positions of the candidate signal lamp groups according to the signal lamp group direction to obtain candidate signal lamp group position marks; if the candidate signal lamp group position mark is the same as the signal lamp group position mark, determine the candidate signal lamp group as the target signal lamp group.

[0199] In an example, two candidate signal lamp groups appear in the sample signal lamp picture. Assume that the signal lamp group direction is from top to bottom, and assume that the signal lamp group position mark is 01. After position marking, the candidate signal lamp group position mark of the upper candidate signal lamp group is 01, and the candidate signal lamp group position mark of the lower candidate signal lamp is 02. Since the candidate signal lamp group position mark 01 is the same as the signal lamp group position mark 01, the upper candidate signal lamp group is determined as the target signal lamp group. After obtaining the target signal lamp group, determine the target signal lamp based on the signal lamp arrangement direction and the signal lamp position mark. This process is similar to the process of the above embodiment and will not be elaborated here.

[0200] The advantage of this embodiment is that the target signal lamp can be accurately and quickly determined from multiple candidate signal lamp groups based on the signal lamp group information, the arrangement direction of the signal lamps, and the signal lamp position markers, improving the determination accuracy and efficiency.

[0201] In one embodiment, the target signal lamp identification information includes a signal lamp spacing threshold; step 107 includes:

[0202] Calculate the spacing between any two candidate signal lamps to obtain a target spacing;

[0203] If the target spacing is greater than the signal lamp spacing threshold, calculate the distance between the candidate signal lamp and the target central axis to obtain a signal lamp distance;

[0204] Determine the candidate signal lamp with the minimum signal lamp distance as the target signal lamp.

[0205] The advantage of this embodiment is that the target signal lamp can be determined from multiple candidate signal lamps with a relatively large spacing, with high accuracy.

[0206] In one embodiment, the target signal lamp identification information includes the signal lamp area position, the total number of signal lamps, the signal lamp size, the signal lamp arrangement direction, and the signal lamp position marker; step 107 includes:

[0207] Generate a region for the target frame based on the signal lamp area position to obtain a target signal lamp region;

[0208] If the candidate signal lamp is within the target signal lamp region, determine the candidate signal lamp as the first candidate signal lamp;

[0209] Obtain the size of the first candidate signal lamp to obtain a first candidate size;

[0210] If the first candidate size is the same as the signal lamp size, determine the first candidate signal lamp as the second candidate signal lamp;

[0211] Mark the positions of the total number of second candidate signal lamps according to the signal lamp arrangement direction to obtain candidate position markers;

[0212] For each second candidate signal lamp, if the candidate position marker is the same as the signal lamp position marker, then determine the second candidate signal lamp as the target signal lamp.

[0213] The advantage of this embodiment is that the target signal lamp can be accurately determined from multiple sample candidate signal lamps based on the signal lamp area position, the total number of signal lamps, the signal lamp size, the signal lamp arrangement direction, and the signal lamp position marker, improving the determination accuracy.

[0214] In one embodiment, after step 107, the signal lamp identification method provided in this embodiment further includes:

[0215] Perform a status detection on the target signal lamp to obtain the signal lamp status;

[0216] If the signal lamp status is the first status, obtain the speed of the target vehicle to get the target vehicle speed; wherein, the first status indicates that the target vehicle is prohibited from passing in the driving direction of the target vehicle;

[0217] Perform a zebra crossing detection on the target signal lamp picture through a preset zebra crossing detection model to obtain the zebra crossing detection status;

[0218] If the zebra crossing detection status is the first zebra crossing status, and if the target vehicle speed is greater than a preset speed threshold, determine that the target vehicle runs a red light.

[0219] Specifically, the zebra crossing detection model is a deep neural network model. For example, the zebra crossing detection model can be a YOLO (You Only Look Once) model, a convolutional neural network model. If the zebra crossing detection status is the first zebra crossing status, it means that there is no zebra crossing in the target first frame (it may be that there is no zebra crossing on the road itself, or it may be that the target vehicle has passed the zebra crossing), or the position of the zebra crossing in the target first frame is abnormal (for example, the target vehicle has driven onto the zebra crossing). At this time, if the target vehicle speed is greater than the speed threshold, it is determined that the target vehicle runs a red light. Generally, the speed threshold is zero.

[0220] Based on the combined effect of the zebra crossing detection status and the target speed, the red light running judgment of the target vehicle is realized, which has high accuracy.

[0221] Please refer to Figure 8 , this application embodiment also provides a signal lamp recognition device, which can implement the above signal lamp recognition method, Figure 8It is a block diagram of the module structure of the signal light recognition device provided by the embodiment of the present application. The device includes: an image acquisition module 801, a position acquisition module 802, a direction acquisition module 803, a detection module 804, a type reading module 805, an information determination module 806, and an identification module 807. Among them, the image acquisition module 801 is used to acquire a signal light image in front of the target vehicle during driving to obtain a target signal light image; the position acquisition module 802 is used to acquire the position of the target vehicle to obtain the target vehicle position; the direction acquisition module 803 is used to acquire the driving direction of the target vehicle to obtain the target vehicle driving direction; the detection module 804 is used to perform signal light detection on the target signal light image through a preset signal light detection model to obtain candidate signal lights; the type reading module 805 is used to read type mapping information stored in a preset intersection knowledge base according to the target vehicle position and the target vehicle driving direction to obtain the target signal light arrangement type; the intersection knowledge base also stores candidate type annotation information, and the candidate type annotation information includes candidate signal light arrangement types and candidate signal light identification information; the information determination module 806 is used to determine the candidate signal light identification information as the target signal light identification information if the target signal light arrangement type is the same as the candidate signal light arrangement type; the identification module 807 is used to perform signal light identification on the candidate signal lights according to the target signal light identification information to obtain the target signal light; among them, the target signal light is the signal light of the target vehicle in the target vehicle driving direction.

[0222] In one embodiment, before reading parameters of type mapping information stored in a preset intersection knowledge base according to the target vehicle position and the target vehicle driving direction to obtain the target signal light arrangement type, the signal light recognition device further includes a construction module for constructing the intersection knowledge base.

[0223] It should be noted that the specific implementation manner of this signal light recognition device is basically the same as the specific embodiment of the above signal light recognition method, and will not be elaborated here.

[0224] The embodiment of the present application also provides an electronic device. The electronic device includes: a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory. When the program is executed by the processor, the above signal light recognition method is implemented. The electronic device can be any intelligent terminal including a tablet computer, an in-vehicle computer, etc.

[0225] Please refer to Figure 9 , Figure 9 which shows the hardware structure of an electronic device in another embodiment. The electronic device includes:

[0226] The processor 901 can be implemented in ways such as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0227] The memory 902 can be implemented in forms such as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 902 and are called by the processor 901 to execute the traffic light recognition method of the embodiments of the present application;

[0228] The input / output interface 903 is used to implement information input and output;

[0229] The communication interface 904 is used to implement communication interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or through wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0230] The bus 905 transmits information between the various components of the device (such as the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0231] Among them, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904 achieve communication connections with each other inside the device through the bus 905.

[0232] The embodiments of the present application also provide a storage medium. The storage medium is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned traffic light recognition method.

[0233] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include memories remotely located relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0234] The embodiments described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art will know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0235] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figures, or combine certain steps, or different steps.

[0236] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0237] Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, and appropriate combinations thereof.

[0238] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above figures are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0239] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item) of the following" or its similar expression refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0240] In several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical or other forms.

[0241] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0242] In addition, the functional units in each embodiment of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0243] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store programs such as USB flash drives, mobile hard disks, read-only memories (ROM for short), random access memories (RAM for short), magnetic disks, or optical discs.

[0244] The preferred embodiments of the embodiments of this application have been described above with reference to the accompanying drawings, which does not limit the scope of the rights of the embodiments of this application. Any modifications, equivalent replacements, and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of this application shall fall within the scope of the rights of the embodiments of this application.

Claims

1. A signal lamp recognition method, characterized in that, The method includes: Obtaining a signal light image in front of the target vehicle to obtain a target signal light image; Obtaining the position of the target vehicle to obtain a target vehicle position; Obtaining the driving direction of the target vehicle to obtain a target vehicle driving direction; Performing signal light detection on the target signal light image through a preset signal light detection model to obtain candidate signal lights; Reading the type mapping information stored in a preset intersection knowledge base according to the target vehicle position and the target vehicle driving direction to obtain a target signal light arrangement type; the intersection knowledge base also stores candidate type annotation information, and the candidate type annotation information includes a candidate signal light arrangement type and candidate signal light recognition information; If the target signal light arrangement type is the same as the candidate signal light arrangement type, determining the candidate signal light recognition information as the target signal light recognition information; Performing signal light recognition on the candidate signal lights according to the target signal light recognition information to obtain a target signal light; wherein, the target signal light is the signal light of the target vehicle in the target vehicle driving direction.

2. The method according to claim 1, characterized in that, Before reading the type mapping information stored in the preset intersection knowledge base according to the target vehicle position and the target vehicle driving direction to obtain the target signal light arrangement type, the method further includes: Constructing the intersection knowledge base, including: Obtaining a signal light image in front of the sample vehicle to obtain a sample signal light image; Performing annotation based on the sample signal light image to obtain the candidate signal light arrangement type and the candidate signal light recognition information; Obtaining the position of the sample vehicle to obtain a sample vehicle position, and obtaining the driving direction of the sample vehicle to obtain a sample vehicle driving direction; Performing first information association on the sample vehicle position, the sample vehicle driving direction, and the candidate signal light arrangement type to obtain the type mapping information; Performing second information association on the candidate signal light arrangement type and the candidate signal light recognition information to obtain the candidate type annotation information; Performing information merging based on the type mapping information and the candidate type annotation information to obtain the intersection knowledge base.

3. The method according to claim 2, wherein The performing annotation based on the sample signal light image to obtain the candidate signal light arrangement type and the candidate signal light recognition information includes: Performing signal light detection on the sample signal light image through the signal light detection model to obtain sample candidate signal lights; Performing signal light arrangement type annotation based on the sample candidate signal lights to obtain the candidate signal light arrangement type; Performing signal light screening on the sample candidate signal lights based on the sample vehicle driving direction to obtain sample target signal lights; wherein, the sample target signal lights are the signal lights of the sample vehicle in the sample vehicle driving direction; Performing signal light annotation based on the candidate signal light arrangement type and the sample target signal lights to obtain the candidate signal light recognition information; wherein, the candidate signal light recognition information represents the information for recognizing the sample target signal lights from the sample candidate signal lights.

4. The method according to claim 3, wherein Performing signal light screening on the sample candidate signal lights based on the driving direction of the sample vehicle to obtain sample target signal lights, including: Grouping the sample candidate signal lights to obtain candidate signal light groups; Determining the central axis of the sample signal light image to obtain the target central axis; Calculating the distance between the candidate signal light group and the target central axis to obtain the target distance; Determining the candidate signal light group with the minimum target distance as the target signal light group; Performing screening on the sample candidate signal lights in the target signal light group based on the driving direction of the sample vehicle to obtain the sample target signal lights.

5. The method according to claim 4, characterized in that, Performing signal light annotation on the candidate signal light based on the candidate signal light arrangement type and the sample target signal light to obtain the candidate signal light recognition information, including: Performing position annotation on the sample target signal light based on the target signal light group to obtain the signal light position; wherein, the signal light position includes the signal light arrangement direction and the signal light position mark; wherein, the signal light arrangement direction represents the arrangement direction of the sample candidate signal lights in the target signal light group, and the signal light position mark represents the position of the sample target signal light in the signal light arrangement direction; If the candidate signal light arrangement type is the first type, performing first information merging on the signal light arrangement direction and the signal light position mark to obtain the candidate signal light recognition information; wherein, the first type represents that the sample candidate signal lights in the target signal light group are arranged in the signal light arrangement direction.

6. The method according to claim 5, characterized in that Performing signal light annotation on the candidate signal light based on the candidate signal light arrangement type and the sample target signal light to obtain the candidate signal light recognition information, further including: If the candidate signal light arrangement type is the second type, performing light group annotation on the candidate signal light group to obtain the signal light group information; wherein, the second type represents that there are at least two candidate signal light groups and the sample candidate signal lights in the target signal light group are arranged in the signal light arrangement direction, and the signal light group information represents the information for finding the target signal light group from the candidate signal light group; Performing second information merging on the signal light group information, the signal light arrangement direction and the signal light position mark to obtain the candidate signal light recognition information.

7. The method according to claim 4, wherein Grouping the sample candidate signal lights to obtain candidate signal light groups, including: Establishing a coordinate system with a preset first direction as the abscissa direction and a direction perpendicular to the first direction as the ordinate direction; Based on any two of the sample candidate signal lights, obtaining a first sample candidate signal light and a second sample candidate signal light; In the coordinate system, obtaining the coordinates of the first sample candidate signal light to obtain a first abscissa and a first ordinate, and obtaining the coordinates of the second sample candidate signal light to obtain a second abscissa and a second ordinate; If the first abscissa is the same as the second abscissa, adding the first sample candidate signal light and the second sample candidate signal light to the same candidate signal light group; If the first ordinate is the same as the second ordinate, add the first sample candidate signal lamp and the second sample candidate signal lamp to the same candidate signal lamp group.

8. A signal lamp recognition device, characterized in that, The device includes: An image acquisition module, configured to acquire a signal lamp image in front of the target vehicle to obtain a target signal lamp image; A position acquisition module, configured to acquire the position of the target vehicle to obtain a target vehicle position; A direction acquisition module, configured to acquire the driving direction of the target vehicle to obtain a target vehicle driving direction; A detection module, configured to perform signal lamp detection on the target signal lamp image through a preset signal lamp detection model to obtain candidate signal lamps; A type reading module, configured to read type mapping information stored in a preset intersection knowledge base according to the target vehicle position and the target vehicle driving direction to obtain a target signal lamp arrangement type; the intersection knowledge base further stores candidate type annotation information, and the candidate type annotation information includes a candidate signal lamp arrangement type and candidate signal lamp identification information; An information determination module, configured to determine the candidate signal lamp identification information as target signal lamp identification information if the target signal lamp arrangement type is the same as the candidate signal lamp arrangement type; An identification module, configured to perform signal lamp identification on the candidate signal lamps according to the target signal lamp identification information to obtain target signal lamps; wherein, the target signal lamps are the signal lamps of the target vehicle in the target vehicle driving direction.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the signal lamp identification method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the signal lamp identification method according to any one of claims 1 to 7 is implemented.

Images