Traffic signal recognition method, device and equipment and storage medium

Abstract

The application provides a traffic signal recognition method, device and equipment and a storage medium. The method comprises the following steps: in response to collecting a traffic light image of a target intersection at a current time, extracting an image feature of the traffic light image; determining a traffic light type of the traffic light according to a matching degree between the image feature and a plurality of preset traffic light types; recognizing an attribute of the traffic light according to the image feature of the traffic light; and determining a traffic indication signal at the current time according to the traffic light type and the attribute corresponding to the traffic light. The application does not depend on a high-precision map, and adopts an end-to-end processing process, which is beneficial to avoiding error accumulation caused by a plurality of processing stages, so as to improve the recognition accuracy and the traffic signal recognition performance to a certain extent.

Description

Technical Field

[0001] This application belongs to the field of autonomous driving technology, specifically relating to a traffic signal recognition method, device, equipment, and storage medium. Background Technology

[0002] In order to safely and compliantly pass through urban intersections during autonomous driving, the vehicle's autonomous driving system should correctly identify the traffic signals indicated by the traffic lights at the intersections.

[0003] Among the commonly used traffic signal recognition schemes, some rely on high-precision maps, which not only incur high data maintenance costs but also suffer from poor robustness and applicability. Other schemes are based on machine learning models to identify traffic signals. These schemes involve multiple processing stages, including preprocessing to learn the features of various traffic lights and postprocessing to select the appropriate traffic lights. Each processing stage introduces errors, leading to error accumulation and poor accuracy. Therefore, the performance of commonly used traffic signal recognition schemes is generally poor. Summary of the Invention

[0004] This application proposes a traffic signal recognition method, apparatus, device, and storage medium that does not rely on high-precision maps and reduces error accumulation through an end-to-end processing mechanism, thereby improving applicability and traffic signal recognition performance.

[0005] The first aspect of this application provides a traffic signal recognition method, the method comprising:

[0006] In response to acquiring the traffic light image of the target intersection at the current moment, the image features of the traffic light image are extracted;

[0007] The traffic light type is determined based on the matching degree between the image features and multiple preset traffic light types; any traffic light type is used to indicate the direction of travel.

[0008] The attributes of the traffic lights are identified based on their image features; these attributes are used to indicate driving status.

[0009] Based on the traffic light type and attributes corresponding to the traffic light, determine the traffic indication signal for the current moment.

[0010] In some embodiments of this application, the plurality of traffic light types include: U-turn traffic lights, disc traffic lights, left-turn traffic lights, straight-ahead traffic lights, and right-turn traffic lights;

[0011] The traffic light attribute set includes: the color of the traffic light, the flashing state of the traffic light, and the countdown state of the traffic light.

[0012] In some embodiments of this application, determining the traffic light type based on the matching degree between the image features and a plurality of preset traffic light types includes:

[0013] The pre-deployed attention algorithm is invoked to calculate the correlation parameters between the image features of the traffic lights and the features of the multiple traffic light types;

[0014] The traffic light type corresponding to the feature with the highest correlation parameter is determined as the reference type of the traffic light;

[0015] The traffic light type is determined based on the reference type.

[0016] In some embodiments of this application, determining the traffic light type based on the reference type includes:

[0017] The reference type is determined as the traffic light type of the traffic light;

[0018] or,

[0019] The traffic light type is determined based on the historical attributes within the preset historical time period of the traffic light and the reference type. The preset historical time period refers to the time period that is adjacent to the current time and earlier than the current time.

[0020] In some embodiments of this application, identifying the attributes of the traffic light based on its image features includes:

[0021] Calculate the probability distribution of the image features of the traffic light corresponding to a preset set of traffic light attributes. The probability distribution includes multiple probability values, and any probability value represents the probability that the attribute of the traffic light is the corresponding attribute.

[0022] The traffic light attribute with the highest probability value in the probability distribution is determined as the attribute of the traffic light.

[0023] In some embodiments of this application, before extracting the image features of the traffic light image, the method further includes:

[0024] Multiple traffic light images of the target intersection are captured by multiple preset cameras, and each camera corresponds to one of the multiple traffic light images.

[0025] In some embodiments of this application, extracting image features from the traffic light image includes:

[0026] Extract features from each of the multiple traffic light images to obtain multiple two-dimensional image features;

[0027] For any two-dimensional image feature, a three-dimensional image feature corresponding to the two-dimensional image feature is generated based on the parameters of the camera corresponding to the two-dimensional image feature. The multiple three-dimensional image features corresponding one-to-one with the multiple two-dimensional image features are the image features of the traffic light image.

[0028] In some embodiments of this application, extracting image features from the traffic light image includes:

[0029] Extract features from each of the multiple traffic light images to obtain multiple two-dimensional image features;

[0030] The multiple two-dimensional image features are stitched together to obtain panoramic image features;

[0031] The panoramic image features are converted into three-dimensional image features based on the camera parameters of the multiple cameras, and the three-dimensional image features are the image features of the traffic light image.

[0032] A second aspect of this application provides a traffic signal recognition device, the device comprising:

[0033] The extraction module is used to extract image features from the traffic light image of the target intersection in response to the acquisition of the traffic light image at the current time.

[0034] The determining module is used to determine the traffic light type based on the matching degree between the image features and multiple preset traffic light types; any traffic light type is used to indicate the direction of travel.

[0035] The recognition module is used to identify the attributes of the traffic light based on its image features; the attributes of the traffic light are used to indicate the driving status.

[0036] The determining module is further configured to determine the traffic indication signal at the current time based on the traffic light type and attributes corresponding to the traffic light.

[0037] An embodiment of the third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0038] An embodiment of the fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method described in the first aspect above.

[0039] The technical solution provided in this application embodiment, after acquiring the traffic light image of the target intersection at the current time, extracts the image features of the traffic light image, and then determines the traffic light type and attributes of the traffic light in the image from a preset set of multiple traffic light types. The traffic light type is used to indicate the direction of travel, and the traffic light attributes are used to indicate the travel status. Based on this, this application embodiment can determine the traffic signal at the current time according to the traffic light type and attributes. Clearly, the technical solution of this application embodiment does not rely on high-precision maps, and the implementation of this technical solution is an end-to-end processing process, not involving multiple processing stages such as learning various traffic light features in the preprocessing stage and selecting traffic lights in the postprocessing stage. This not only reduces the learning cost but also helps avoid the accumulation of errors caused by multiple processing stages, thereby improving the accuracy of recognition and the performance of traffic signal recognition to a certain extent.

[0040] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0041] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0042] In the attached diagram:

[0043] Figure 1 A flowchart of a traffic signal recognition method according to an embodiment of this application is shown;

[0044] Figure 2 This illustration shows the data flow of a traffic signal recognition method provided in an embodiment of this application. Figure 1 ;

[0045] Figure 3 This illustration shows the data flow of a traffic signal recognition method provided in an embodiment of this application. Figure 2 ;

[0046] Figure 4 This invention provides a schematic diagram of the structure of a traffic signal recognition device according to an embodiment of the present application.

[0047] Figure 5 This illustration shows a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0048] Figure 6 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0049] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0050] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0051] First, the technical scenarios involved in this application will be explained.

[0052] This application relates to autonomous driving technology and artificial intelligence technology. In urban driving environments, road conditions are generally complex. To ensure traffic safety and efficiency, traffic lights are typically installed at intersections to regulate vehicle movement and waiting. Therefore, for vehicles employing autonomous driving, the ability of the autonomous driving system to recognize traffic signals indicated by intersection traffic lights is particularly important for the safety performance of the autonomous driving system.

[0053] Currently, commonly used traffic signal recognition schemes all rely on visual detection and recognition technology to detect traffic signals emitted by traffic lights (i.e., red, green, and yellow lights). For example, a common traffic signal recognition scheme marks the spatial locations of red and green lights and various attributes of the traffic lights on a high-precision map. Then, it projects the spatial locations (i.e., three-dimensional (3D) locations) on the high-precision map onto the acquired traffic light images for detection and recognition. This method not only heavily relies on high-precision maps but also requires high accuracy of the intrinsic and extrinsic parameters of the vehicle-mounted camera, resulting in poor robustness and limited applicability. Another commonly used traffic signal recognition scheme takes an image acquired by at least one vehicle-mounted camera as input, detects red and green lights in the image, segments the detected lights, and then identifies the color, shape, and other attribute features of each segmented light image. After determining the positional features of the segmented light, it fuses the above features to obtain the traffic signal indicated by the light. This recognition scheme relies on the shapes of traffic lights (red and green lights) input in the preprocessing stage, requiring complex recognition logic and post-processing fusion logic. It is not an end-to-end algorithm, leading to error accumulation and poor accuracy. Another common traffic signal recognition scheme processes the acquired image along two task lines: one identifies traffic lights issuing signals in the image coordinate system, and the other detects lane lines in bird's-eye view (BEV) space. The detection results from both task lines are then correlated to obtain the traffic signal recognition result. However, this scheme requires a large amount of labeled data during the training phase, resulting in high training costs. Furthermore, it is not an end-to-end algorithm, again leading to error accumulation and poor accuracy.

[0054] To address the above issues, the technical solution provided in this application no longer relies on high-precision maps and employs an end-to-end processing procedure. After acquiring traffic light images, image features are extracted from the traffic light images. Then, the traffic light type and attributes in the image are determined from a set of preset traffic light types. This not only reduces the learning cost but also helps avoid error accumulation caused by multiple processing stages, thereby improving the accuracy of recognition and enhancing traffic signal recognition performance to a certain extent.

[0055] The implementing entity of this technical solution can be any mobile device that supports traffic signal recognition, including vehicles, robots, and other devices. Such devices can support traffic signal recognition in the embodiments of this application through an autonomous driving system. This autonomous driving system can be equipped with execution modules for functions such as image feature extraction, coordinate dimension transformation algorithms, and classification algorithms.

[0056] The following description, in conjunction with the accompanying drawings, illustrates a traffic signal recognition method, apparatus, device, and storage medium according to embodiments of this application. Specific embodiments are described in detail below. These specific embodiments can be combined with each other, and similar or identical concepts or processes may not be repeated in some embodiments. The embodiments of this application will now be described in conjunction with the accompanying drawings.

[0057] See Figure 1 , Figure 1 This application provides a flowchart of a traffic signal recognition method according to an embodiment of the present application. The method specifically includes the following steps:

[0058] Step S101: In response to acquiring the traffic light image of the target intersection at the current time, extract the image features of the traffic light image.

[0059] The traffic light image includes an image of a traffic light. The target intersection can be the intersection that is closest to the autonomous vehicle in the direction it is traveling.

[0060] It should be noted that, in this embodiment of the application, the autonomous driving system can call on the on-board camera to periodically collect images in the forward driving direction during the autonomous driving process, so as to detect the intersections passed in the forward driving direction in a timely manner, and then identify the traffic signal of the nearest intersection, so as to drive or wait according to the instructions of the traffic signal.

[0061] For example, the period at which an autonomous driving system calls on the onboard camera to capture images can be expressed in frames per second (FPS). This period can be greater than or equal to 15 FPS and less than or equal to 40 FPS. In practical implementation scenarios, the period can be set according to the needs of urban areas. In one optional example, the period is, for example, 30 FPS.

[0062] In practical implementation scenarios, vehicle-mounted cameras may include multiple cameras. In some embodiments, the autonomous driving system can acquire multiple traffic light images of the target intersection through multiple preset cameras, with each camera corresponding to one of the multiple traffic light images. It should be understood that this application embodiment should acquire images of the intersection in the direction of the vehicle's forward travel; therefore, the preset multiple cameras can be multiple forward-facing cameras of the vehicle. For example, the preset multiple cameras can be three forward-facing cameras, each with a different focal length. Correspondingly, the autonomous driving system can call each of the three forward-facing cameras at 30 FPS to acquire images of the intersection, obtaining three images at each acquisition time.

[0063] Optionally, the image features are 3D image features. In some embodiments, after acquiring multiple traffic light images, the autonomous driving system can extract features from each traffic light image to obtain multiple two-dimensional (2D) image features. Then, for any two-dimensional image feature, a three-dimensional image feature corresponding to that two-dimensional image feature is generated based on the parameters of the camera corresponding to that two-dimensional image feature. The multiple three-dimensional image features corresponding one-to-one with the multiple two-dimensional image features constitute the image features of the traffic light image. In other embodiments, after acquiring multiple traffic light images, the autonomous driving system can extract features from each traffic light image to obtain multiple 2D image features. Then, the multiple two-dimensional image features are stitched together to obtain BEV image features. Based on the camera parameters of the multiple cameras, the panoramic image features are converted into 3D image features, and the 3D image features constitute the image features of the traffic light image.

[0064] For example, an autonomous driving system can call a pre-set convolutional neural network (CNN) to extract 2D image features from each image, and can call a pre-set transformer network based on the inverse perspective mapping (IPM) algorithm to convert 2D image features into 3D image features.

[0065] In this way, the position of each pixel in the image in 3D space can be perceived, thereby perceiving the image content in 3D space. This is helpful in knowing the position and orientation of traffic lights, so as to accurately identify traffic signals.

[0066] Step S102: Determine the traffic light type based on the matching degree between the image features and a plurality of preset traffic light types.

[0067] Any one of the traffic light types is used to indicate the direction of travel. The multiple traffic light types may include: U-turn traffic lights, disc traffic lights, left-turn traffic lights, straight-ahead traffic lights, and right-turn traffic lights.

[0068] It should be understood that the types of traffic lights installed at a target intersection can be related to the actual scenario of the intersection. If the target intersection connects to relatively few road segments, such as two road segments, the traffic flow at the target intersection will be relatively small. In this case, the road conditions at the target intersection are relatively simple, and the traffic lights at the target intersection may only include circular traffic lights. If the target intersection connects to relatively many road segments, such as four road segments, the traffic flow at the target intersection will be relatively large. In this case, the road conditions at the target intersection are relatively complex, and the traffic lights at the target intersection may include U-turn traffic lights, circular traffic lights, left-turn traffic lights, straight-ahead traffic lights, and right-turn traffic lights.

[0069] In this embodiment of the application, each traffic light type among multiple traffic light types can be treated as a query. Then, the features of each query are extracted and injected into the matching algorithm so that the matching algorithm can determine the traffic light type of each traffic light. The matching algorithm can be, for example, an attention algorithm.

[0070] For example, the autonomous driving system can call a pre-deployed attention algorithm to calculate the correlation parameters between the image features of the traffic light and the features of the multiple traffic light types, determine the traffic light type corresponding to the feature with the largest correlation parameter as the reference type of the traffic light, and then determine the traffic light type of the traffic light based on the reference type.

[0071] The correlation parameter can be used to characterize the similarity between image features and traffic light type features. The correlation parameter can be implemented as cosine distance or Euclidean distance, for example.

[0072] In some embodiments, the autonomous driving system can directly determine the reference type as the traffic light type of the traffic light.

[0073] In real-world scenarios, traffic light signals exhibit temporal characteristics. For example, a green light (indicating passage) may remain on for a certain period before counting down or flashing to indicate an impending signal change. In this scenario, based solely on the current traffic light signal, inappropriate driving decisions may be made. For instance, if the green light has been flashing for some time and is about to change, even if the current signal still indicates passage, the impending signal may indicate waiting. Making a driving decision in this situation could pose a safety hazard. Furthermore, autonomous vehicles may encounter situations where their camera's view is obstructed by another vehicle. In such cases, the current traffic light image may not be sufficient to capture the traffic signal.

[0074] In view of this, in some other embodiments, the autonomous driving system can store traffic light attributes for a preset time period at the target intersection. These attributes can be traffic light attributes from several consecutive frames of images. For a detailed description of the traffic light attributes, please refer to the following embodiments; they will not be repeated here. Furthermore, after determining the traffic light type, the autonomous driving system can determine the traffic light type based on the historical attributes within the preset historical time period and the reference type. The preset historical time period refers to a time period adjacent to and earlier than the current time.

[0075] Optionally, the length of the historical period can be between 30 seconds and 3 minutes. Specifically, it can be flexibly set according to the implementation scenario. For example, it can be set to 2 minutes.

[0076] For example, if the reference type of the traffic light is a left-turn traffic light, that is, the reference type of the traffic light at the current moment is a left-turn traffic light, and the historical attributes of the traffic light in the 30 seconds prior to the current moment are all flashing, then the autonomous driving system can determine that the traffic light type of the target intersection is about to switch to a straight-ahead traffic light.

[0077] For example, if the traffic light is partially obscured at the current moment, but the reference type of the traffic light is determined to be a straight-ahead traffic light based on the unobstructed portion, and the historical attributes of the traffic light in the 60 seconds prior to the current moment are, for example, countdowns, then the autonomous driving system can determine that the traffic light type at the target intersection is about to switch to a disc traffic light.

[0078] It is evident that by adopting this implementation method, which uses the historical attributes of traffic lights and takes temporal influencing factors as characteristic factors for identifying traffic signals, the stability of the identified traffic indication signals can be improved, and the signal remains robust even in partially obscured scenarios, thereby improving the recognition performance.

[0079] Step S103: Identify the attributes of the traffic light based on its image features.

[0080] The traffic light attribute set may include: the color of the traffic light, the flashing state of the traffic light, and the countdown state of the traffic light.

[0081] Any traffic light attribute is used to indicate the driving status, such as green indicating that it is permissible to proceed, yellow indicating that one should prepare to stop, red indicating that one should wait, yellow flashing indicating that the vehicle should slow down to pass through the intersection, green flashing indicating that the light is about to change, and so on. This application will not list them all here.

[0082] In some embodiments, each attribute of the traffic light can be treated as a type feature, and a classification layer can be set for each type feature. After determining the traffic light type, the image features of the traffic light can be input into each classification layer, so that each classification layer calculates the similarity between the image features and the corresponding attribute features of that classification layer, thereby determining the attribute of the traffic light. Each classification layer can be implemented, for example, as a fully connected layer.

[0083] For example, the autonomous driving system can call each classification layer to calculate the probability distribution of the image features of the traffic light corresponding to a preset set of traffic light attributes. The probability distribution includes multiple probability values, where each probability value represents the likelihood that the attribute of the traffic light is the attribute of the corresponding classification layer. The higher the probability value, the more likely the traffic light is to possess that attribute. Based on this, the autonomous driving system can determine the traffic light attribute with the highest probability value in the probability distribution as the attribute of the traffic light.

[0084] Step S104: Determine the traffic indication signal for the current moment based on the traffic light type and attributes corresponding to the traffic light.

[0085] Based on the aforementioned meanings of traffic light types and attributes, after determining the traffic light type and attributes at the target intersection at the current time, the autonomous driving system can determine the driving direction indicated by the traffic light based on the traffic light type, and determine the driving status indicated by the traffic light based on the traffic light attributes. For example, if the traffic light type is a left-turn traffic light, the color is green and it is not flashing, it can be determined that the traffic signal at the target intersection at the current time allows vehicles to turn left; as another example, if the traffic light type is a disc traffic light, the color is yellow and it is flashing, it can be determined that the traffic signal at the target intersection at the current time requires vehicles to slow down and pass through the target intersection.

[0086] As can be seen, the technical solution provided in this application embodiment can determine the traffic indication signal at the current time based on the traffic light type and attributes. Clearly, the technical solution in this application embodiment does not rely on high-precision maps, and its implementation is an end-to-end processing process. It does not involve multiple processing stages such as learning various traffic light features in the preprocessing stage or selecting traffic lights in the postprocessing stage. This not only reduces learning costs but also helps avoid error accumulation caused by multiple processing stages, thereby improving the accuracy of recognition and enhancing traffic signal recognition performance to a certain extent.

[0087] The various embodiments of the traffic signal recognition method described above can be executed using a pre-deployed algorithm model. The algorithm models may contain different algorithm networks, and thus, the processing flow of the data involved in this technical solution may differ depending on the algorithm model composed of different algorithm networks.

[0088] The traffic signal recognition method of this application will be described below with reference to an exemplary algorithm model.

[0089] The following example illustrates the traffic signal recognition method of this application.

[0090] refer to Figure 2 , Figure 2 This illustration shows the data flow of a traffic signal recognition method provided in an embodiment of this application. Figure 1 , Figure 2 The illustrated algorithm model includes a convolutional network arranged according to the data flow direction, a 3D position encoding network, an image feature encoding network, a traffic light type recognition network, and a classification layer. The image feature encoding network can be implemented as a transformer encoding network, and the traffic light type recognition network can be implemented as a transformer decoding network.

[0091] For the current frame, the autonomous driving system can utilize the vehicle's three forward-facing cameras to capture images of the target intersection. These three images are taken from different angles of the intersection, and each image may include images of the traffic lights. The autonomous driving system then inputs these three images into a convolutional network to extract 2D image features from each image. This 2D image feature extraction is then fed into a 3D position encoding network. The 3D position encoding network calculates the 3D position of each pixel constituting the traffic light based on each 2D image feature and its corresponding camera parameters (intrinsic and extrinsic parameters), obtaining the 3D image features of the traffic light. These 3D image features are then input into an image feature encoding network. The image feature encoding network encodes the 3D image features and the features of each traffic light type, and then transmits the encoded features to a traffic light type recognition network. The traffic light type recognition network can calculate the correlation parameters between the image features of traffic lights and the features of each traffic light type, and smooth the largest correlation parameter based on the historical traffic light attribute sequence. The smoothed traffic light type feature is the traffic light type of the target intersection. The autonomous driving system can call the classification layer to determine the attributes of the traffic lights at the target intersection. Furthermore, the autonomous driving system can determine the traffic signal at the target intersection based on the traffic light type and its attributes.

[0092] It should be understood that Figure 2 This is merely an illustrative description of the algorithm model for an embodiment of this application and does not constitute a limitation on the technical solutions of the embodiments of this application. In other implementations, the algorithm model may also be implemented in other ways.

[0093] refer to Figure 3 , Figure 3 This illustration shows the data flow of a traffic signal recognition method provided in an embodiment of this application. Figure 2 , Figure 3 The illustrated algorithm model includes a convolutional network arranged according to the data flow direction, an image feature stitching network, a 3D position encoding network, a traffic light type recognition network, and a classification layer. In this example, the image feature stitching network can be implemented as a view transformer network, the 3D position encoding network can be implemented as a transformer encoding network, and the traffic light type recognition network can be implemented as a transformer decoding network.

[0094] use Figure 3The illustrated algorithm model, for the current frame, involves the autonomous driving system using the vehicle's three forward-facing cameras to capture images of the target intersection. A convolutional network is then used to extract the 2D image features from each of these three images. These 2D features are then transmitted to an image feature stitching network, which stitches them together to form the BEV (Battery Electric Vehicle) image features of the target intersection. These BEV image features are then transmitted to a 3D position encoding network. Based on the camera parameters of the three cameras and the BEV image features, the 3D position encoding network calculates the 3D positions of each pixel that makes up the traffic light, obtaining the 3D image features of the traffic light. These 3D image features are then input into a traffic light type recognition network. The roles of the traffic light type recognition network and the classification layer, as well as the data they interact with, can be found in [reference needed]. Figure 2 The relevant descriptions are not elaborated here.

[0095] The technical solution provided in this application embodiment, after acquiring the traffic light image of the target intersection at the current time, extracts the image features of the traffic light image, and then determines the traffic light type and attributes of the traffic light in the image from a preset set of multiple traffic light types. The traffic light type is used to indicate the direction of travel, and the traffic light attributes are used to indicate the travel status. Based on this, this application embodiment can determine the traffic signal at the current time according to the traffic light type and attributes. Clearly, the technical solution of this application embodiment does not rely on high-precision maps, and the implementation of this technical solution is an end-to-end processing process, not involving multiple processing stages such as learning various traffic light features in the preprocessing stage and selecting traffic lights in the postprocessing stage. This not only reduces the learning cost but also helps avoid the accumulation of errors caused by multiple processing stages, thereby improving the accuracy of recognition and the performance of traffic signal recognition to a certain extent.

[0096] This application also provides a traffic signal recognition device for performing the traffic signal recognition method provided in any of the above embodiments. Figure 4 As shown, the device includes: an extraction module 41, a determination module 42, and an identification module 43.

[0097] Extraction module 41 is used to extract image features of the traffic light image in response to the acquisition of the traffic light image of the target intersection at the current time; determination module 42 is used to determine the traffic light type of the traffic light based on the matching degree between the image features and a plurality of preset traffic light types; any traffic light type is used to indicate the direction of travel; recognition module 43 is used to recognize the attributes of the traffic light based on the image features of the traffic light; the attributes of the traffic light are used to indicate the driving status; the determination module 42 is also used to determine the traffic indication signal at the current time based on the traffic light type and attributes corresponding to the traffic light.

[0098] Optionally, the multiple traffic light types include: U-turn traffic lights, disc traffic lights, left-turn traffic lights, straight-ahead traffic lights, and right-turn traffic lights; the traffic light attribute set includes: the color of the traffic light, the flashing state of the traffic light, and the countdown state of the traffic light.

[0099] Optionally, the determining module 42 is further configured to call a pre-deployed attention algorithm to calculate the correlation parameters between the image features of the traffic light and the features of the multiple traffic light types; determine the traffic light type corresponding to the feature with the largest correlation parameter as the reference type of the traffic light; and determine the traffic light type of the traffic light based on the reference type.

[0100] Optionally, the determining module 42 is further configured to determine the reference type as the traffic light type of the traffic light; the determining module 42 is further configured to determine the traffic light type based on the historical attributes of the traffic light within a preset historical period and the reference type, wherein the preset historical period refers to a period that is adjacent to the current time and earlier than the current time.

[0101] Optionally, the recognition module 43 is further configured to calculate the probability distribution of the image features of the traffic light corresponding to a preset set of traffic light attributes, wherein the probability distribution includes multiple probability values, and any probability value represents the probability that the attribute of the traffic light is the corresponding attribute; and to determine the traffic light attribute with the highest probability value in the probability distribution as the attribute of the traffic light.

[0102] Optionally, the traffic signal recognition device further includes: a data acquisition module, used to acquire multiple traffic light images of the target intersection through multiple preset cameras, wherein the multiple cameras correspond one-to-one with the multiple traffic light images.

[0103] Optionally, the extraction module 41 is further configured to extract features from each of the multiple traffic light images to obtain multiple two-dimensional image features; for any two-dimensional image feature, a three-dimensional image feature corresponding to the two-dimensional image feature is generated according to the parameters of the camera corresponding to the two-dimensional image feature, and the multiple three-dimensional image features corresponding one-to-one with the multiple two-dimensional image features are the image features of the traffic light image.

[0104] Optionally, the extraction module 41 is further configured to extract features from each of the multiple traffic light images to obtain multiple two-dimensional image features; stitch the multiple two-dimensional image features together to obtain panoramic image features; and convert the panoramic image features into three-dimensional image features according to the camera parameters of the multiple cameras, wherein the three-dimensional image features are the image features of the traffic light images.

[0105] The traffic signal recognition device and the traffic signal recognition method provided in this application are based on the same inventive concept and have the same beneficial effects as the methods they employ, operate, or implement.

[0106] This application also provides an electronic device for performing the traffic signal recognition method described above. Please refer to... Figure 5 It illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 5 As shown, the electronic device 5 includes: a processor 500, a memory 501, a bus 502, and a communication interface 503. The processor 500, the communication interface 503, and the memory 501 are connected via the bus 502. The memory 501 stores a computer program that can run on the processor 500. When the processor 500 runs the computer program, it executes the traffic signal recognition method provided in any of the foregoing embodiments of this application.

[0107] The memory 501 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 503 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0108] Bus 502 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Memory 501 is used to store programs. After receiving an execution instruction, processor 500 executes the program. The traffic signal recognition method disclosed in any of the foregoing embodiments of this application can be applied to processor 500, or implemented by processor 500.

[0109] The processor 500 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 500 or by instructions in software form. The processor 500 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 501. The processor 500 reads the information in memory 501 and, in conjunction with its hardware, completes the steps of the above method.

[0110] The electronic device provided in this application embodiment and the traffic signal recognition method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0111] This application also provides a computer-readable storage medium corresponding to the traffic signal recognition method provided in the foregoing embodiments. Please refer to... Figure 6 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the traffic signal recognition method provided in any of the foregoing embodiments.

[0112] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0113] The computer-readable storage medium provided in the above embodiments of this application and the traffic signal recognition method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0114] It should be noted that:

[0115] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0116] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0117] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0118] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A traffic signal recognition method characterized by, The method includes: In response to acquiring the traffic light image of the target intersection at the current moment, the image features of the traffic light image are extracted; The traffic light type is determined based on the matching degree between the image features and multiple preset traffic light types; any traffic light type is used to indicate the direction of travel. The attributes of the traffic lights are identified based on their image features; these attributes are used to indicate driving status. Based on the traffic light type and attributes corresponding to the traffic light, determine the traffic indication signal for the current moment.

2. The method according to claim 1, characterized in that, The various traffic light types include: U-turn traffic lights, disc traffic lights, left-turn traffic lights, straight-ahead traffic lights, and right-turn traffic lights; The traffic light attribute set includes: the color of the traffic light, the flashing state of the traffic light, and the countdown state of the traffic light.

3. The method according to claim 1 or 2, characterized in that, The step of determining the traffic light type based on the matching degree between the image features and a plurality of preset traffic light types includes: The pre-deployed attention algorithm is invoked to calculate the correlation parameters between the image features of the traffic lights and the features of the multiple traffic light types; The traffic light type corresponding to the feature with the highest correlation parameter is determined as the reference type of the traffic light; The traffic light type is determined based on the reference type.

4. The method according to claim 3, characterized in that, Determining the traffic light type based on the reference type includes: The reference type is determined as the traffic light type of the traffic light; or, The traffic light type is determined based on the historical attributes within the preset historical time period of the traffic light and the reference type. The preset historical time period refers to the time period that is adjacent to the current time and earlier than the current time.

5. The method according to claim 1 or 2, characterized in that, The step of identifying the attributes of the traffic light based on its image features includes: Calculate the probability distribution of the image features of the traffic light corresponding to a preset set of traffic light attributes. The probability distribution includes multiple probability values, and any probability value represents the probability that the attribute of the traffic light is the corresponding attribute. The traffic light attribute with the highest probability value in the probability distribution is determined as the attribute of the traffic light.

6. The method according to claim 1, characterized in that, Before extracting the image features of the traffic light image, the process also includes: Multiple traffic light images of the target intersection are captured by multiple preset cameras, and each camera corresponds to one of the multiple traffic light images.

7. The method according to claim 6, characterized in that, The extraction of image features from the traffic light image includes: Extract features from each of the multiple traffic light images to obtain multiple two-dimensional image features; For any two-dimensional image feature, a three-dimensional image feature corresponding to the two-dimensional image feature is generated based on the parameters of the camera corresponding to the two-dimensional image feature. The multiple three-dimensional image features corresponding one-to-one with the multiple two-dimensional image features are the image features of the traffic light image.

8. The method according to claim 6, characterized in that, The extraction of image features from the traffic light image includes: Extract features from each of the multiple traffic light images to obtain multiple two-dimensional image features; The multiple two-dimensional image features are stitched together to obtain panoramic image features; The panoramic image features are converted into three-dimensional image features based on the camera parameters of the multiple cameras, and the three-dimensional image features are the image features of the traffic light image.

9. A traffic signal recognition device, characterized in that, The device includes: The extraction module is used to extract image features from the traffic light image of the target intersection in response to the acquisition of the traffic light image at the current time. The determining module is used to determine the traffic light type based on the matching degree between the image features and multiple preset traffic light types; any traffic light type is used to indicate the direction of travel. The recognition module is used to identify the attributes of the traffic light based on its image features; the attributes of the traffic light are used to indicate the driving status. The determining module is further configured to determine the traffic indication signal at the current time based on the traffic light type and attributes corresponding to the traffic light.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-8.

Images